Web Reconnaissance Using Webscour

We have all been there performing a vulnerability assessment or penetration test on a large network and having to analyze data quickly and discover potential security holes.  If you are like me, one of the main tools you use is nmap.  When I start an internal assessment I like to run nmap to quickly identify interesting systems that need closer examination.  Often systems with open ports 80 and 443 are good targets as they may have more exposures than just a file server.  The problem comes when you go through the nmap results and need to manually visit each page to determine if it is something that you need to examine. For example, you would want to examine the web interface to a SCADA system in detail, but not necessarily so a Jet Direct printer without a password. This is where I have found the script webscour.pl (released by Cyberis) very useful.  Much of the information and techniques contained in this post were originally documented by Paul Asadoorian on his site and the guys over at Pentesticles.  I was recently performing an internal vulnerability assessment on a large class A network and ended up discovering over 18,000 active hosts.  As part of the task I wanted to examine those systems that were running a web server, in this case just on the standard ports of 80 and 443.  The first step was to run a normal nmap scan and output it into the .gnmap format.  Then we used a shell script, gnmap-parser, to create host files sorted by port so that we would have separate host files for port 80 and port 443.

This creates a host file with each IP address on a separate line.  We then sorted the data and fed it into webscour.pl using this command:





cat /home/results_of_gnmap_port_80.txt | awk –F:’{print $1}’ | ./webscour.pl /tmp/output.html

The output is a nice HTML file with the header information and a snapshot of the website.

Now you can ignore the boring information and target those systems that might be running the SCADA web interface with default passwords enabled, as seen below.

I really like how the gnmap-parser tool creates the port files which can then be used as a host file.  In addition to feeding the results into webscour.pl they can also be used for quick scans using some of Metasploits’s auxiliary scanners.  By using the techniques illustrated here you can quickly identify security holes in web servers for exploitation and additional analysis.