Tips for an Information Security Analyst/Pentester career - Ep. 73 - Web App recon tools (pt.1)

In this episode, I want to mention some tools for web app recon I often use (dirb, gobuster, dirsearch and nikto).

The best tool for the job is by all means Burp Suite (especially its Pro version), but it's so rich of features and so complete that it deserves a specific consideration, so I'll be talking about it an upcoming post.

Web directory discovery 

When pentesting a web server, we don't know what directories are forming it.

We don't know whether some vulnerable pages are there, or if we might find login pages or any sort of entry points into the system.

Of course, we could browse to each individual directories, which is something any penetration tester needs to do regardless, but there's also an easier way to solve this problem.

We can use some wordlist-based tools, which scan a website for all the possible directory names included in the given wordlists and return any matches they find.

This way, we can often find hidden directories we couldn't access manually.

Of course, when scanning web directories, we want to mostly focus on those returning a 200(found) code.

When a directory returns a 403 code, this usually means only authorized users, logged in with valid credentials, can view it.

dirb 

It's a very simple but powerful tool.

Its syntax is dirb url wordlist.

If we don't indicate any wordlists, dirb uses a default one (/usr/share/dirb/wordlists/common.txt). 

Sometimes, that's not the best wordlist we can use. For example, for some OSCP lab machines, I had to use a larger wordlist (/usr/share/dirb/wordlists/big.txt). 

In the screenshot below, dirb found a /backend directory, which reveals a login page.

GoBuster

It's a tool written in Ruby, very simple but effective.

Its syntax is gobuster dir -u url -w wordlist.

As a wordlist, I often use Dirbuster wordlists, as I found gobuster works much faster.

Dirsearch

It's a Python script requiring both a URL (-u) and the specific page extension (-e) you want to look for (html, PHP, etc.).

It's faster than other tools but not equally accurate.

Metasploit auxiliary http recon modules

Metasploit may provide the same functionalities as the above tools.

As an example, we can use the auxiliary/scanner/http/dir_scanner module.

We can leverage its integration with the exploitation framework to do more with our recon.

Web server vulnerability scanner

Nikto is a very simple but powerful vulnerability scanner for web apps, a little bit like a Nessus equivalent for web apps.

Its syntax is very simple: nikto -h url.

The tool can report both vulnerabilities and directory listings found.

It might sometimes return some interesting findings.

For example, I discovered a specific OSCP lab machine was subject to Shellshock vulnerability, which I could successfully exploit getting a shell.

Nikto can often return false positives, as well.

In this specific case, it found out some interesting directory indexing and an outdated Apache version, subject to multiple vulnerabilities.

Wrap-up

As I often repeat, recon is paramount for pentesting, so these tools can help you get the job done.

I mostly recommend them for small engagements, while you'll be mostly using Burp Suite to get the job done, especially for larger pentests.

I'll talk about Burp in an upcoming post.

Thanks for your attention.