Hackers, both white and black hat, depend considerably on open-source intelligence (OSINT) derived from publicly available information. Security professionals’ knowledge of OSINT collection methods and techniques is crucial for assessing threats.
In this post we use tesla.com in some examples. We chose them solely due to the fact that they are a large, robust company with an open bug bounty program anyone can partake in.
Subdomain Enumeration
Amass
Subdomain enumeration is crucial for getting a full view of an organization's external network perimeter. Amass is a robust OSINT tool that can do more than subdomain enumeration. The tool gets information from diverse sources and has proactive developers who include new techniques to discover data.
Amass categorizes its functionalities into subcommand modules, such as intel, enum, viz, track, and DB. Intel subcommands collect intelligence about a target to determine where to start attacking. Enum module enumerates and maps a target to discover possible attack loopholes. Viz displays results for analysis while track enables a comparison of findings across enumerations to identify changes in the attack surface.
Some of the Amass information-gathering techniques include:
1. Web archives such as LoCArchive, ArchiveToday, and ArchiveIt
2. APIs: AlienVault, BufferOver, Mnemonic, and NetworksDB
3. Certificates: active pulls, CertDB, Entrust, Censys, and CertSpotter
4. Scrapping information from Ask, Baidu, Bing, and Dogpile
5. DNS sources: reverse DNS sweeping, brute force technique, and basic enumerations
I’ll focus on the enum
subcommand as it’s the most useful for discovering additional targets on a penetration test. For a complete list of options available, read the documentation available on GitHub:
https://github.com/OWASP/Amass/blob/master/doc/user_guide.md
To run a quick enumeration, execute amass enum -w <path_to_wordlist> -d target.com
. This will perform DNS enumeration using the specified word list. Useful options include -ipv4
, which will output IPv4 IP addresses and -asn
to search by Autonomous System Numbers (ASN) .
Assetfinder
Assestfinder is another great subdomain enumeration tool. While not as robust as amass, it’s much faster. Assetfinder utilizes a variety of sources, including crt.sh, certspotter, hackertarget, threatcrowd, wayback machine, dns.bufferover.run, facebook, virustotal and findsubdomains to identify subdomains.
Command syntax is straightforward: assetfinder <domain>
Google Dorking
Google Dorking can be used to extract tons of information about an organization.It can be used to search for admin panels, filetypes, keywords, and subdomains.
Looking at Bugcrowd, we see *.tesla.com is in scope, however, there are a few out-of-scope domains. Google has an "exclude", -
, search operator perfect for filtering out keywords from our results.
In this example, we removed “www” because we want to see subdomains, and removed “feedback” because it’s out of scope.
The search results show the subdomains, forums.tesla.com, and appplayer.tesla.com. This method is much slower for subdomain enumeration when compared to automated tools such as amass or assetfinder but can be fine-tuned to look for specific information.
The filetype:
query is also useful in narrowing down the results that are returned. Common usage of the filetype query would be to only show results from PDF or xlsx to look for sensitive information or common/default passwords. The Google Hacking Database maintains a database of several example queries for reference.
Censys
With Censys, you can scan the entire internet and deploy banner grabs and deep protocol handshakes to achieve greater visibility and a precise picture of live targets on the internet. Censys gathers information about IPv4 hosts, websites, and certificates.
Once a user types an IP or domain name the tool returns information, such as the network, routing information, and available protocols (telnet, SSH, and HTTP).
Censys provides additional information on open ports and running services. The screenshot below shows Censys’ search functionality for a host.
Cencys also supports searching certificates. This is useful when performing OSINT on organizations with a minimal internet presence. Searching through certificates will assist in finding assets that automated tools might miss.
DNS Dumpster
DNSDumpster is another great easy tool to perform quick subdomain discovery. DNSDumpster is helpful in that it enumerates DNS servers, MX, TXT, and A records. All of the output can be downloaded into an easy to read and use xlsx document. DNSDumpster also provides a target map that displays the subdomain and domain relationships. Below is an example of a quick DNSDumspter search that maps subdomains to IPs.
Credential Harvesting Tools and Methods
Dehashed
Dehashed is a great tool to search for email addresses, usernames, IP addresses, names, and other information. The tool matches user login credentials with a large repository of publicly available information assets leaked from previous data breaches.
Dehashed has over 24,000 aggregated data wells and more that 12 billion data assets. The tool’s breached assets list is regularly updated and has an API to support custom tool integration.
Hunter
Hunter.io provides an easy way to obtain email addresses, the URL the address was discovered, and supports CSV export functionality. Hunter(dot)io requires a valid account to use and limits free accounts to 50 queries per month. Hunter.io also provides an API which is used in the popular OSINT tool, theHarvester.
Breach-Parse
Breach-parse is an amazing tool for quickly parsing breached passwords lists. While it may not be as up to date as Dehashed, having it available offline and ready anytime makes it essential to have in your toolset.
The tool will output 3 files:
- A master file, containing username: password
- A username file, containing only usernames
- A password file, containing only passwords
This output is excellent for credential stuffing when paired with BurpSuits Intruder feature.
Email Hippo
Email Hippo has a tool to quickly verify an email address. It's not the best for validating a lot of email addresses since it doesn't accept files as input. Another nice feature is the ability to copy results to clipboard, or export to excel/PDF. Keep in mind the free version allows 100 daily email verifications.
Mapping Web-Serves
Aquatone
Aquatone is a tool for visual inspection of websites across a large amount of hosts and is convenient for quickly gaining an overview of HTTP-based attack surface.
Aquatone is great when paired with amass. The amass output, domain names or IP’s can be piped (|
) to Aquatone and it will then scan the targets, identify web servers and create a report to get a visual overview of an organization's web servers. Aquatone groups similar web pages together which makes the report easier to navigate.
Eyewitness
Eyewitness is a better HTTP/S screenshot tool when there's Nmap or Nessus output to work with. Eyewitness has given better and more consistent results using xml output. Eyewitness has a slight advantage over aquatone when it comes to command line options and supports RDP/VNC screenshots as well.
To use eyewitness with xml output run eyewitness -f <full_path_to_xml_file> --web
. It's important to use the full path and not the relative path to the xml file or eyewitness might not be able to locate the file.
Password Spraying/Credential Stuffing
Attacking Login Forms with BurpSuite
With the credentials obtained from public data breaches, we can use BurpSuite to attack all login forms discovered using the techniques outlined above. High-value targets include VPN portals, admin panels, file servers, and Office365.
We'll use on of the subdomains we discovered using amass as an example.
Visiting the files.****.com site, discovered with amass, presents a login portal.
Let’s use BurpSuite to attempt to login using the credentials collected with breach-parse. Configure your browser to work with Burp. Intercept the login attempt and send it to Intruder by right-clicking on the request.
Head over to the intruder tab and clear the preselected payload markers.
Change the attack type to pitchfork and set payload markers on the username and password parameters. We use pitchfork for this scenario because we have username/password pairs.
Switch over to the “Payloads” tab and load the username list obtained from breach-parse in payload set 1, and passwords in payload set 2.
Start the attack and review the responses. Look for alterations in response length and codes. It’s common to see a 301 response, which redirects the user to the home page or a 200 response with a longer response length.
If the credential stuffing attack is unsuccessful, switch the payload type to cluster bomb and select a new password list. Be mindful of account lockout policies in place.
Attacking Office365 with Ruler
If an organization is using O365 with basic authentication enabled, we can use ruler to enumerate valid users, discover weak passwords, and dump the global address list (GAL).
With your curated list of emails and passwords obtained using methods outlined in this post or other popular and well-documented methods such as theHarvester and Prowl, we're ready to begin the attack.
The command go run ruler.go --o365 --domain <target domain> --debug --noencrypt --insecure -b brute --users <path to files of emails one per line> --passwords <path to file of passwords one per line> -v
will perform a brute force attack against the specified domain using the supplied user and password list.
With the compromised account, we can log in to the users O365 account and access resources such as emails and OneDrive files that could contain sensitive information. If the organization supports basic authentication on O365, it's likely their VPN portals lack multi-factor authentication (MFA). By taking advantage of no MFA on VPN portals, an external attacker can easily gain access to the internal network and begin findings a path to become an elevated user.
Alternatively, we can continue attacking O365 with ruler by extracting all email addresses stored in the organization's address book by dumping the GAL with the command. ruler --email user@targetdomain.com abk dump --output <path to output file>
and repeat the brute force attack.
As you can see, OSINT is a critical step in identifying potential attack paths. OSINT heavily influences the amount of information gathered during enumeration and lack of proper OSINT will lead to a poor assessment of an organizations external network.
References
Amass: https://github.com/OWASP/Amass
Assetfinder: https://github.com/tomnomnom/assetfinder
Ruler.go: https://github.com/sensepost/ruler
theHarvester: https://github.com/laramies/theHarvester
Prowl: https://github.com/nettitude/Prowl
Spray.sh: https://github.com/Greenwolf/Spray
TrustedSec Owning O365: https://www.trustedsec.com/blog/owning-o365-through-better-brute-forcing/
All recommendations and case studies presented by Tevora are for information purposes only and not intended to be for the purposes of providing information security advice. Any use of this information for other means is prohibited.