OSINT Tools and Techniques

OSINT, or Open Source Intelligence, is the act of gathering data from distributed and freely accessible sources. OSINT tools are used to gather and correspond data from the Web. Data is accessible in different structures, including text design, documents, images, etc. The analysis and collection of information from the Internet or other publicly available sources is known as OSINT or Open Source Intelligence. This is a technique used by intelligence and security companies to gather information. This article provides a look at some of the most useful OSINT tools and techniques.

Maltego

Maltego was created by Paterva and is utilized by law enforcement, security experts, and social engineers for gathering and dissecting open-source information. It can gather large amounts of information from various sources and utilize different techniques to produce graphical, easy-to-see outcomes. Maltego provides a transformation library for the exploration of open-source data and represents that data in a graphical format that is suitable for relation analysis and data mining. These changes are inbuilt and can likewise be altered, dependent on necessity.

Maltego is written in Java and works with every operating system. It comes pre-installed in Kali Linux. Maltego is widely used because of its pleasant and easy-to-understand entity-relationship model that represents all the relevant details. The key purpose of this application is to investigate real-world relationships between people, web pages or domains of organizations, networks, and internet infrastructure. The application may also focus on the connection between social media accounts, open-source intelligence APIs, self-hosted Private Data, and Computer Networks Nodes. With integrations from different data partners, Maltego expands its data reach to an incredible extent.

Recon-ng

Recon-ng is a surveillance tool that is identical to Metasploit. If recon-ng is being operated from the command line, you will enter an environment, such as a shell, in which you can configure options and reconfigure and output reports for different report forms. A variety of helpful features are offered by the virtual console of Recon-ng, such as command completion and contextual support. If you want to hack something, use Metasploit. If you want to gather public information, use the Social Engineering Toolkit and Recon-ng to carry out surveillance.

Recon-ng is written in Python, and its independent modules, key list, and other modules are mainly used for data collection. This tool is preloaded with several modules that use online search engines, plugins, and APIs that can assist in collecting the target information. Recon-ng, like cutting and pasting, automates time-consuming OSINT processes. Recon-ng does not suggest that its tools can carry out all OSINT collection, but it can be used to automate many of the more common forms of harvesting, allowing more time for the stuff that still needs to be done manually.

Use the following command to install recon-ng:

To list the available commands, use the help command:

Suppose we need to gather some subdomains of a target. We will use a module named “hacker target” to do so.

Now, the program will gather related information and show all the subdomains of the target set.

Shodan

To find anything on the Internet, especially the Internet of Things (IoT), the optimum search engine is Shodan. While Google and other search engines index search only the Internet, Shodan indexes almost everything, including webcams, water supplies to private jets, medical equipment, traffic lights, power plants, license plate readers, smart TVs, air conditioners, and anything you may think of that is wired into the internet. The greatest benefit of Shodan lies in helping defenders to locate vulnerable machines on their own networks. Let us look at some examples:

• To find Apache servers in Hawaii:
  apache city:”Hawaii”
• To find Cisco devices on a given subnet:
  cisco net:”214.223.147.0/24”

You can find things like webcams, default passwords, routers, traffic lights, and more with simple searches, as it is simpler, clearer, and easier to use.

Google Dorks

Google hacking, or Google dorking, is a hacking tactic that utilizes Google Search and other Google apps to identify security flaws in a website’s configuration and machine code. “Google hacking” involves using specialized Google search engine operators to find unique text strings inside search results.
Let us explore some examples using Google Dork to locate private information on the Internet. There is a way of identifying .LOG files that are unintentionally exposed on the internet. A .LOG file contains clues on what system passwords could be or the different system user or admin accounts that could exist. Upon typing the following command in your Google search box, you will find a list of products with exposed .LOG files before the year 2017:

The following search query will find all the web pages that contain the specified text:

Some other very powerful search operators include the following:

• inurl: Searches for specified terms in the URL.
• filetypes: Searches for specific file types, which can be any file type.
• site: Limits the search to a single site

Spyse

Spyse is a cybersecurity search engine that can be used to quickly find internet assets and conduct external identification. The advantage of Spyse is partly due to its database methodology, which avoids the issue of long scanning times on queries for data collection. With several services operating at the same time, and reports that can take a very long time to return, cybersecurity specialists may know how inefficient scanning may be. This is the main reason why cybersecurity professionals are shifting towards this awesome search engine. The Spyse archive holds over seven billion important data documents that can be downloaded instantly. Using 50 highly functioning servers with data split into 250 shards, consumers can profit from the biggest scalable online database available.

In addition to supplying raw data, this cyberspace search engine also focuses on demonstrating the relationship between various areas of the Internet.

The Harvester

The Harvester is a Python-based utility. Using this program, you can obtain information from numerous public outlets, such as search engines, PGP key servers, and SHODAN device databases, such as addresses, sub-domains, administrators, employee names, port numbers, and flags. If you want to determine what an intruder can see in the company, this instrument is useful. This is the default Kali Linux tool, and you just have to upgrade The Harvester to use it. For installation, issue the following command:

The basic syntax of The Harvester is as follows:

Here, -d is the company name or the domain you want to search, and -b is the data source, such as LinkedIn, Twitter, etc. To search emails, use the following command:

The ability to search for virtual hosts is another fascinating feature of harvester. Through DNS resolution, the application validates whether several hostnames are connected with a certain IP address. This knowledge is very important because the reliability of that IP for a single host relies not just on its level of security but also on how safely the others hosted on the same IP are wired. In fact, if an attacker breaches one of them and gets access to the network server, then the attacker can easily enter every other host.

SpiderFoot

SpiderFoot is a platform used for capturing IPs, domains, email addresses, and other analysis objectives from multiple data outlets, including platforms such as “Shodan” and “Have I Been Pwned,” for Open Source Information and vulnerability detection. SpiderFoot can be used to simplify the OSINT compilation process of finding information about the target by automating the gathering process.

To automate this process, Spiderfoot searches over 100 sources of publicly available information and manages all classified intel from the various sites, email addresses, IP addresses, networking devices, and other sources. Simply specify the goal, pick the modules to run, and Spiderfoot will do the rest for you. For example, Spiderfoot can gather all the data necessary to create a complete profile on a subject you are studying. It is multiplatform, has a cool web interface, and supports almost 100+ modules. Install the Python modules specified below to install spiderFoot:

Creepy

Creepy is an open-sourced intelligence platform for Geolocation. Using various social networking sites and image hosting services, Creepy gathers information about location tracking. Creepy then displays the reports on the map with a search methodology based on the precise location and time. You can later view the files in depth by exporting them in CSV or KML format. Creepy’s source code is available on Github and is written in Python. You can install this awesome tool by visiting the official website:
http://www.geocreepy.com/

There are two main functionalities of Creepy, specified by two specific tabs in the interface: the “mapview” tab and the “targets” tab. This tool is very useful for security personnel. You can easily predict the behavior, routine, hobbies, and interests of your target using Creepy. A small piece of information that you know may not be of much importance, but when you see the complete picture, you can predict the next move of the target.

Jigsaw

Jigsaw is used to obtain knowledge about workers in a company. This platform performs well with large organizations, such as Google, Yahoo, LinkedIn, MSN, Microsoft, etc., where we can easily pick up one of their domain names (say, microsoft.com), and then compile all the emails from their staff in the various divisions of the given company. The only downside is that these requests are launched against the Jigsaw database hosted at jigsaw.com, so we depend solely on the knowledge inside their database that they allow us to explore. You can obtain information about major corporations, but you might be out of luck if you are investigating a less-famous startup company.

Nmap

Nmap, which stands for Network Mapper, is unarguably one of the most prominent and popular social engineering tools. Nmap builds on previous network monitoring tools to provide quick, comprehensive scans of network traffic.

To install nmap, use the following command:

Nmap is available for all the operating systems and comes pre-equipped with Kali. Nmap operates by detecting the hosts and IPs running on a network using IP packets and then examining these packets to include details on the host and IP, as well as the operating systems they are running.

Nmap is used to scan small business networks, enterprise-scale networks, IoT devices and traffic, and connected devices. This would be the first program an attacker would use to attack your website or web application. Nmap is a free and open-source tool used on local and remote hosts for vulnerability analysis and network discovery.

The main features of Nmap include port detection (to make sure you know the potential utilities running on the specific port), Operating System detection, IP info detection (includes Mac addresses and device types), disabling DNS resolution, and host detection. Nmap identifies the active host through a ping scan, i.e., by using the command nmap -sp 192.100.1.1/24, which returns a list of active hosts and assigned IP addresses. The scope and abilities of Nmap are extremely large and varied. The following include some of the commands that can be used for a basic port scan:

For a basic scan, use the following command:

For banner grabbing and service version detection scans, use the following command:

For Operating System detection and aggressive scans, use the following command:

Conclusion

Open Source Intelligence is a useful technique that you can use to find out almost anything on the Web. Having knowledge of OSINT tools is a good thing, as it can have great implications for your professional work. There are some great projects that are using OSINT, such as finding lost people on the Internet. Out of numerous Intelligence sub-categories, Open Source is the most widely used because of its low cost and extremely valuable output.