There are three main reasons why companies conduct pen tests.

Pen tests are more comprehensive than vulnerability assessments alone. Penetration tests and vulnerability assessments both help security teams identify weaknesses in apps, devices, and networks. However, these methods serve slightly different purposes, so many organizations use both instead of relying on one or the other. 

Vulnerability assessments are typically recurring, automated scans that search for known vulnerabilities in a system and flag them for review. Security teams use vulnerability assessments to quickly check for common flaws.

Penetration tests go a step further. When pen testers find vulnerabilities, they exploit them in simulated attacks that mimic the behaviors of malicious hackers. This provides the security team with an in-depth understanding of how actual hackers might exploit vulnerabilities to access sensitive data or disrupt operations. Instead of trying to guess what hackers might do, the security team can use this knowledge to design network security controls for real-world cyberthreats.

Because pen testers use both automated and manual processes, they uncover known and unknown vulnerabilities. Because pen testers actively exploit the weaknesses they find, they're less likely to turn up false positives; If they can exploit a flaw, so can cybercriminals. And because penetration testing services are provided by third-party security experts, who approach the systems from the perspective of a hacker, pen tests often uncover flaws that in-house security teams might miss. 

Cybersecurity experts recommend pen testing. Many cybersecurity experts and authorities recommend pen tests as a proactive security measure. For example, in 2021, the U.S. federal government (link resides outside ibm.com) urged companies to use pen tests to defend against growing ransomware attacks. 

Pen testing supports regulatory compliance. Data security regulations such as the Health Insurance Portability and Accountability Act (HIPAA) and the General Data Protection Regulation (GDPR) mandate certain security controls. Penetration tests can help companies prove compliance with these regulations by ensuring their controls work as intended.

Other regulations explicitly require pen tests. The Payment Card Industry Data Security Standard (PCI-DSS), which applies to organizations that process credit cards, specifically calls for regular "external and internal penetration testing" (link resides outside ibm.com).

Pen tests can also support compliance with voluntary information security standards, like ISO/IEC 27001 (link resides outside ibm.com).

All penetration tests involve a simulated attack against a company's computer systems. However, different types of pen tests target different types of enterprise assets.

1. Application pen tests
2. Network pen tests
3. Hardware pen tests
4. Personnel pen tests

Application pen tests

Application pen tests look for vulnerabilities in apps and related systems, including web applications and websites, mobile and IoT apps, cloud apps, and application programming interfaces (APIs).

Pen testers often start by searching for vulnerabilities that are listed in the Open Web Application Security Project (OWASP) Top 10 (link resides outside ibm.com). The OWASP Top 10 is a list of the most critical vulnerabilities in web applications. The list is periodically updated to reflect the changing cybersecurity landscape, but common vulnerabilities include malicious code injections, misconfigurations, and authentication failures. Beyond the OWASP Top 10, application pen tests also look for less common security flaws and vulnerabilities that may be unique to the app at hand.

Network pen tests

Network pen tests attack the company's entire computer network. There are two broad types of network pen tests: external tests and internal tests.

In external tests, pen testers mimic the behavior of external hackers to find security issues in internet-facing assets like servers, routers, websites, and employee computers. These are called “external tests” because pen testers try to break into the network from the outside.

In internal tests, pen testers mimic the behavior of malicious insiders or hackers with stolen credentials. The goal is to uncover vulnerabilities a person might exploit from inside the network—for example, abusing access privileges to steal sensitive data. 

Hardware pen tests

These security tests look for vulnerabilities in devices connected to the network, such as laptops, mobile and IoT devices, and operational technology (OT).

Pen testers may look for software flaws, like an operating system exploit that allows hackers to gain remote access to an endpoint. They may look for physical vulnerabilities, like an improperly secured data center that malicious actors might slip into. The testing team may also assess how hackers might move from a compromised device to other parts of the network.

Personnel pen tests

Personnel pen testing looks for weaknesses in employees' cybersecurity hygiene. Put another way, these security tests assess how vulnerable a company is to social engineering attacks.

Personnel pen testers use phishing, vishing (voice phishing), and smishing (SMS phishing) to trick employees into divulging sensitive information. Personnel pen tests may also evaluate physical office security. For example, pen testers might try to sneak into a building by disguising themselves as delivery people. This method, called "tailgating," is commonly used by real-world criminals.

Before a pen test begins, the testing team and the company set a scope for the test. The scope outlines which systems will be tested, when the testing will happen, and the methods pen testers can use. The scope also determines how much information the pen testers will have ahead of time:

• In a black-box test, pen testers have no information about the target system. They must rely on their own research to develop an attack plan, as a real-world hacker would.
   

• In a white-box test, pen testers have total transparency into the target system. The company shares details like network diagrams, source codes, credentials, and more.
   

• In a gray-box test, pen testers get some information but not much. For example, the company might share IP ranges for network devices, but the pen testers have to probe those IP ranges for vulnerabilities on their own.

With a scope set, testing begins. Pen testers may follow several pen testing methodologies. Common ones include OWASP's application security testing guidelines (link resides outside ibm.com), the Penetration Testing Execution Standard (PTES) (link resides outside ibm.com), and the National Institute of Standards and Technology (NIST) SP 800-115 (link resides outside ibm.com).

Regardless of which methodology a testing team uses, the process usually follows the same overall steps.

1. Reconnaissance

The testing team gathers information on the target system. Pen testers use different recon methods depending on the target. For example, if the target is an app, pen testers might study its source code. If the target is an entire network, pen testers might use a packet analyzer to inspect network traffic flows.

Pen testers often draw on open source intelligence (OSINT) as well. By reading public documentation, news articles, and even employees' social media and GitHub accounts, pen testers can glean valuable information about their targets.

2. Target discovery and development

Pen testers use the knowledge that they gained in the recon step to identify exploitable vulnerabilities in the system. For example, pen testers might use a port scanner like Nmap to look for open ports where they can send malware. For a social engineering pen test, the testing team might develop a fake story, or "pretext," they use in a phishing email to steal employee credentials.

As part of this step, pen testers may check how security features react to intrusions. For example, they might send suspicious traffic to the company's firewall to see what happens. Pen testers will use what they learn to avoid detection during the rest of the test.

3. Exploitation

The testing team begins the actual attack. Pen testers may try a variety of attacks depending on the target system, the vulnerabilities they found, and the scope of the test. Some of the most commonly tested attacks include:

• SQL injections: Pen testers try to get a webpage or app to disclose sensitive data by entering malicious code into input fields.
   

• Cross-site scripting: Pen testers try planting malicious code in a company's website.
   

• Denial-of-service attacks: Pen testers try to take servers, apps, and other network resources offline by flooding them with traffic.
   

• Social engineering: Pen testers use phishing, baiting, pretexting, or other tactics to trick employees into compromising network security.
   

• Brute force attacks: Pen testers try to break into a system by running scripts that generate and test potential passwords until one works.
   

• Man-in-the-middle attacks: Pen testers intercept traffic between two devices or users to steal sensitive information or plant malware.

4. Escalation

Once pen testers have exploited a vulnerability to get a foothold in the system, they try to move around and access even more of it. This phase is sometimes called "vulnerability chaining" because pen testers move from vulnerability to vulnerability to get deeper into the network. For example, they might start by planting a keylogger on an employee's computer. Using that keylogger, they can capture the employee's credentials. Using those credentials, they can access a sensitive database.

At this stage, the pen tester's goal is maintaining access and escalating their privileges while evading security measures. Pen testers do all of this to imitate advanced persistent threats (APTs), which can lurk in a system for weeks, months, or years before they're caught.

5. Cleanup and reporting

At the end of the simulated attack, pen testers clean up any traces they've left behind, like back door trojans they planted or configurations they changed. That way, real-world hackers can't use the pen testers' exploits to breach the network.

Then, the pen testers prepare a report on the attack. The report typically outlines vulnerabilities that they found, exploits they used, details on how they avoided security features, and descriptions of what they did while inside the system. The report may also include specific recommendations on vulnerability remediation. The in-house security team can use this information to strengthen defenses against real-world attacks.

Pen testers use various tools to conduct recon, detect vulnerabilities, and automate key parts of the pen testing process. Some of the most common tools include:

1. Specialized operating systemts
2. Credential-cracking tools
3. Port scanners
4. Vulnerability scanners
5. Packet analyzers
6. Metasploit

Specialized operating systems: Most pen testers use OSs designed for penetration testing and ethical hacking. The most popular is Kali Linux, an open source Linux distribution that comes preinstalled with pen testing tools like Nmap, Wireshark, and Metasploit.

Credential-cracking tools: These programs can uncover passwords by breaking encryptions or launching brute-force attacks, which use bots or scripts to automatically generate and test potential passwords until one works. Examples include Medusa, Hyrda, Hashcat, and John the Ripper.

Port scanners: Port scanners allow pen testers to remotely test devices for open and available ports, which they can use to breach a network. Nmap is the most widely used port scanner, but masscan and ZMap are also common.

Vulnerability scanners: Vulnerability scanning tools search systems for known vulnerabilities, allowing pen testers to quickly find potential entryways into a target. Examples include Nessus, Core Impact, and Netsparker.

Web vulnerability scanners are a subset of vulnerability scanners that assess web applications and websites. Examples include Burp Suite and OWASP's Zed Attack Proxy (ZAP).

Packet analyzers: Packet analyzers, also called packet sniffers, allow pen testers to analyze network traffic by capturing and inspecting packets. Pen testers can figure out where traffic is coming from, where it's going, and — in some cases — what data it contains. Wireshark and tcpdump are among the most commonly used packet analyzers.

Metasploit: Metasploit is a penetration testing framework with a host of functions. Most importantly, Metasploit allows pen testers to automate cyberattacks. Metasploit has a built-in library of prewritten exploit codes and payloads. Pen testers can select an exploit, give it a payload to deliver to the target system, and let Metasploit handle the rest.