How To Spot a Penetration Tester in Your Network (and Catch the Real Bad Guys at the Same Time)

I've been re-reading the Mandiant report on the notorious APT1 group, and it occurred to me that the tools and techniques used by this relatively unsophisticated (but very successful) group are similar to those used by penetration testers. That isn't to say that penetration testers, or pen testers as they are colloquially known, are unsophisticated - the objective for a pen test is to simulate a computer attack. This simulation does not usually include evading detection - in fact, it's usually quite the opposite. Due to limitations of time a tester is easy to detect because they are trying to discover (and perhaps exploit) as many vulnerabilities as possible within a short window of opportunity. The same methodology used by pen testers is often used by some so-called APT groups and script kiddies with little thought given to the trail they are leaving. By understanding how a pen test works you will be able to better detect malicious actors attempting to attack your computer systems.

A pen test is traditionally divided into these broad stages:

• Footprinting and Reconnaissance
• Scanning
• Exploitation
• Privilege Escalation
• Lateral Movement

These serve as a guide (or high-level methodology in our vernacular) - pen testers and malicious actors need to be pragmatic to achieve their goals. They might skip a stage or return to an earlier stage depending on the scenario. The stages described are part of a loose framework and not a rigid step-by-step set of instructions.

It's import to keep in mind that these phases can both occur outside of the network OR inside if an attacker has obtained a foothold via a Remote Access Toolkit (RAT) as part of a client-side campaign.

Lurking in the Shadows

The first stage of a pen test is Footprinting and Reconnaissance. This stage involves accumulating and examining a lot of data about the target network from publicly available sources, such as DNS, WHOIS services, company websites, and social networking sites, such as Facebook, Twitter and LinkedIn. The key here is that the information obtained is free and authorised.

One technique used to obtain information about an organisation is performing a DNS zone transfer in order to obtain a copy of a DNS server's database for a particular domain. This database can give an attacker a wealth of information, including other name servers, host names, MX (Mail eXchange) records, and so on. Although it's uncommon now for DNS servers to allow DNS zone transfers that doesn't stop pen testers, and malicious actors, trying. A request for a DNS zone transfer might be an indicator that someone is in the early stages of performing reconnaissance against your network and or organisation.

DNS zone transfers are easy to detect using the following Snort rules:

alert tcp $EXTERNAL_NET any -> $HOME_NET 53 (msg:"DNS zone transfer TCP"; flow:to_server,established; content:"|00 00 FC|"; offset:15; classtype:attempted-recon; sid:9000000; rev:0;)

alert udp $EXTERNAL_NET any -> $HOME_NET 53 (msg:"DNS zone transfer UDP"; flow:to_server; content:"|00 00 FC|"; offset:14; classtype:attempted-recon; sid:9000001; rev:1;)

Social networking sites are a treasure trove for a pen tester targeting your organisation. Pen testers will profile your employees in order to determine how best to craft a phishing email. Phishing emails attempt to acquire usernames, passwords and financial details by masquerading as a trusted entity. Phishing emails may also contain links to websites infected with malware or malicious attachments. The look and feel of these fake emails are identical to those from the legitimate source.

For example, if an employee is known to have just returned from a conference in Hawaii (because they posted a Facebook update saying what a great time they'd had), then an attacker might craft a phishing email requesting he open an attachment that contains information regarding next year's conference. Of course, the attachment is malicious and upon opening it the employee's system is exploited.

Phishing has proven very popular with APTs because it provides a very high success rate. Verizon's 2013 Data Breach Investigations Report refers to the delivery of malware using phishing emails as the Assured Penetration Technique (recycling the APT acronym) since success is almost guaranteed. The best way to avoid your employee's being exploited in this way is to educate them. Warn them of the dangers of over-sharing on social networking sites and educate them about not clicking on links or opening attachments in unsolicited messages.

Anybody Home?

The next stage is traditionally Scanning. This phase involves taking information obtained in the Reconnaissance and Footprinting stage and determining which systems are listening for network connections.

A popular tool for scanning is Nmap. Nmap can perform ping sweeps, port scans and even basic vulnerability scans. In its default configuration Nmap will send a specially crafted ICMP echo request to determine whether a system is live. This packet differs from a normal ICMP echo request in that it contains no payload. Usually an ICMP echo request will contain a timestamp, a sequence number and a series of alphabetic characters.

The following is a normal ICMP echo request captured using tcpdump:

10:39:16.777274 IP 172.16.150.134 > 172.16.150.2: ICMP echo request, id 30216, seq 1, length 64
  0x0000:  4500 0054 0000 4000 4001 b5ff ac10 9686  E..T..@.@.......
  0x0010:  ac10 9602 0800 2578 7608 0001 944d af51  ......%xv....M.Q
  0x0020:  22dc 0b00 0809 0a0b 0c0d 0e0f 1011 1213  "...............
  0x0030:  1415 1617 1819 1a1b 1c1d 1e1f 2021 2223  .............!"#
  0x0040:  2425 2627 2829 2a2b 2c2d 2e2f 3031 3233  $%&'()*+,-./0123
  0x0050:  3435 3637                                4567

And the following is an ICMP echo request sent using Nmap captured using tcpdump:

10:39:11.831271 IP 172.16.150.134 > 172.16.150.2: ICMP echo request, id 50491, seq 0, length 8
  0x0000:  4500 001c 3444 0000 2c01 d5f3 ac10 9686  E...4D..,.......
  0x0010:  ac10 9602 0800 32c4 c53b 0000            ......2..;..

An ICMP echo request with an empty payload is usually indicative of a Nmap scan and should trigger alarms when detected in your network. The following Snort rule will detect ICMP echo requests sent using Nmap:

alert icmp $EXTERNAL_NET any -> $HOME_NET any (msg:"ICMP ping nmap"; dsize:0; itype:8; classtype:attempted-recon; sid:9000002; rev:1;)

Nmap will also send an ICMP timestamp request to determine whether a given host is live. If you don't expect ICMP timestamp requests on your network then their presence might also be an indicator of an Nmap scan. The following simple Snort rule will detect ICMP timestamp requests:

alert icmp $EXTERNAL_NET any -> $HOME_NET any (msg:"ICMP Timestamp Request"; itype: 13; sid:9000003; rev:1;)

Many organisations block ICMP at their network perimeter in an attempt to reveal any information about systems on their network. To deal with this scenario, Nmap will also send TCP SYN packets to ports 80 and 443. If a TCP port is open then a host will respond with a TCP SYN/ACK packet (or in some rare cases with a TCP SYN packet). If a TCP port is closed then a host should respond with a TCP RST packet. In either case a response is sent indicating that the host is live.

This type of host discovery is stealthy since it never actually completes the TCP three-way handshake. (In the event that TCP ports 80 or 443 are open on the scanned host and it responds with a SYN/ACK packet, the scanner will respond with a RST packet to tear down the connection.) One possible method to detect these kinds of scans is to write IDS signatures to look for TCP SYN packets destined for hosts that you know do not have TCP ports 80 or 443 open. For example, a TCP SYN packet destined for your mail server might be considered unusual behaviour.

The following simple Snort rule will detect a TCP SYN packet destined for ports 80 or 443 on your mail servers:

alert tcp $EXTERNAL_NET any -> $SMTP_SERVERS 80,443 (msg:"Potential nmap SYN scan"; flags:S; sid:9000004; rev:1;)

To really ensure that you are thwarting attempts to scan your network, ensure that your firewall only allows legitimate traffic required to conduct your business, including your internal firewalls. Networks are intended to be porous - not hollow. If an attacker is able to breach your perimeter then having your internal firewalls properly configured can thwart their attempts to gain further access.

I Feel Used

The third stage of a pen test is Exploitation. In this stage any vulnerabilities or weaknesses found during Scanning are actively exploited. Different exploitation techniques are used depending on the vulnerabilities identified. Because the information security landscape is constantly evolving as vulnerabilities are discovered and patched it is difficult to identify one exploit that is indicative of a pen test. A tool often used by pen testers is Metasploit, which contains over 1000 exploits. The goal of exploitation is to abuse the system to achieve some nefarious goal, which is achieved using a payload. There are over 200 payloads supported by Metasploit, many of which can be trivially detected when not obfuscated to avoid detection.

Like pen testers, malicious actors have common toolkits that they use for exploitation. They might amend them to suit their purposes, but they broadly remain the same. For example, variants of the Auriga malware are mentioned in both the Verizon and Mandiant reports. The appendices to the Mandiant APT1 report contains details of many others and how they can be detected.

God Mode

Once a host inside your network has been compromised a pen tester will try to escalate their privileges and move laterally to another host until her goal has been reached, which is usually to obtain system administrator access or access to a particular system. A pen tester will usually perform these steps manually, but a malicious actor would probably automate these steps by utilising backdoor and command and control features of the existing malware to download additional malware. A malicious actor will also try to evade detection, which a pen tester will not tend to do.

Privilege escalation involves elevating access to the host's resources that are normally protected from the user resulting the user being allowed to perform unauthorised actions. This might be accomplished by logging a user's keystrokes, capturing stored credentials, or even dumping password hashes from the domain controller.

A pen tester will not usually look to evade detection, but this step is key to malicious actors to ensure their continued success. This is usually achieved by disabling security controls in place to detect malicious behaviour, such as Anti-Virus (AV) software or Host Intrusion Prevention Systems (HIPS), or unlinking the malicious process from the process list. Since a regular user is not normally authorised to perform these actions privilege escalation is required to complete this step.

A malicious actor will also attempt to hide their presence on the network. Dell SecureWorks discovered the use of a known (and fairly old) program called HTran during their investigation of the RSA breach in 2011. This purpose of this program is to disguise the source or destination of Internet traffic. However, they also found a debugging message generated by the program when an error occurred that betrayed the location of the hidden host.

The following Snort rules were released by SecureWorks to detect these debugging messages, which could indicate the presence of HTran on your network:

alert tcp $EXTERNAL_NET any -> $HOME_NET any (msg:"HTran Connection Redirect Failure Message"; flow:established,from_server; dsize:<80; content:"|5b|SERVER|5d|connection|20|to|20|"; depth:22; reference:url,www.secureworks.com/research/threats/htran/; sid:9000005; rev:1;)

alert tcp $EXTERNAL_NET any -> $HOME_NET any (msg:"HTran Connection Redirect Failure Message (Unicode)"; flow:established,from_server; dsize:<160; content:"|5b00|S|00|E|00|R|00|V|00|E|00|R|005d00|c|00|o|00|n|00|n|00|e|00|c|00|t|00|i|00|o|00|n|002000|t|00|o|002000|"; depth:44; reference:url,www.secureworks.com/research/threats/htran/; sid:9000006; rev:1;)

Pen testers and malicious actors will spread across the network looking to further their access in order to achieve their goals. Hosts are exploited, privileges escalated and detection evaded all the while searching for and capturing the desired data and sending it out of the network.

One way to spread throughout a network is to gain unauthorised access to systems by leveraging someone's authorised access. The Verizon report showed that authentication-based attacks (i.e. guessing, cracking, or reusing valid credentials) factored into about four out of every five breaches involving hacking.

One of the first things that an attacker will try to further their access is to re-use credentials they have already obtained. After all, this is much easier than guessing or cracking passwords, and it's surprising how often it succeeds. The guidance here is not to re-use passwords wherever possible - although this is sometimes easier said than done when managing an enterprise with 1000s of workstations.

Conclusion

This blog post is just the tip of the iceberg. There are many tools and techniques shared by pen testers and malicious actors. Next time you receive a pen test report request guidance on how exploitation of an identified vulnerability can be detected, so you can ring the alarm bell when a real-world attacker does something similar. Also, analyse your logs after the pen test to see what a real attack might look like there. Finally, a packet capture during the test will provide a wealth of information that can be used to discover indicators of an attack and create IDS signatures. Identifying malicious actors in your network is just the beginning - read our blog post on dealing with targeted attacks for a suggested incident response process.

Published date:  20 June 2013

Written by:  Will Alexander