Second of a two-part series.

Hackers use some basic techniques to hide their malware from antivirus (AV) software (See the first part of this two-part post). But here I'll explain the more advanced obfuscation methods and showcase new tactics and technologies you can use to detect evasive malware.

1. Antidisassembly and Debuggers (Protectors)
Malware authors know how malware researchers work and what tools they use to hunt for threats. For instance, researchers and programmers often use disassemblers and debuggers to figure out what certain code does. There are many tools and techniques that can detect disassemblers and debuggers, including built-in Windows functions. Many of these techniques were created to help defeat pirates, who might use these tools to help crack protected software. Unfortunately, malware authors can use these techniques to see whether they're running on a malware analyst's computer. If malware detects these tools, they can stop running or change their behaviors to make the analyst's job much more difficult.

2. Rootkits
At the highest level, rootkits are a combination of tools or techniques that allow malware to burrow into a system and hide from your operating system. Computer processors have various privilege levels of execution (ring 0-3), and attackers can exploit these levels of privilege to trick programs that run at higher levels. For instance, operating systems like Windows and Linux have user space and kernel space. At the highest level, you just need to know the kernel space (ring0) has higher privileges than user space (ring3). If you had a program that needed to list the files in a directory, you could call a user space function to do it, but you could also call a kernel function to do it.

If a malicious program can get kernel privileges, it can actually "lie" to programs that run in user space. So if you have a program using a user space function call to scan a file system, a kernel rootkit can trick that program when it parses files. When the user space function gets to malicious file, the rootkit can say, "These are not the files you are looking for" or, more specifically, simply pass over those files and not return them as results to the user space program. To make matters worse, virtualization adds a new layer to rootkit tricks because its hypervisor runs below (with higher privileges than) the kernel.

In short, malware can sometimes use rootkit functionality to hide from local AV, by hiding files, network connections, or other things from the operating system itself. That said, most AV now runs with its own kernel-mode driver and protections to avoid common rootkit tricks.

3. Code, Process, and DLL Injection
Process or dynamic-link library (DLL) injection represents a variety of techniques a program can use to run code under the context of another process. Malware authors often leverage these techniques to get their malware code to run through a necessary and required Windows process. For instance, they might inject explorer.exe, svchost.exe, notepad.exe, or another legitimate Windows executable. By picking a process Windows requires, the malware can make itself more difficult for AV software to find and kill. Malware can also hook processes with known network capabilities to help mask any malicious traffic. Over time, Microsoft has patched many of the process or code injection techniques criminals have exploited, but researchers and attackers constantly find new techniques, such as the recently discovered AtomBombing.

There are just a few of the techniques malware can use to evade AV. Other examples include binding (attaching to a legitimate program) and timing attacks (sleeping to avoid automated analysis), though there are many more.

How can anti-malware software detect or defeat these evasions? Unfortunately, there's no simple answer, and the arms race will continue. However, there is one great weapon in our AV arsenal — behavioral malware detection.

Many of the evasion techniques I've highlighted either involve changing malware code to avoid signature-based detection and static analysis, or performing behaviors that seem obviously malicious in hindsight. While malware may be able to change how it looks, it can't change what it does, at least if it wants to accomplish its goals of infecting your computer, creating a back door, or encrypting your files. So, many advanced detection solutions create a system that recognizes malware based on its behavior.

In general, these solutions create a "sandbox" that acts like a victim's computer, with all the normal accompanying software. When this system receives new and suspicious files, it executes them in these sandbox environments to see what they do. By monitoring for hundreds of known malware behaviors, including known evasion techniques, these solutions can accurately and proactively tell if the executable is malicious. Behavioral analysis (further driven by machine learning) is likely the future of malware defense, and defense in general.

That said, criminals know about sandboxes, and some of the latest evasion tricks specifically target sandboxes. They include trying to fingerprint sandbox systems (using a number of techniques from CPU timing checks to known registry entries), delayed or timed execution, and even detection of human interactions (checking if someone moved the mouse recently to see if a machine is being used by a human or is automated). If the malware can detect a sandbox using these techniques, it doesn't run to avoid analysis.

Furthermore, underground malware sellers have already created protectors that can detect some sandboxes. However, some advanced detection solutions take this into account, too. Rather than just using off-the-shelf virtualization environments, some solutions might use full system code emulation and create sandbox environments where they can see every instruction a malicious program sends to the physical CPU or memory. This true visibility allows the more advanced malware solutions to detect and avoid some of malware's trickier sandbox evasion techniques.

The malware arms race is never-ending, and one of the first steps toward stronger defenses is to continue educating yourself on the latest tricks that threat actors are using. I hope these two posts helped you understand some of the techniques cybercriminals use to smuggle malware past legacy AV solutions and why behavioral analysis has become such a critical component of modern security. 

Related Content:

Learn from the industry’s most knowledgeable CISOs and IT security experts in a setting that is conducive to interaction and conversation. Click for more info and to register.