3:26 PM -- Last month, I explained the process I go through when analyzing a new executable file, like an .exe or .pif file, that I suspect is malicious. (See Playing With Malware.) Virustotal is definitely at the top of my list of tools for figuring out the purpose of suspect binary -- it scans the files against 31 different antivirus scanning engines. But the files I submit to Virustotal regularly either hit a generic signature, such as a downloader, or none at all. (A downloader is simply an executable whose sole purpose is to download additional executable files.)

So when signatures aren't enough, behavioral analysis is the next step in dissecting malware. Behavioral analysis focuses on the activity of the suspicious file, such as changes to the filesystem, registry, running processes, and open network ports. If the file is not an executable, however, it's obviously unlikely to exhibit behavior that can be analyzed, so that's why signatures still have usefulness in the world of antivirus products.

I used to run suspect files on machines that were dedicated to doing nothing other than becoming infected. But I have since become spoiled with online malware analysis sites like Anubis, Sunbelt Software's CWSandbox, Norman Sandbox and the new ThreatExpert. There are still times when running this analysis manually is important, such as when there are discrepancies between what the different online services report. But that's not very often.

Knowing just what to look for is important. So what changes to the filesystem, registry, running processes, and network activity are bad exactly? Take a look at the reports from CWSandbox and Anubis. Each one provides details on these types of changes. For example, the suspect file I submitted was from an IRC bot I captured in the wild. It created a new file in C:\WINDOWS\system32\ and executed it. It modified the registry to make sure that the new file would run every time Windows started. And, finally, it joined a command and control IRC server to receive commands from the bot herder.

But beware. These solutions are not always completely accurate, so don't assume they are always correct or provide all the information. And the malware may behave differently because the malware author has included anti-reversing measures to prevent malware analysts from determining the exact behavior and purpose of the malware. Most modern malware also includes capabilities to tell if it's running within a virtual machine and will behave differently.

Make sure you use more than one of the online tools when doing your analysis so you can better detect everything. The reports noted above showed that the bot connected to an IRC server with a password and nickname, but neither report found the actual IRC channel name and channel password. The email I received from Norman Sandbox reported the channel information, but not the IRC server password. That information corroborated with what I saw in the wild.

Next time you're faced with a system on your network that's behaving badly or oddly, take a look at the running processes, any executable files that have changed or been modified around the time the behavior started, and processes with live network connections. If any of them are suspicious, submit them to a couple of the online services listed above. You'll probably then be able to get the information you need to track down additional infected machines in your environment.

If you have any questions or comments, please let me know via the comments link below.

– John H. Sawyer is a security geek on the IT Security Team at the University of Florida. He enjoys taking long war walks on the beach and riding pwnies. When he's not fighting flaming, malware-infested machines or performing autopsies on blitzed boxes, he can usually be found hanging with his family, bouncing a baby on one knee and balancing a laptop on the other. Special to Dark Reading