2:50 PM -- While at Microsoft's Bluehat conference and Toorcon's invite-only security conference in Seattle, I had the opportunity to sit down and talk to guys who actually write some of the software for security hardware appliances.

Normally, that's outside of the realm of the things I work on, but intrusion protection systems (IPSes) have a special place in my heart because they are one of the few things that actively mitigate attacks on networks. Think of them as intrusion detection systems tied in with a firewall that proactively make decisions about what to let through or drop/block.

A few things struck me while talking with these guys. Without going into too much depth, there are a few things everyone should know about IPSes before throwing down thousands of dollars. One common question a lot of executives ask: Why can't you just block everything you suspect as possibly bad? The best response I heard is that it's for the same reason you can't make an airplane out of the same material from which you build its black-box flight recorder.

The same principles apply where IPS is concerned. While an airplane constructed from the same ruggedized metal used with black boxes couldn't take off because it would be too heavy, an IPS suffers from similar performance issues -- so a big part of the game is being smart about what you look for.

Another critical thing to understand is what the IPS vendor is trying address with the rules it writes. There is at least one company that writes signatures for all known vulnerabilities. That sounds like a good thing, right? Well, maybe not so much. Lots of vulnerabilities are locally exploitable only. They aren't actually exploitable from the Internet.

So why would you want your IPS to block something on the network level that can't attack the network? In the off chance that maybe the IPS catches the exploit en route to the target, through email perhaps. But that's pretty unlikely and shouldn't be an IPS's job anyway. There are other products for that.

The more likely answer is that vendors do this so they have a marketing announcement every time they write one of these patches. You may think because of my handle (RSnake) I like that kind of snake oil, but I'm having none of it.

Also, lots of these vendors benchmark themselves against an exploit program called Metasploit. If you test the various IPSes with Metasploit exploits with no options turned on, they pretty much all perform the same. However, if you start tweaking those options, you start seeing some pretty big differences, as not all IPSes are made to handle obfuscation techniques. Hackers use those techniques to bypass the IPSes, thus defeating the entire purpose you purchased it for in the first place, turning that reasonably priced IPS into a very expensive pizza box. So it would seem like that would be a good thing to check for, wouldn't it?

Just because you write a signature for an exploit doesn't mean the attacker can't tweak that exploit to evade the signature. That's why it's important to go with a vendor that programmatically understands filter evasion. Buyer beware. If you don't have someone inhouse to test products before you buy them, you might want to consider hiring someone to do unbiased, independent research for you.

— RSnake is a red-blooded lumberjack whose rants can also be found at Ha.ckers and F*the.net. Special to Dark Reading.