A penetration test can help validate whether your vulnerability management program is effective or not. It also helps determine whether the remediation steps for regular vulnerability scans are being performed properly.

And more importantly, it can also shed light on how easily or not an outside attacker can gain access to your company’s crown jewels.

But the big question is: Should the pen test be done internally or by a third party? Some companies choose to do it themselves, while others have implemented policies that dictate pen tests must be performed by a third-party firm. If you’re subject to the Payment Card Industry Data Security Standard (PCI DSS), then the costs associated with third-party pen tests could easily become a hard pill to swallow since the PCI DSS requires pen tests to be conducted annually.

The PCI Security Standards Council earlier this week released a supplemental document addressing PCI DSS Requirement 11.3 on penetration testing. It states that the required annual pen test can be performed by a qualified internal resource or by a qualified third party.

But whether you decide to run your own pen test or outsource it, there are several things to consider: when to conduct the test, what information to provide the pen testing team, what they should look for, and how to prioritize remediation.

An obvious concern is the unexpected downtime of a production system, which could lead to loss of productivity, customers, and revenue. So consider what systems will be tested, when they can be tested, and the types of attacks that can be performed. For example, any attack that could cause instability in a critical server application should only be performed on a test system or during a scheduled maintenance window.

Before a pen test begins, arm the testers with a list of contacts for 24-hour support. That way, if a server is accidentally knocked offline at 2 a.m. during a pen test, they can notify the right person to bring the system back online. Also, if any serious vulnerability is discovered that would allow an external attacker access, the issue can be resolved immediately to prevent an actual compromise.

Pen testers should also be on the lookout for any signs of previous or current system breaches. At SANS 2008, I spoke to two professional pen testers who detailed a pen-testing engagement where they stumbled upon several backdoors that had previously been created by an attacker. They had to pause their testing and notify the company of the compromise before they could continue the pen test, which could have potentially destroyed important evidence.

The pen test report, meanwhile, is valuable because it is the lasting part of the pen test. Whether you perform an internal pen test or hire a third party, the report should include an executive overview, a description of how the testing was conducted, and the prioritized results that include the impact to the organization and suggestions for remediation. The report ultimately needs to provide a roadmap for the company to view its affected assets, their vulnerabilities, and the associated impact through successful exploitation. In the end, you need to understand the quantifiable risk to their business based on the findings.

If you’re leaning toward an internal pen test, there are key things to consider: Do you have the staff and resources to dedicate to an annual pen test? Is the team qualified to do pen testing? If the answer is “no” to those two questions, then your best choice is to initially hire a third party to do the pen test this year, hire some additional staff, and get your team trained so they’re ready for next year’s pen test. A third consideration: Can your internal team be unbiased when testing an environment that they are familiar with and may have even have had a hand in building? If not, internal pen testing may not be the best choice.

Even if you do go with internal pen testing, external third-party pen testing should be considered every three to five years as a good routine follow-up and verification to make sure nothing critical has been overlooked by your internal pen testers.

To help in your putting together an internal pen testing team, check out the brand new “SEC 560 – Network Penetration Testing and Ethical Hacking” course written and taught by Ed Skoudis of Intelguardians.

Pen testing comes with an interesting paradox, too. Kevin Johnson, who developed and taught the new SANS “SEC 542 Web Application Penetration Testing In Depth,” said it best: “It’s the only job I know of where the client hopes that I fail.”

Have a comment on this story? Please click "Discuss" below. If you'd like to contact Dark Reading's editors directly, send us a message.