Pen Testing Requirements

[salamed]

Hello,

The PCI seems to be vague about what constitutes an acceptable "penetration test" (11.3). Responses would be appreciated.

Thank you

[admin]

This is a good question and one that is covered in depth on the PCI blog (http://pcianswers.com/2006/11/13/what-should-a-penetration-test-include/). Here's some of the text:

Some people have asked (and others added to the confusion) about what is required by PCI DSS regarding requirement 11.3 requiring an annual penetration test. Here are some answers to those questions?

Who?
The requirement does not specify who must perform the penetration test, but much like other requirements it can be done by virtually any party. The penetration test is typically performed by a third-party with experience in offering these services. There is NO requirement that it must be performed by the QSA or ASV. In fact, it can be performed by the entity themselves as long as they have the necessary skills and cover the appropriate areas (but this is rare.)


The responsibility of the QSA or auditor is to make sure the scope (both depth and breath) is appropriate, the methodology is in line with industry best practices (http://www.isecom.org/osstmm/), and that all high and medium risk vulnerabilities have been remediated and re-checked.

What type?
People ask what needs to be included in the penetration test and some even say that it should include things such as: war-dialing, physical security testing, social engineering (pretexting), and many other things.
PCI DSS v1.1 clearly states that it should address the following areas:
Network-layer penetration tests
Application-layer penetration testsThis means the testing should address all electronic network attacks and electronic application attacks of public facing applications. It does NOT include physical security testing or social engineering (pretexting). One thing it should include is war-dailing (checking modems) because these are methods of electronic remote access.

When?
The requirements call for an annual penetration test. This means that a report on compliance (ROC) cannot be submitted until the testing has been performed, vulnerabilities remediated, and re-checked to make sure they are no longer an issue.


It is not acceptable to include in the report that the penetration test “will be completed within one year.” It needs to be performed and the results addressed before submission of the report.

[adam.muntner]

This is a good question and one that is covered in depth on the PCI blog (http://datasecurity.wordpress.com/2006/11/13/what-should-a-penetration-test-include/). Here's some of the text:

(snip)

In fact, it can be performed by the entity themselves as long as they have the necessary skills and cover the appropriate areas (but this is rare.)



This point can not be repeated enough!

Running NMAP and NESSUS does not a pentest make.

[mdahn]

This is made clear in the SAP v1.1 where it requires:
1) Network-layer testing, and
2) Application-layer testing

[adam.muntner]

This is made clear in the SAP v1.1 where it requires:
1) Network-layer testing, and
2) Application-layer testing

I wrote a more in depth post in the comments of the blog post mentioned in this q/a forum message

http://pcianswers.com/2006/11/13/what-should-a-penetration-test-include/#comment-1294

[jbhall56]

Just to better define a pen test so that QSAs know what to look for in a pen test and the process. FYI This is NOT an endorsement for any particular tool, but tool names are used for illustrative purposes.

Pen testing starts with the results of an nmap, Nessus, Qualys or other vulnerability scanning process. A pen tester uses these results as reconnaissance to define WHAT may be an effective target and HOW they can attack the target.

From there they typically proceed with tools such as Metasploit, Core Impact or SAINTexploit to actually conduct the pen test. Or they may use actual exploits downloaded from the various sources to conduct testing. My preference is for pen testers to use recognized tools versus running whatever they download from wherever just because it is usually a safer process as well as the tools such as Core Impact or SAINTexploit provide reporting similar to Nessus and Internet Scanner and document what the pen tester tried, what worked and what did not work.

So, as a QSA, now that you know the process, as part of your assessment you can reconcile the vulnerability results to what the pen tester tested and make sure that all vulnerabilities that were identified in the vulnerability process were tested as part of the pen test and also make sure that vulnerabilities were false positives or were real and have been addressed.

[adam.muntner]

That's a good description of a network/server pentest, and I'm sure you know this, but it's worth saying that there's a LOT more to it than running some automated tools.

Information gathering, service misconfigurations, scripted automation of command line network clients, and numerous special purpose command line tools in the right hands play a big part. If the only thing a pentester can find and exploit are things that can be had with automated point and click tools, they aren't providing a very good value. I would argue that the highest value holes you can find are the ones that automated tools can't find, because what automated pentest tools can find are typically the kind of thing you can identify with VA scanning. If you can compromise a system or retrieve sensitive data in ways the automated tools can't find... that's the real value of using a pentester vs a VA scanner.

Same goes for web app assessments of course.

[jbhall56]

While I agree that there is more to pen testing than just running Metasploit and similar tools, there is a VERY fine line here and I have serious concerns about some pen testing approaches.

First, while pen testing can be done by anyone, I cannot stress enough that pen testing is an art form and cannot be done by just any geek off of the street. Good pen testers know their craft inside and out. I'm not suggesting that you hire "former" hackers as I think this is a very unsafe practice, but a pen tester should have the knowledge of a hacker.

Next, a pen test still needs to respect a target's systems. I know of a number of competitors that literally go into pen testing with the attitude that they are now justified in "blowing things up." They conduct their testing accordingly and are not satisfied until they have trashed as much of their target's equipment as they can.

Then there are those people that run whatever real, unsanitized exploits they can get off of the Web. First, how do you know it's the real exploit and, second, how do you know you can trust to get it off when you are done? This sort of approach creates all sorts of havoc on the operations end, particularly when the pen tester has no way of cleaning up after themselves. This results in systems with God knows what on them, that have to be rebuilt in order to be considered trusted. All of which takes time away from log reviews and other security monitoring activities in a lot of organizations.

Pen testing needs to be complete, but done VERY carefully and by people that know what they are doing and what the exploits they are using do to their targets. The process also needs to respect the fact that the target has a business to run and cannot to be so disruptive that it's a costly and, therefore, pointless exercise as viewed by management.

[RickB]

Does the pen test, application and network layers, need to be performed externally or also on the internal network?

[jbhall56]

11.3 states that pen testing is required to be performed at least annually, but does not explicitly specify if it should be done externally or internally. However, based on later statements about Web servers being added, it is implied that it is only externally.

That said, there is nothing wrong with conducting an internal pen test against servers that are in the PCI scope. I typically encourage people to conduct annual internal pen tests.

[mdahn]

The penetration test is required for EXTERNAL or INTERNET FACING addresses. It should also be done internally, but PCI only requires the external testing.

[Nerdboy]

We often recommend to our clients that they negotiate a pen-test of their staging environement into their primary pen-test. This should be scheduled first and should give some indications on the holes that exist, or what portion of the pen-test may be harmfull to the produciton servers.

Once those issues have been worked out, then by all means get the full test run against the production networks.