Are you paying too much for your Penetration Test? By, James DeLuccia IV

As a security professional I have spent many an evening evaluating a client’s defenses using myriad of attack vectors and tools. Over time tools have replaced much of the art and the work from the effort. The only component really missing from the field of common knowledge was the craftsmanship in bringing these tools together to provide a thorough and accurate test. I would submit that this final component has been addressed through the brilliant development and documentation provided by the folks at VulnerabilityAssessment.co.uk.

They have published a template that includes nearly every step, tool, configuration, command line, execution instructions, and report formating necessary to deliver a very reasonable penetration engagement. They have the report available via HTML and PDF. I strongly recommend visiting their site, downloading the materials, and comparing their template with your most recent penetration test. As someone in the industry, you will find incredible similarities and a great Thank You should escape your lips as you marvel at the thoroughness. The reason this is so valuable is that conducting “standard” security assessments upwards to penetration assessments requires a very natural path that includes discovery and expoloitation. Of course, the specific vulnerability discovered and the exploit code used will vary, but not so much the tools anymore and certainly not the result. The end result is a checklist that provides a very precise approach that can allow a junior security professional to evaluate any wired enterprise.

Does this excuse the need for a penetration test? NO. It does however, place a bit of responsibility on those who require a penetration test and those who are paying for such tests. The only reason to pay for these types of engagements are two fold. One, the independent 3rd party set of eyes is not as sensitive as those running the network and WILL find items that were considered “business as usual”. This external tester is not bounded by the company culture and the societal side of the organization, and will present information that may have not been reported due to allegiances in the organization. Two, a checklist (even as good as this one) is not an adequate replacement for a truly talented security penetration professional. This checklist only raises the bar, and shines a bright light on what is typically considered a black-box product (money in – report out).

So, are you paying too much for your penetration test? Is your provider merely following these steps and charging you full bill rate for something that has become automated, but charge based on the manual effort it used to take? These are important questions, because we are talking about an evaluation that is intended to rigorously check our environments to proactively eliminate threats. Of course, there is the opposite truth that states you receive what you pay for in the world, and demanding pennies for a service that a senior professional who has spent at least 10 years honing their craft is unlikely to deliver the quality you are seeking.

How can you get a better value on your NEXT penetration test?

  • First, review the last penetration report and find the methodology. Compare this methodology to that provided in the link above.
  • You want to see that your provider’s methodology is more robust with more tools and more checks. For all gaps – highlight them, and skim the report to see if they were done in addition, but separate from the methodology.
  • Well, if you find your report is lacking chalk it up to a learning experience – no sense crying over spilled milk (that is unless you haven’t paid said “service provider”, and haven’t officially accepted the work –> then you can get more accurate information on the work they conducted)
  • Ideally, your report will have at least as much as listed in the checklist with perhaps some good substitutes to the tools and steps implemented. Remember we are not trying to throw darts in their hard work, but instead trying to ensure that everyone comes out better (they get paid for good work, and you get an accurate risk score).
  • Now you want to check who did the work on your report. This is sometimes an overlooked item, and probably THE most critical component. YOU must require the CV / Resume of the individual actually conducting the penetration test from your provider. Get this in writing – i.e. Rocco or Vinny (my dogs – who by the way are not available for consulting work) will conduct the penetration test for X client. The BIO of this individual should be indicative of someone passionate about the technology hosted in your environment. For instance – if you have a complete Microsoft (soup to nuts) implementation, a CV showing strong Unix and Java is worthless, but someone with .NET, Microsoft training experience, and membership with groups that cater towards this type of work is preferred. Essentially you want to vet whomever is doing this work, and this should comply with the rigorous standards you place on anyone touching (or hopefully not touching) your critical applications. The Institute of Internal Auditors (IIA) has a great listing of how you evaluate individuals, but simply – check they do not have criminal backgrounds, are in good standing with peer groups, are current on their security clearances and certifications, and are vouched for by the company.
  • Albeit checking vendors and the service provider personnel is usually done within the Vendor-Approval process, but it is key to consider with these types of engagements, because you know what to look for and they will normally submit their most senior persons for review by the vendor board. Check – Check – and Re-Check, these penetration assessments are VERY important and you should embrace them (both to secure your company’s intellectual property, your customer’s and your own personal sensitive information – unless you want a letter about identity theft, and protecting the jobs of everyone in the organization – too many companies to list that have gone underwater as a result of a penetration), but do so with confidence.
  • As an educated consumer, you now realize that the majority of the work provided on the checklist and potentially 60% of the service provider’s methodology that is to be conducted can be done through automation, and should not cost the rate of the senior level practitioner you have approved. This is where you, the client, requests greater value. Have the practitioner focus their custom scripting and manual checks on the key systems that matter to your organization (i.e. those that represent ingress points, are publicly available, are part of your remote-office architecture, and, of course, your 3rd party outsourced operations). This portion of the effort is only possible if they have open attack vectors (meaning – they are your web server, and you can open a web browser to the server). Said another way – the practitioner can only use their incredible talents if there is something vulnerable, and therefore you only pay for what they actually discover and penetrate. Consider paying a low fee for the discovery, and then allow them to expand it as able, and bill for the normal hourly rate. This places an incentive to the vendor to provide the practitioner enough time. It gives the practitioner a real challenge – not just a fill in the box, but an honest challenge to demonstrate his/her talents. Best of all, you get a very customized product that provides a more realistic measurement of the risk profile.

The above may be coined by service providers as a “Whitebox or Blackbox” service. The simple explanation is that a Blackbox service is conducted with zero to minimal information on the attack targets. This is great if you have secret data centers and you want to see if they can be discovered and possible attacked (such as those found in your remote office of one in Auburn Indiana). A Whitebox (the preferred method and in line with the theme of this article) engagement has the client providing the asset information and even the type of services. This provides the service provider with sufficient information to skip the basic discovery techniques and move straight into the assessment portion of their methodology.

A final point when commissioning a penetration test – these take considerable effort (as listed in the check list) + the custom script development and execution, and therefore tend to cost nearly double a vulnerability assessment (i.e. run nessus hit print). The common response to selecting the sample of devices is to choose those that are most valuable, based upon a risk profile. [Here is where I throw a stone at the “business as usual” concept in security where penetration tests should be locked into the critical assets. My reasoning on this is as follows – you are paying someone to attack your most important assets. (These are probably your most watched, patched, secured, and backed up devices in the entire environment.) The likelihood that these are vulnerable is significantly low. That doesn’t mean a risk based approach to conducting a penetration assessment isn’t valuable – it is extremely valuable, if done appropriately and for the right reasons] Unfortunately as these systems are interconnected, the simplest appliance (Linksys Router) in a remote office may open the same data, as the main firewall / web server. Therefore, it is even more important to have a full penetration assessment on all digital-ingress points using the criteria discussed above. If this is not possible, then request a vulnerability assessment on every public device, and then pull those vulnerable from the list for the penetration test. In this manner the penetration test is really testing your LAYERED security. Consider this concept – if you had a vulnerability assessment and patched everything, and then had a penetration test what would happen? If the scoped devices were those that were found vulnerable were patched the vendor would conduct another vulnerability scan to determine if there were any NEW openings, and then try and exploit those. If they are able to then they will demonstrate how they can hop into your environment via these exposed systems. What is the point? What are you looking to determine? If it is to validate thoroughly that systems are secure and the architecture supports this fact – have a penetration test based on the payment schedule mentioned. Otherwise, consider a different service that addresses your needs.

Tests can be conducted internally also – which are far quicker and better for the client looking to secure all devices. The reason is you bypass the security devices and test at the host level. If the hosts are secure then the security devices are adding a layer of confidence, but if not then the host will most probably be vulnerable from the external segment (especially if we are considering this a patching issue). Yes, to those screaming, it does negate the firewall (add twenty acronyms of additional technology) from stopping everything, but then again isn’t it simpler to secure the single device correctly, and then layer the security to limit breaches (as would be validated in a penetration-beach head building engagement)?

Bottom line – consider the reason for the penetration / assessment, and be sure you are getting these needs addressed. I personally know many talented security professionals that use automated tools for about 10% of their penetration engagements and create on-the-fly code to exploit systems. These are the individuals you want conducting your engagements, and I want you to have them too. Avoid cheap imitations, limit risk, and demand quality.

Best regards,

James DeLuccia IV
A special thanks to my great friend Clement for his kind review of this post – his site contain vast expertise on both Security testing and the leading site for CISSP preparation.

The CISSP and SSCP Open Study Guides Web Site http://www.cccure.org
The Professional Security Testers Warehouse http://www.professionalsecuritytesters.org

Advertisements

49 responses to “Are you paying too much for your Penetration Test? By, James DeLuccia IV

  1. Let’s set aside the issue of you promoting a specific company (because in full disclosure they link to your blog and mine.)

    I have found that examining the report is not a good way to determine if you have paid for a “good” penetration test. Reviewing the report will only tell you if the person used a similar checklist. Also, because most companies only show what they used and what worked vs. what didn’t work (or what they did not find) a simple inspection of their report may not tell much.

    My 10 years of experience has shown me that to know if you are getting a good value from the test is usually by examining the findings. Good penetration testers will almost always find some method of exploitation (as long as the terms of engagement permit all attack vectors.) It is very rare for a skilled tester to not find anything (i.e. social engineering, web application issues, physical security, modem remote access, session hijacking, etc.)

    Even in cases where the network is small and you only have 1 IP for the firewall and 1 IP for the mail server, there are still attacks that will easily provide remote access to the network.

    I agree with you in theory that you should review their methodology and question why they do what they do and what they are missing… but in practice you cannot simply look at the report and compare it against a procedural checklist.

  2. Datasecurity,

    Absolutely true!!! I agree a 100% that additional review of what they found is incredibly valuable. I am merely trying to establish a litmus test to help those responsible for requesting such services to be done with a greater understanding of what should and should not be done.

    I find it also very important that the trend that is blackbox (not penetrating testing, but in fact the a client gives money to a services firm with the pitch of “let our experts take care of it”) service selling has become dangerous in the price war and market share challenge of service firms, because automation and other factors (as outlined in the post) actually degrade the value of the service itself.

    That said, to fully disclose – I am currently independent of ANY services firms and ANY product firm. My passion is for enhancing the information security, risk management, and regulatory compliance space. I only linked to Clement’s site, as he reviewed my post to ensure it was objective and fair. If there are other sites that provide value similar to his I would be proud to add them to this post.

    A sensitive and important topic,

    I thank you for the response and thoughtful additions,

    James

  3. This is such a tough subject!

    I think possibly the biggest problem is one of time.

    If I get one day to do a pen test, and the client is fairly security conscious, then I will probably give a report that says they are secure.

    Give me 6 months and I probably own the entire network or most of it. So it is equally about time and cost as it is about following a good methodology.

    How thorough can you be in a week, a day or a month? What is an accepted risk? These are all things that have to be discussed with the client in advance of a test and after it too.

    For me, a good working methodology and a skilled team are pre-requisites. The most important factor is time/money.

    I know of people who spend in excess of a year to compromise a system without being detected, and a client wants me/you to do it in a week/month!!!

    It’s all about acceptable risk and as far as I can see, most risks are acceptable.

    Thanks for the blog and link to http://www.vulnerabilityassessment.co.uk/index.htm – very useful.

  4. Pingback: Penetration Test Costs – Digital Sniper

  5. “Are you paying too much for your Penetration Test?

    By, James DeLuccia IV | Payment Card Security
    & IT Controls Explained” ended up being a beneficial posting.
    If only there was a whole lot more web blogs like this specific one
    in the actual web. At any rate, thanks for your personal precious time,
    Robbie

  6. I always spent my half an hour to read this webpage’s articles every day along with a cup of coffee.

  7. I am genuinely delighted to read this blog posts which
    includes plenty of helpful information, thanks for providing these information.

  8. Great delivery. Solid arguments. Keep up the good work.

  9. On the pother hand the Linux VPS has its own advantages.

    Joomla web hosting offers options to both the individuals and companies.
    Businessman B also discovers that there are
    many issues his business website is facing especially in terms of performance and usage.

  10. Howdy, i read your blog from time to time and i own a similar one and i was
    just curious if you get a lot of spam remarks?
    If so how do you reduce it, any plugin or anything you can advise?
    I get so much lately it’s driving me insane so any assistance is very much appreciated.

  11. I have learn some excellent stuff here. Certainly price bookmarking for revisiting.
    I wonder how so much effort you put to make this kind of magnificent informative website.

  12. Great beat ! I would like to apprentice while you amend your web
    site, how could i subscribe for a blg web site? The account aided me a acceptable deal.
    I had been tiny bit acquainted off this yoour broadcast provided bright clear concept

  13. I think this is the best and the most informative article on how much we should pay for penetration testing, Great Work

  14. Awesome! Its genuinely remarkable article, I have got much clear idea on the topic of from this paragraph.

  15. You can search for these exact things in Options/About in your iPhone or
    iPod. My three favorite activities are Touch Plantation, Touch
    Retailer, and Tap Bass. Since it has happened with every sequel of iPhone.

  16. Underneath The browse option could be the firmware once it’s identified.
    Jailbreaking an iPhone is really 100% legal. Nonetheless, this way is illegal and contrary to the ruler of the makers.

  17. This check is vital as a result of acceptance of
    graphics computer programs for example games.
    Thus giving off one of the most outstanding screen results that you’ll notice on the
    pill up to now.

  18. So every one of you who jailbreak will get to savor in these
    jailbreak applications. The player must change-up
    their method based on which predators can be found.

  19. I don’t understand this article… You should check http://www.redsn0w.us/ for more
    details

  20. Study on as we share details about this wonderful telephone!
    Huge volumes with cheaper manufacturing expenses and Reduce level of high quality.
    We will be specifically using the firmware edition 6.2.two.

  21. Never be afraid showing photos together with your ex girlfriends!
    For the same purpose, they charge reduced than double-hung windows.
    This very easy step is overlooked by many people. Before carrying this out make sure
    you work step two.

  22. Sometimes, following it is end to update and restarting your computer,
    it has new updates. Unlike WhatsApp, you can sign-up with Kik from your e-mail account.
    Attempt this only following you’ve done the initial tip.

  23. Using ingenious technology and innovation the Romans made an Empire that withstood
    the test of energy. Each auction could be conducted using a
    different list of terms including bid increments,
    variety of auction rounds and expense reimbursement for the stalking horse.

    If you’re still at a loss, it is possible to contact the
    buyer care team either by email, live chat, or phone during standard west coast business hours.

  24. If you are going for finest contents like I do, only pay a quick visit this
    web page daily as it gives feature contents, thanks

  25. Pick the bigger include key in order to complete this portion.
    Once you’ve decided an application in that case
    your alternative will be to purchase it. It is a personalized radio station for
    diehard music lovers.

  26. I am sure this post has touched all the internet visitors, its really really fastidious post
    on building up new web site.

  27. I’m impressed, I must say. Rarely do I encounter a blog that’s both educative and entertaining, and let me tell you, you have hit the
    nail on the head. The issue is something which not enough people are speaking intelligently about.
    I’m very happy that I found this in my search for something relating to this.

  28. We shall see some of the apps you must have,
    in the subsequent checklist. If the Administrator has no password, the
    pc is wide open up to the whims of anybody who logs on. Picasa is a Google service utilized
    to store pictures.

  29. It is about discovering numerous messaging applications to
    remain linked spherical the clock. They have
    since fixed this but it was a large problem.
    The audio results add to the fun quotient of the game.

  30. When you jailbreak you can customize you
    iPhone in whichever way suits you best. Thus, what
    is the difference between between Jailbreaking and Unlocking?

    You can eliminate the applying by visiting ‘Packages’.

  31. They are resilient to any type of tear and wear and also provide less
    problems of staining and washing. It’s going to also show you if your
    friends like the same shows you do. You are cured by rev
    Uninstaller with this alcohol.

  32. This feature enable you to connect to a variety of sites.
    So you see, now you understand how to speed-up Windows
    7! Past update models have gone for $99 to $199 for previous versions of the OS.

  33. For now the college is using an empirical approach to the i – Pad which has generated mixed
    results. Hippie earrings are distinctive,
    handmade from natural components like stones, precious gems and beads.
    Being a parent is a fulfilling and rewarding experience.

  34. You can also unlock your gadget after jailbreak it.
    It has offered millions on millions of its products globally.
    Click on on that software obtain and set up it.

  35. If it is downloaded, then you are prepared to obtain themes and putting in them on your iphone.

    Step 12: On this page, choose the boot logo you want to use.
    You must choose Cydia or your jailbreak will
    be ineffective.

  36. Amazon is 1 business which has come a long way in a extremely brief span of time.
    No specific understanding or abilities are required.

    It turns out; this conduct is not uncommon for Mike.

    It’s functionality that’s looooooong long lasting.

  37. Nu încercați niciodată să facă orice SEO la toate până când veți fi siguri de ceea ce performante .

    Apoi, faceți clic pe butonul Concluzie a închide
    pachetul și a aplica setările .

  38. And there’s nothing illegitimate relating to this.

  39. This program removes plans and purposes from your Windows PC.

    Customers will be able setting their particular picture to the Start display.

    Utilize them correctly and you’ll reap the rewards.

  40. Subsequently, you would convey along your leg on to the floor, accompanied by your knee.
    It is the biblical story about Joseph’s musical retelling, by Andrew Lloyd Webber,.

  41. That’s simply because a few key attributes have been left out.
    Come quickly Apple iphone five, so that we see the gossips
    R.I.P. There is also no signup or setup essential
    to use Picasa if you are operating Android Open up
    Source.

  42. You can connect to a work community to accomplish your work virtually anywhere.
    These recommendations is likely to make it more challenging for your regular computer hacker to invade your privacy.

  43. Another option should be to develop into a tester. So go-ahead and follow the web link, and get your brand-new iPad.

    Scroll down to see the model. There are also several free programs which are good
    fun.

  44. Whilst I typically love to tweet and share updates frequently, this week I’ve
    been in tweet-overload. Apple, becoming the creator of iPod Touch, has limited the
    abilities of iPod Contact. Make sure to continue keeping the House button.

  45. I blog frequently and I truly appreciate your information. This article has
    really peaked my interest. I am going to take a note of your blog and
    keep checking for new details about once a week.
    I subscribed to your RSS feed too.

  46. The issue lies in this specific design of iPod. The power button is
    the one on the top of the phone, that you (duh) use to energy the phone on and off with.
    No particular knowledge or skills are required.

  47. The link may be the topping around the dessert, as far as your dress goes.
    Natalie: What assistance are you experiencing proper
    who wants to get into this marketplace? Cause
    being is your only price is food.

  48. Counteract a nonvolatile solvent with a solvent and counteract
    a solvent with the alkaline soap. Pills are not incredibly unpopular which
    is easy to see why.

  49. I don’t know how/if this relates, but some companies offer free pentests if they don’t get in, like this one for example:
    https://silentbreach.com

    Does it guaranty a good level of service ? It looks like they are confident they can get in..

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s