What a Pentest Requires, Part 3: Severity and Reporting

Once a finding is proven, two things decide whether anyone can act on it: how it was scored, and what the report around it contains. Both are places where a document either survives an audit or falls apart at the first serious question, and both are easy to check before you commit to a vendor.

This is part three of a series on what a penetration test actually requires. Parts one and two covered licensing and scope and methodology, depth and evidence.

Scoring: a label is not a severity

A severity score needs a stated method and enough context for someone else to check it. If the report uses CVSS, it should give the version, the score and the full vector. CVSS v3.1 and v4.0 score differently, so their numbers cannot be compared directly. CWE and OWASP mappings help classify findings, but they do not replace the demonstrated impact.

To check a score yourself, use FIRST's specification for the version the report used: CVSS v3.1 or CVSS v4.0.

A bare label saying "Critical" leaves a reader without the assumptions needed to compare it or challenge the rating. The vector string is what lets someone else re-derive your score and disagree with it in a structured way, which is the point.

The tell that scoring is not being done at all is internal inconsistency. I reviewed a report recently where the same missing server-side CAPTCHA was marked Critical on one asset and Medium on another, inside the same document. An SSRF was marked Critical where the CVSS calculator puts it Medium to High. An exposed metrics endpoint was High while near-identical database metadata leaks were Medium. None of the 49 findings carried a score, a vector or a CWE. That is not scoring. It is software putting labels next to findings with nobody checking the output.

Where severity should actually come from

There is a second failure that looks like the opposite of sloppiness and is just as misleading. A tool or reviewer can assign severity from the vulnerability class alone: see SQL injection, print Critical. That leaves out the impact and constraints of the actual finding.

Rating on confirmed impact produces lower and more variable numbers. Error-based SQL injection that leaks schema detail through verbose error messages is not the same risk as injection that exfiltrates the database, even though both are "SQLi". We rate what was validated, which sometimes means publishing a Low where a competing tool publishes a Critical and looks more impressive for it.

If you are comparing two reports, this is worth understanding before you conclude that the scarier one did more work. Ask for the CVSS vector behind one Critical, chosen at random. The answer tells you immediately which kind of scoring you are looking at.

What the report has to contain

Everything above, the scope, the methodology, the proof and the scores, ends up in one document, and for most clients that document is the only thing they will ever see. OWASP's reporting structure guidance lists what a report should cover: an executive summary, scope, methodology, findings, supporting evidence and remediation. Agree the scoring method with your provider before testing starts.

Beyond the structure, three properties separate an audit-grade report from a printout:

• The finding count holds from the first page to the last. If the cover says 49 and the body contains 33, someone has inflated the headline and hoped nobody counted. Have it corrected before anyone relies on the report.
• The client's name is correct throughout. This sounds too obvious to state. It is not. A report carrying another organisation's name in places is a template that was never properly reviewed, and it tells you how much human attention the rest received.
• A named reviewer stands behind the work. If your assessor has requirements for the reviewer's qualifications, sign-off or independence, raise them before testing starts. A signature on its own does not guarantee the assessor will accept the report.

One assessment, several audiences

A practical point that gets missed: the developer who has to fix the bug and the board member who has to decide whether to fund the fix need very different documents, and squeezing both into one produces something that serves neither.

Agree up front which views you need: a summary for leadership, technical evidence for security, remediation detail for engineering and, if you need one, a shareable statement of the work. Ask to see an example of each. Not every provider or engagement includes all four, or an attestation.

That last distinction matters more than it appears. A full technical report is a sensitive document. Handing it to a prospect's procurement team because they asked for "the pentest report" distributes reproduction steps for live vulnerabilities to people with no reason to have them.

The questions to ask

1. Which severity method and version are used? If it is CVSS, ask for the full vector and the evidence behind its assumptions.
2. Are the relevant classifications explained? A filled-in CWE field is not proof of quality. Check that the CWE and OWASP mappings fit the finding.
3. Count the findings on the cover and in the body. Thirty seconds, and it catches headline inflation.
4. Who signs it, and will they defend it in an audit? A signature that will not stand behind the work is decoration.
5. What can I share externally? If the only artefact is the full technical report, you will end up either over-sharing or unable to answer customer security reviews.

Part four covers quality assurance, remediation that works on your actual stack, and being honest about how the testing was done.