Not all web security assessments provide the same level of assurance.
A Web Vulnerability Assessment (VA) typically focuses on identifying potential security weaknesses, often using automated scanning tools. A Web Application Penetration Test (PT) goes further by investigating whether vulnerabilities can be exploited and assessing their potential impact.
However, some penetration testing services rely heavily on automated scanning, with limited manual testing. So, how can you tell the difference?
1. What Does a Web Vulnerability Assessment Typically Find?
Automated vulnerability scanners are useful for identifying common security weaknesses, including:
- Missing or misconfigured HTTP security headers, such as Content-Security-Policy and Strict-Transport-Security.
- Outdated software, libraries, and components with known vulnerabilities.
- Weak TLS/SSL configurations and insecure cipher suites.
- Missing security flags on cookies, such as Secure and HttpOnly.
- Directory listing enabled on web servers.
- Exposure of server version information and technology banners.
- Known vulnerabilities in web servers, frameworks, and other components.
These findings are useful, but many scanner-generated reports focus on identifying potential weaknesses rather than demonstrating their actual impact on the target application.
For example, a report may state that a software component is outdated or affected by a known vulnerability, then assign it a severity rating. However, it may not demonstrate whether the vulnerability is exploitable in the target environment or explain what an attacker could actually achieve.
Some reports may also assign a High or Medium severity rating without adequately explaining the rationale behind it. For example, a report may simply state that a finding has a CVSS score of 5.8 (Medium), without providing the corresponding CVSS vector string, which identifies the individual metrics used to calculate the score.
A good vulnerability report should provide the CVSS score and its corresponding vector string, where applicable, so that the scoring rationale can be examined. However, the vector alone does not demonstrate exploitability or actual impact in the target environment.
A severity label alone does not establish the actual risk to your organisation. The report should explain the vulnerability’s relevance to the target application and provide appropriate technical evidence to support its assessment.
This is one way to distinguish a scan-heavy assessment from a more in-depth penetration test: does the report merely identify a potential weakness, or does it investigate exploitability and explain the potential impact?
2. What Does a Penetration Test Do Differently?
A penetration test combines automated tools with manual investigation to determine whether security weaknesses can be exploited and what their consequences could be.
For example, a penetration tester may:
- Test access controls: Determine whether one user can access another user’s sensitive information by manipulating request parameters.
- Test authentication: Investigate whether login or authentication controls can be bypassed.
- Test business logic: Determine whether application workflows, transaction limits, or business rules can be circumvented.
- Validate vulnerabilities: Investigate whether a suspected weakness is exploitable in the target environment and establish its potential impact.
- Chain vulnerabilities: Determine whether multiple weaknesses can be combined to achieve a more serious outcome.
The difference is not simply the number of vulnerabilities reported. It is the depth of investigation, the evidence supporting the findings, and the understanding of the risks to the application and its business functions.
A good penetration testing report should explain the vulnerability, provide appropriate supporting evidence, describe its potential impact, justify its risk rating, and offer practical remediation guidance.
3. How Can You Tell Whether You Are Getting a Penetration Test?
When evaluating a provider, ask these questions:
- Does the report mainly list scanner findings, or does it demonstrate that vulnerabilities have been investigated and validated?
- Does the testing examine application-specific risks, such as broken access controls and business logic flaws?
- Does the report explain what an attacker could achieve and the potential impact on the organisation?
- Are severity ratings supported by a CVSS vector string, where applicable, a clear rationale, and appropriate evidence?
- Does the provider describe the manual testing methodology and the areas covered?
A report filled with generic findings about missing headers, outdated software, and configuration weaknesses is not, by itself, evidence of a thorough penetration test. Equally, a short report does not necessarily mean the testing was inadequate.
The methodology and depth of testing matter more than the number of findings.
4. Does a Penetration Test Need to Be 100% Manual?
No.
A well-conducted penetration test can use automated scanners, specialist tools, scripts, and manual techniques. Automation can improve efficiency and coverage, allowing testers to spend more time investigating complex or application-specific issues.
The problem is not the use of automation. The problem arises when automated scanning forms nearly the entire assessment, with little meaningful manual validation or exploitation testing, yet the deliverable is presented as providing the assurance expected from a penetration test.
There is also no universal percentage of manual testing that automatically makes an engagement a penetration test. The appropriate depth depends on the application, scope, complexity, testing objectives, and applicable requirements.
Buyers should therefore evaluate the actual methodology and work performed rather than relying on a provider’s claimed percentage of manual testing.
Looking for a Web Application Penetration Test?
At Perennial Consultancy, we adopt an AI-augmented, human-led testing approach that combines the efficiency of AI with the expertise and judgement of experienced penetration testers. AI enhances our testing process, while human-led validation ensures that reported findings are grounded in technical evidence rather than assumptions or AI-generated speculation.
Our commitment to Zero Hallucination means we focus on verified findings, defensible risk ratings, and evidence-based impact assessments. We go beyond identifying potential vulnerabilities to investigate their exploitability, understand their implications, and provide practical remediation guidance.
