What should a security audit report include?

Question

Background

I'm in charge of auditing a medium-scale web application. I have audited web applications several times before, but I've always wrote a short PDF quickly explaining what I encountered and usually I'm the one who's gonna be fixing those vulnerabilities so I never cared for the actual contents of the report.

In my current job things are done in a way more organized fashion. First I have to write the report, then the project manager will review it, then he'll decide whether I'll be the one to fix the issues or someone else.

Question

What should such report contain? I'm looking for a general outline of how it should be organized.

---

Update: Because I couldn't find anything here on Security.SE about audit reports, I decided to make this question a bit broader and include any kind of security audit rather than just web applications. I think it'll be useful to more people in this case.

Answer 1

There's a couple of ways that I've seen this done, each has it's pros and cons.

As noted by @RoryAlsop below a common point for both approaches is that the executive summary should, as much as possible, be written for a business audience (assuming that it's a test you're doing for a 3rd party or the report will be passed to management).

• Reporting by finding. Here you list the findings, usually ranked by severity (e.g. CVSS score or some other scale like severity/likelihood). You then list out the technical details of the finding and potential mitigations if you have that information. This kind of report gets to the point quite quickly and plays well with tool output.
• Reporting by methodology. Assuming here that you're following a defined testing methodology, the report is structured along the lines of the methodology and includes a section for each area of the review. The sections detail what testing was done and the outcome (e.g. either a finding or the fact that there was no finding in this section). The advantage here is that you're showing your workings and that someone reading the report can get a good idea that you've actually tested something and it was ok rather than you just having missed it out. The downside is that it tends to be a longer report and harder to automate. One other gotcha is that you need to make sure that the testers don't just follow the methodology and they actually engage brain to look for other things.

In terms of format for the findings, I usually include the following

• Title (descriptive gets used in the table and linked to the detail)
• Description - technical description of what the issue is and importantly under what circumstances it is likely to cause a security issue (e.g. for Cross-Site Scripting one of the potential issues is use to grab session tokens which could allow an attacker to get unauthorised access to the application)
• Recommendations - How the issue should be resolved, where possible include specific details on vendor guidance to fix it (e.g. things like removing web server versions from headers have specific instructions for Apache/IIS etc)
• References - Any links to additional information that's relevant to the finding (e.g. links to the relevant OWASP page for Web app. issues)
• Severity. As I mentioned above this could be CVSS or something more general like High/Medium/Low based on impact and likelihood.
• Other classification as needed by the client. For instance some client might need things lined up against a standard or policy or something like OWASP Top 10.

One other point to make is that if your do a lot of tests it's well worth having a database of previous findings to avoid having to look up references repeatedly and to make sure that severities are consistent.

Answer 2

In first: It's like writting a book, first line will keep reader or not. (The Intro is to write at least.)

Something like intro, begin, content, end.

• Introduction
• Goals
  • Refresh terms of contract.
  • Description of targets
  • Description of methods
• Execution
  • Step by step: goal, action, reaction
  • Observations -> questions
  • furter operations, step by step
  • New observations...
  • furter again... and again if in case
• Conclusion
  • success or not
  • what clearly wrong, having to change
  • what fail, having to be corrected
  • purposes

Some trick having to keep in mind:

1. Your work is not intended to be read by technical people, so try to stay simple, but
2. Your work have to be read by technical people, for validation or application of your recommendations, so don't forget anything!! Your work must to be exact, comprehensive and incontestable.
3. As your job concern security failure, making some obfuscation, like using John Doe:XXXXX for couple username, password may be a good practice, they have to be mentionned at intro, but this could be usefull for further discuss.

So for staying light for the administrative people, Introduction and Conclusion have to be explained in simple manner, maybe using pictural language, and

for staying light for technical people who have to work around your text, be exact and detailed. Maybe a simple console dump with some comments could be enough.

Answer 3

I've never written a security audit report, though in my role I tend to receive them. The best one that we had looked over our whole product at specific areas in interest. The report was broken down into those areas. Overall the format was:

1. Title
2. Executive summary - a brief overview of the purpose and scope of the audit. And high level comments, on the main areas of concern, and more importantly cover those areas that are done well
3. Assessment methodology
4. System description - covering the deployment being investigated
5. Then sections on each area/target
6. Summary and recommendations

Section 5 was broken down for each target as:

1. Introduction
2. Objective
3. Significance
4. Assessement (covering methodologies and actual vulnerabilities)
5. Conclusion

Answer 4

I think what Rook says it very true, though this is more about the core of report, rather than its structure, to be placed after the report format has been designed.

Try the STAR model (Situation, Task, Action & Result): I have seen great reports written with this model under the hood. The great thing about it is that it can be used in almost all contexts: you would just need to adjust to what is relevant to you. In this case, you could structure your report around this model and use what Rook described to fill in the structure.

Also, even if you have no actual findings, you could still write a full report based on the STAR model and still deliver something that is professional and coherent.

Answer 5

Adnan,

Exciting question! Too often I feel that our industry strives for the latest and greatest fad in security. We go after the latest exploits, spend serious cash on the latest tools and blame layer 8 for the gaps. I know that is a gross generalization, but I wanted to underscore the importance of this topic -- Reporting!

I have my opinions as to what should be included in a Vulnerability report. From a structure perspective a thorough report will have the following:

• A title page: this will indicate the report name, the agency or department it is for, the date as to when the report was published.

• A table of contents: Seems obvious, but these documents can get lengthy, include this as courtesy.

• An executive summary: This will be a high level summary of the results, what was found and what the bottom line is.

• An introduction: A simple statement of your qualifications, the purpose of the audit and what was in scope.

• Findings: This section will contain your findings and will list the vulnerabilities or issues that should be re-mediate. This listing should be ordered by critical levels, of which are hopefully defined by internal policies (i.e. if your vulnerability scanner finds a high critical vulnerability, based upon how that vulnerability is implemented in your environment, it may not be a true high critical, so internal policies should assist in defining the critical levels)

• Methodologies: Here you will discuss tools used, how false positives were ruled out, what processes completed this audit. This is to provide consistency and allow your audits to be repeatable in the event a finding is disputed or deemed not worthy of fixing by management.

• Conclusion: Basic conclusion, summarize the information you have already put together.

• Appendixes: This will be any extra attachments needed for reference.

Some of the folks over working on the PTES are laying down some good foundations. While the focus there is penetration testing, I would think that many of the methodologies, especially reporting, could be transposed for an audit. You can check them out at: http://www.pentest-standard.org/index.php/Reporting

Answer 6

This may sound like a cop-out but it isn't, and that is to simply ask what format they like. I've performed all sorts of reviews and most companies have a format of what information they like and how they like it. Ask to see previous reports as a template, it will save you loads of time.

Answer 7

After a Penetration Test or Hybrid Application Analysis the resulting report is centered around the findings. There should be a high level overview that discusses the flaws and their collective impact on the system.

A finding is any security violation. This includes any CWE violation, but the most common web application findings fall under the OWASP top 10. Each finding should have steps to reproduce the problem, a severity, the impact of this flaw, recommendations for fixing the issue and links with more information. For example if you find an XSS vulnerability, show a screen cap of an alert box and the URL you used to trigger the issue and link to the OWASP page on XSS. If you can access the cookie with XSS, then the issue can be used to hijack a session, otherwise it could be used to undermine CSRF protection.

What if you can't find anything? Keep looking! The CWE system is massive, and even the most seasoned develops will make mistakes. There are vulnerabilities that affect literately every application I have touched. Clickjacking, lack of brute force protection, and Information disclosure is probably the most common. For example username/email address disclosure via the forgotten password or signup feature. Displaying version numbers (http headers or anywhere else). Verbose error messages, local paths, internal ip addresses.... anything that might be useful to an attacker.