Now that you’ve hacked that application to pieces, it’s time to wrap up the project. In this article I look at some of the steps for closing out an application security test and doing it the right way.
This article continues the series on the application security testing process which includes the following articles:
How To Kick Off Application Security Testing – The Right Way
How To Scope A Web Application Security Test
How To Map A Web Application Like A Pro
How To Write An Application Security Finding
How To Create An Awesome Application Security Report
After working your butt off during an application security test, it might be tempting to let up once the hands-on testing is finished. This would be a mistake. There are a number of things you should be thinking about as a tester to properly wrap up the project, even after you have finished writing the report. As with other parts of the application security testing process, missing some of these steps may undermine the hard work you did during testing, cause problems for you in the future such as during a retest, or decrease the likelihood of a client requesting your services down the road.
Review A Checklist
One step I encourage you to take before closing out your testing is to review a checklist to ensure that you have covered all the appropriate tests. Clients are paying significant money to ensure that you provide full coverage of their application and investigate all attack surfaces. After having performed many web application security tests, it can be tempting for a tester to think that they can remember all the required tests. Don’t fall into this trap. Sooner or later you will miss a check and someone will call you out for it. If you miss one check, a client may start questioning your credibility and begin wondering whether you missed other things as well. Even worse is if you miss a check and it ends up being exploited. You don’t want to be that guy in the meeting trying to explain why you missed the check and why your client is being mentioned in the press. There are a ton of well-written checklists available on the Internet and OWASP is a great place to start. Simply cut and paste the list into a spreadsheet and at some point during the test check off each item. Even better, add a short note to each check describing what you did. This will help if you need to revisit a finding and it is something that can benefit a future tester working on the same application.
Review The Previous Report
Another item to review at the end of the test is the previous report. Ideally you should review this report prior to starting testing to make sure you understand issues that were identified. You can leverage the previous report to inform and jumpstart your own testing and it can often save you a lot of time. If the previous tester was able to pull off a complicated attack, you can recreate it much more quickly to confirm whether the issue still exists, without having to spend a lot of time creating it from scratch. You can also improve upon the previous tester’s work by trying attack vectors they may not have thought of or had time to try.
At the same time, it is good to review the previous report one more time to make sure you covered all the previous findings prior to finishing testing. You don’t want to fail to report something that was flagged in the previous report. If a client hasn’t changed the code, they will be expecting to see the same findings from the previous report. Clients are very good at finding inconsistencies especially when they are paying a lot of money for your services. Being called out in the report review meeting for missing a previously reported finding is no fun. Read the previous report and save yourself the discomfort.
Review Scanner Results
Continuing with the theme of not missing vulnerabilities, it is a good idea to review the scan results one more time. This includes reviewing the output from third party scanning tools and from your proxy’s scanner such as Burp Active Scanner. Over the years, scanners have greatly increased the number of findings they report. It can be easy to miss a finding in the list, especially in the low risk and informational findings. For example, Burp lists its SSN finding with a risk rating of informational, most likely because it is very frequently a false positive. However, if your app is leaking SSNs and you miss it because you ignored the informational findings, you are going to have some explaining to do.
Write the report
Never underestimate the importance of the putting together a comprehensive, well-written report tailored for the client. This is the only deliverable you have as a tester and since the client has very little visibility into the actual testing process, the report is the primary way they judge the success of the entire application security project. See this post for some tips on how to create awesome application security report.
Request A Peer Review
Another word of caution during the reporting phase – never issue a report to a client, even in draft form, unless it has been peer reviewed by another person. It is too easy to miss your own errors after you have been staring at the same report for hours. A peer review is good not only from a grammar and spellcheck perspective, but also to get another opinion on things such as whether the right findings were used, the findings were explained clearly, and whether risk levels are appropriate. Before you issue that high risk finding that might keep a client from going live with an important application, get another opinion from a colleague and make sure you are both on the same page before going to battle with the client over the risk level.
Send the Report
Once the report has been peer reviewed and corrections made, send the report to the client using a secure form of communication. Generally I’d recommend not sending reports through email but if you have to, send the report encrypted for an extra layer of security. A portal is great way to deliver reports. The client can be given an account and they can download the report as needed. You can also enforce two-factor authentication to increase security. Not only does this improve security for a sensitive report but also it shows your client that you are serious about security.
Schedule The Review Meeting
When you send the report to the client, indicate that you would like to have a review meeting to walk through the report. Even if you put together the most comprehensive, detailed report possible, it is still likely that the client will have at least a question or two about your work. By offering this meeting, you are being proactive in helping the client get the most out of the report and in the end, they are more likely to take the correct steps in improving the security posture of the application. If the client doesn’t respond, follow up with the project manager and have them schedule a meeting. Have them invite all the relevant parties including the application owner, architect, developers, and business owners.
Host The Review Meeting
The review meeting is your opportunity to present your work to the client. As the tester, you know the report better than anyone so you should be the one to drive the meeting. Use the time to walk through your report, finding by finding. This is your chance to show your knowledge and expertise in security testing but more importantly, to help the client get the most out of your report so that they can improve the security of their application in a cost-effective manner. Give a quick description of each finding and ask the other client team if they understand the issue and more importantly if they are clear on the remediation. The whole point of this project is to fix security issues in the application, so spending time on remediation is a wise investment of time.
Before wrapping up the meeting, remind the client that you’ll be happy to retest any findings they remediate. It is usually better to bundle them into a single retest although it may be possible to quickly test a single finding if it is important to them, for instance to help them go forward with a release. Finally, make sure you thank everyone for his or her support during the test. It usually takes a number of people working together to have a successful project, so this is your chance to recognize the people who made it possible.
Once the report has been delivered and presented, there are still some housekeeping issues you should address before officially closing out the project. The issues here are making sure you securely save all the data generated during testing for future reference and to avoid it being exposed in the case of a lost laptop or your computer being hacked.
All application scanner data should be exported and saved. If you haven’t already, back up the Burp state for the test if you aren’t already using a Burp project. If the client has a question at some point, you may need to go back into your scan results or Burp history to get the answer, especially if the site is no longer available. This data can also be helpful during retesting and for the next test if the n tester has questions.
Finish up any notes or comments in your information file and make sure that credentials and URLs are saved. Also make sure your screenshots are properly named and organized into a folder. Once you have all your data organized including the report, upload it to your company’s internal share and then delete it from your computer. Don’t put your company in the position of having to explain why sensitive data about vulnerabilities in their client’s application was lost or stolen.
There you have it. The test is over and properly wrapped up so now you can kick back, relax, and brag to your coworkers about how you knocked that test out of the park.
Good luck and happy testing!
Project Close Out Checklist:
Review a checklist
Review previous report
Review scanner results
Write the report
Request A Peer review
Securely send the report
Schedule review meeting
Host review meeting
Upload Burp state or project
Upload scan results
Upload project information (file with login information, URL, etc)
Remove data from laptop