Scoping a critical time where the test team sits down with the client and all involved parties to extract as much information as possible prior to testing. This information can be extremely important in a number of ways including helping you validate the assumptions in the SOW, setting expectations for how the test will proceed, and preventing things such as scope creep, an unhappy client, and even legal trouble.
Scoping answers so many questions regarding an application security test that I like to think of it as the what, when, where, why, and how of an application security testing. Let’s walk through each of these and discuss how to get the most of a scoping session with your client.
- By a domain, IP address, or specific web pages
- By functions – user versus admin sections
- By components – web app, app server, API
- By type – web app, mobile app, thick client app
This is where gathering as much information prior to the test comes in handy. Even before you sit down for the scoping meeting, you should ask for documentation related to the application and its architecture. Request architecture designs, network diagrams, requirement documents, security requirements, and anything else the client is willing to give you. My general rule is the more information the better. I can always sort through it on my own to determine what is important. I have even asked clients to give me access to their information portal in order to obtain all the relevant project documents. Review this information and you’ll be much more prepared for the scoping meeting including knowing what questions to ask.
The scoping meeting is a great time to do an application walkthrough. Getting a hands on tour of the application is absolutely the best way to know what you will be testing. An application walkthrough can help you understand what to test and what to exclude, how the application works, how to navigate tricky workflows, and give you a better understanding of roles in the application.
During the application walkthrough, question the application owners about the application. Ask them about the sensitivity of the data. Ask about attack scenarios they worry about. They may come up with threats that you never thought about that will help shape your testing. It is also a good time to ask any questions you might have from the prior documentation you received. Perhaps the architecture diagram wasn’t clear or they left out information regarding their SSO implementation. If you have invited developers and an architect to the meeting, it is even better for getting your technical questions answered.
The where of scoping defines where the testing will take place. Normally this is a testing environment, but it may vary depending on the client’s needs. You may end up testing different environments such as development, UAT, staging, QA, or even production. You may need to modify your testing and take different precautions based on the environment you are in. For instance, you’ll want to know if there are any impacts to the environment if data or configuration settings are changed. What if other users are in the environment? Your testing might impact their test cases or cause a performance hit for them.
If the environment is hosted by a third party, you’ll want to get a signed release giving you permission to perform testing. Sometimes you may need to sign an NDA with the third party organization in order to get access.
If the environment is production, you’ll want to be extremely wary of the potential consequences. Make sure you discuss all potential outcomes with the client during scoping. For instance, what if there is an accidental denial of service during testing? Will there be an impact to the business if the application goes down? What if the integrity of data is impacted during testing or if configuration settings are changed? What if other systems are affected? You should strongly encourage clients to avoid testing in production, but if they insist, use scoping as a chance to put precautions in place and get all the necessary signoffs.
When defines on the calendar when the testing is going to take place. Specific start and stop times for testing should be determined ahead of time. For one, this sets everyone’s expectations for when the test will begin and end so that the organization can plan appropriately for things such as remediation and production releases. This also allows the client to reach out to other parties who may need to be notified of the test and testing hours. For instance, the intrusion detection team may want to know about the test since testing can set off security alerts. The last thing you want is for the organization going into incident response because of one of your tests. It is also important to have an emergency contact especially if you are testing after hours. If something goes wrong such as a server going down, you want to be able to contact someone immediately.
When also defines the timeframe for deliverables and any updates during testing. Will you be providing daily updates on your progress? Do you need to notify the team if you have a high risk finding? When will the report be due and what is the time frame for the team reviewing it and getting back with you? Setting these expectations during scoping ensures there will be no misunderstandings once the project starts and TCP packets start flying.
The why is something that many testers never think about but which can be useful in terms of tailoring the test and its methodology. The why answers the question of what triggered the test for this application and it helps you tailor a test based on the client’s specific needs. It may be that the client has internal security standards they follow which required a test. Or perhaps the client has an auditing requirement to uphold such as PCI or FISMA. If so, do they test all their applications? Do they prioritize based on risk or data sensitivity? If the application is part of a regular test cycle, has it been tested before? It is always helpful to review a prior test report. For one you make sure you don’t miss anything that the previous tester identified. It also allows you to be more efficient, focusing on areas that the last tester may not have covered so thoroughly.
The how of testing gets into the mechanics of what is needed to conduct the actual test. How will you log into the application? Do you self register, will they provide credentials, or will you have a role added to your LDAP account? What about roles? Will there be multiple roles to test? Typically you want to test at least at least a couple of roles and enough to allow you to test both horizontal and vertical privilege escalation.
The how also determines what kinds of tests you can perform. You need to understand the limits of what you can do. For instance, if you find a file upload vulnerability, are you allowed to upload a web shell and exploit the server? What about denial of service attacks? What about using BeEF to attack another user if you find a cross site scripting vulnerability? Be prepared to give an overview of your methodology and ask questions about what limits exist for your tests.
When scoping is over, everyone associated with the web application security test including the development, project, business, and test teams should have a much greater understanding of what is expected from everyone, what will be covered in the test, and what they outcomes will be. Even though things may change during the test and issues will come up, scoping lays the groundwork for a successful application security test and ultimately makes addressing those issues much easier when they arise.
Good luck and happy scoping!