I was fortunate enough to speak at Bsides Newcastle 2021. After having spend many months running a range of POC's for various security tools, I realized a testing framework would be beneficial.
Looking at the usual POC model, 2-4 weeks on a dev or staging account is great for the deployment/integration process. However, when it comes to security tooling, there are so many variables to make it a viable tool.
For this I would advocate to make a "lab" environment, and depending on the tooling in question run tests against it.
For example, in a POC for an XDR, attack our lab environment, you could script this. However, it could also be a great opportunity to help grown junior members of the team.
Got a few interns or team embers with less experience? Run a Red / Blue exercise on the test lab. Not only are you growing people professionally, you will soon find any shortcomings with the tooling in question
example,
Looking at a XDR platform
Looking at the usual POC model, 2-4 weeks on a dev or staging account is great for the deployment/integration process. However, when it comes to security tooling, there are so many variables to make it a viable tool.
For this I would advocate to make a "lab" environment, and depending on the tooling in question run tests against it.
For example, in a POC for an XDR, attack our lab environment, you could script this. However, it could also be a great opportunity to help grown junior members of the team.
Got a few interns or team embers with less experience? Run a Red / Blue exercise on the test lab. Not only are you growing people professionally, you will soon find any shortcomings with the tooling in question
example,
Looking at a XDR platform
- How does it output alerts?
- Messaging apps (Teams or Slack)
- Time to detection and notification
- Does it create alerts in real time or does it poll once very x number of minutes
- Does it offer remediation steps?
- If so, how accurate / effective are the suggestions.
One of the further benefits of creating a Security Tooling PoC Framework is that if you have a turn-over of staff, you have a well document decisions making process to describe the reason behind the tooling sets.
This can also aid in future convocations regarding value for money, or allows for a fair test against future tools that would look to replace the current solution.