Introduction
One of our early access testing customers was a financial services company which, due to an "EDR outage incident" a few years ago, decided to move their detection and response in house.
To do this they hired a detection response lead who had previously worked at an MDR company and put together a small team to architect and implement the following:
- Sysmon deployment across endpoints
- A custom SIEM pipeline
- Event storage and detection engine
- An internal platform used by SOC analysts for threat hunting and response workflows
With their pipeline built it was time to populate their internal detection rules repository. A task that turned out to be a little less straight forward than first expected.
The Challenges
Building a detection foundation
Going from a third party vendor with thousands of detections to zero is quite step backwards. There are plenty of open source rules repositories however these couldn't simply be "drag and dropped":
- Open source rules didn't match their rule engine format
- Not knowing which rules are a priority for their "core" initial set
- Difficulty validating which rules were compatible with their current telemetry
Time consuming validation process
Even after a quick sprint implementing the more obvious detections and recreating logic from public rulesets, the team was left with testing to make sure those rules actually worked as intended.
A rule might look correct in theory but without a reliable way to execute the malicious behavior it is difficult to confirm that the rule triggers on the expected activity.
In practice this led to several issues:
- Some rules were written based on assumptions but were never actually validated against real activity
- Attempts at testing often took significantly longer than writing the rules as detection engineers had to find, modify and compile public PoCs
- When new telemetry sources were added or capabilities evolved there wasn't a fast and easy way to retest
As the detection rule library grew, this problem became more significant. Without a rapid and consistent way to execute the behaviors they wanted to detect the team had no reliable method to verify that their detection logic was working as intended.
What the Team Needed
A Detection Validation Tool
Existing tooling for detection validation and emulation does exist but not in the form this group was looking for. They were either large collections of Powershell scripts which didn't represent realistic execution of the malware techniques or tooling designed for red teams which required local infrastructure (a pain to get permission for hosting internally) and lacked the variety of techniques.
What they really wanted was:
- The ability to execute a vast array of malware techniques in isolation
- Execution transparency with clear and detailed logging to help correlate with EDR logs
- Rapid, reliable and repeatable execution
- Known safe payloads to avoid accidental infection
- A checklist of common and obscure techniques to use as a guide
In a market full of automated pentesting, stealth red team end to end frameworks and endless web app dashboards what they really wanted was a local tool built and designed for blue team researcher to get the job done.
Why they chose Combat Theater
Combat Theater on the surface may appear as just a technique execution tool but in practice serves as a multi purpose utility allowing the team to safely execute and validate complex malware techniques without needing to invest significant time into developing custom tooling or PoCs.
Using Combat Theater as a Coverage Checklist
Combat Theater provided a vast library of ready to go malware techniques for multiple categories:
- Injection
- Execution
- Persistence
- Recon
- Defense Evasion
With over 120+ techniques they had a clear list of what is available to attackers without needing to crawl breach analysis papers or online articles to "mine" for TTPs.
Combat Theater's playbooks system also served as a discoverability avenue as techniques were grouped by use for certain criminal groups allowing them to prioritize the types of techniques used by actors targeting their industry.
Executing Techniques Against Their Stack
To accelerate their detection development they put together a sprint plan:
- Each member of the detection engineering team was given a technique category
- They would start executing techniques
- Monitor the telemetry in Sysmon
- Inspect what reached the SIEM
- Build a rule where available or request telemetry to be implemented by the development team
Identifying Telemetry Gaps
Often the team would find that a technique would successfully execute but they would have no visibility for the activity which appeared as:
- Missing or incomplete command line fields
- Missing module load data
- Missing file access events from certain directories
This created a continuous improvement loop where engineers and developers could rapidly improve their capabilities by:
- Reporting missing activity
- Adding Sysmon events / enriching telemetry fields
- Retesting to confirm the fix
Integrating detection testing into workflows
Continuous Validation
As the team's rule repository grew and their capabilities evolved, they discovered that many of their earlier rules could be collapsed into higher fidelity singular rules. With Combat Theater they were able to quickly assess where these possibilities were and streamlined their library.
Results
Faster detection development
By running techniques then building rules the team were able to put together their core rule set in an extremely small time frame.
Improved telemetry visibility
By continuously challenging their detection stack the team were able to rapidly identify weaknesses and prioritise efforts to increase visibility.
Detection coverage efficiencies
By re-testing techniques after stack improvements were made the team were able to optimize their rule library ensuring redundant rules could be rewritten or removed.
A repeatable testing framework
By creating a testing methodology using Combat Theater, continuous improvement and coverage analysis became a simple task rather than a difficult chore.
Conclusion
For this small in-house security team, Combat Theater enabled them to move from writing detections based on assumptions to validating them against real and repeatable behaviour.
Rather than treating detection engineering as a one off task the team adopted a continuous improvement workflow. New telemetry could be introduced and detection logic could be refined all within a consistent and controlled environment.
This shifted their approach from reactive rule writing to proactive validation where coverage could be measured and improvements could be made with confidence.
Combat Theater provided the missing layer by acting as both a technique execution framework and a validation platform while also serving as a practical checklist of behaviours security teams should expect to detect.