Skip to main content
A wooden card catalog cabinet with one drawer pulled open, drawn in dots

What is detection as code? Pros, cons, and what comes next

Detection as code keeps SIEM rules in Git, tests them in CI, and ships them through peer review. Its real pros and cons, and when it still fits.

Explainer
Updated 8 min read

Definition

Detection as code

Also called Detections as code, DaC, Detection-as-code

Detection as code is the practice of managing detection rules like software: rules live as files in a version-control repository, get tested automatically in CI, go through peer review as pull requests, and reach the SIEM through a deploy pipeline. It gives a SOC an audit trail, rollback, and portable formats such as Sigma. Its costs are drift between the repository and the SIEM console, tests built from hand-picked events instead of real history, and a review queue that every tuning change has to wait in.

Security pros who want to use detection as code often (Splunk)
63%
Security pros who use detection as code today (Splunk)
35%
Community rules in the public Sigma repository
3,000+

How it works

Detection as code (DaC) treats detection rules like application code. Panther defines it as "creating detection rules using programming languages, such as Python, and managing them through version control systems like GitHub." Before DaC, an engineer opened the SIEM console and edited the rule in place, leaving little record of who changed what and why. DaC moves the rule into a repository and puts a pipeline in front of production:

  1. Write the rule as a file. The format depends on the platform. Sigma uses vendor-neutral YAML, Elastic's detection-rules repository uses TOML, Splunk's security_content pairs YAML metadata with SPL, and Panther uses Python functions with YAML metadata.
  2. Test it in CI. Every pull request runs schema validation, a syntax check, and unit tests. A unit test is a set of log events that should fire the rule and events that shouldn't.
  3. Review it. A peer reads the diff and approves or asks for changes.
  4. Deploy on merge. A job pushes the merged rule to the SIEM through its API.

A minimal repository, modeled on Tines' walkthrough of Elastic SIEM with GitHub Actions, looks like this:

rules/
  okta_mfa_fatigue.toml       # rule logic and metadata
tests/
  test_okta_mfa_fatigue.py    # events that should and shouldn't fire
.github/workflows/
  pull-request.yml            # validate and unit test every PR
  deploy.yml                  # push to the SIEM after approval

Many teams also attach documentation to each rule. Palantir's alerting and detection strategy (ADS) framework asks for sections including Goal, Blind Spots and Assumptions, False Positives, Validation, and Response, and calls for peer review before a rule reaches production.

Why it matters for a SOC

The case for DaC is real, and most of it comes from software engineering:

  • An audit trail and rollback. Every change has an author, a diff, and a reason, and a bad change can be reverted. Splunk points out that without this, "changes made through UIs get lost or overwritten with no clear traceability."
  • Tests before production. Panther describes keeping malicious and benign log events as permanent test cases and re-running them whenever the rule changes, instead of discarding them after the first deploy.
  • Peer review. A second engineer catches logic errors and over-broad conditions. Sigma's community rules are "peer reviewed by a community of professional Detection Engineers."
  • Portability and shared content. The Sigma repository holds more than 3000 rules, and sigma-cli converts them into each SIEM's query language. Splunk calls DaC "invaluable during SIEM migrations," and its ESCU app ships 1,800+ detection rules from a public repository.

Demand outruns practice. In Splunk's 2025 survey, 63% of security professionals said they would like to use detection as code frequently or always, but only 35% did so at the time.

The costs, and where it strains

The adoption gap has causes, and most of them have nothing to do with how good Git is:

  • Two sources of truth. The repository and the SIEM can disagree. Microsoft's Sentinel documentation warns that edits made in the workspace get overwritten on the next deploy unless exported back to the repository, and that "deleting content from your repository doesn't delete it from your Microsoft Sentinel workspace." Panther's docs say the same of deprecated rules: "simply removing the content from your repository is not enough."
  • Tests that never see production. Unit tests use crafted events. Panther's docs note that lookup() "does not retrieve live data" in unit tests, so enrichment has to be mocked. A rule can pass CI and still flood the queue on day one, because no test replayed it against last month's real logs. Some platforms add that replay outside the unit-test step: Panther offers historical replay against production logs, and Cotool replays every new detection and every tune against your history before it deploys.
  • Review becomes the bottleneck. Every fix waits for a person. Before Ramp automated tuning, its team found that "noisy rules stayed noisy, because tuning them meant someone finding time to open the repo, work out the fix and ship a PR."
  • Tooling churn. Splunk is "shifting future investment from contentctl to Detection Studio," no longer accepts pull requests to the contentctl repository, and warns that continued use "may require customer-managed customization." Teams that built pipelines around a vendor's DaC tool inherit that tool's roadmap.
  • Skills. DaC asks a SOC to run Git, CI, and code review. Teams of analysts without engineering support often stall at the pipeline step, which may explain much of the gap between 63% and 35%.

What comes next: Cotool's view

Cotool, which publishes this site, argues that classic detection as code is on its way out for teams where agents do the authoring and tuning. This is Cotool's position, not settled industry practice, and it rests on two claims.

Versioning belongs inside the detection system. Cotool's SIEM page describes its detections as "real Python in notebooks, not rules managed as code." Every change published through Cotool's MCP server becomes a new version on the detection's Logic tab, with the app's explanation of the change, and can be reverted. Because the version history lives where the rule runs, there's no repository to drift from the console. The platform can also show what a Git diff can't: how a change differs from the published version on the alerts your team has already labeled.

Human review should give way to an automated gate at some points. Andrej Karpathy's autoresearch shows the pattern: an agent edits one file, a fixed metric it can't change scores each attempt, and only improvements are kept. Cotool applies the same idea to detections. A connected agent tests code over a past window, and on publish Cotool keeps the change only if "it still catches every alert your team labeled a true positive"; otherwise it files a tuning proposal for a person to review. On Cotool's product pages, autotuning is fed by triage dispositions, and a tuning change built from 136 closed false positives cuts a 30-day backtest from 142 alerts to 14 while 6 of 6 true positives still fire. Transparency replaces per-change approval: the backtest result, the metrics, and an audit-log entry for every tool call.

Cotool still supports the Git workflow. Its detection page offers GitOps with detections in your repository, pull-request review, and CI deploys. At Ramp, Cotool's agent opens a pull request for each false positive and "an engineer reviews and merges."

The view has limits. An automated gate is only as good as the labels behind it. A model that climbs against 30 days of history can overfit to that month, and a gate can't protect a true positive nobody labeled.

When detection as code is still the right choice

Keep or adopt classic DaC when one of these is true:

  • Your rules run on more than one SIEM. Sigma in Git, converted per platform, avoids maintaining the same logic twice.
  • Change control requires a named approver. Some regulated teams must show a person signed off on each production change.
  • You already run a mature pipeline. Ramp kept its GitHub repository and added agent-written pull requests on top.
  • You have no labeled dispositions. Without labeled true positives, an automated gate has nothing to check, so human review is the only gate available.
  • You publish or consume community content. Sigma, Elastic, Splunk, and Panther all distribute rules as repositories.

If none of these applies and your platform versions rules natively with backtests, the repository can become a mirror instead of the source of truth.

Examples

ImplementationRule formatHow changes ship
Elastic detection-rulesTOML, with a Python CLIUnit tests in CI, then Kibana
Splunk security_contentYAML metadata with SPLBuilt and tested with contentctl-ng, shipped as ESCU
PantherPython rules, YAML metadataPanther Analysis Tool upload from CI
Microsoft Sentinel RepositoriesAnalytics rules, hunting queries, playbooksGitHub or Azure DevOps deploy
SigmaVendor-neutral YAMLsigma-cli converts per SIEM
CotoolPython detectionsNative versions, or GitOps pull requests

Elastic's repository runs checks every time a rule is added, and Tines' walkthrough reuses Elastic's validation script in its own pull-request workflow. Splunk's case studies include Bitstamp, which moved to Python-based DaC with Panther. Microsoft's Sentinel Repositories require the Owner role on the resource group, and custom detection rules as code are in preview and need a Microsoft 365 E5 or equivalent licence.

  • Detection factory: a continuous loop in which agents write, backtest, and tune detections behind automated gates.
  • AI detection engineering: how agents draft, test, and version rules, with benchmark data on how often their rules work.
  • Backtesting: replaying a rule over past logs to measure alert volume and recall before it ships.
  • Alert fatigue: what untested rules produce; see what is alert fatigue.
  • SIEM migration: where portable DaC pays off most; see legacy SIEM vs modern SIEM.

FAQ

What is detection as code?
Detection as code is managing detection rules the way developers manage software. Rules are files in a Git repository, changes go through pull requests and peer review, CI runs tests on every change, and a pipeline deploys merged rules to the SIEM. The goal is an auditable, testable, reversible history for every rule.
What tools are used for detection as code?
Detection as code usually combines a Git host such as GitHub or GitLab, a CI system such as GitHub Actions, and a vendor's rule format and deploy tooling. Examples include Elastic's detection-rules repository with TOML rules, Panther's Python rules and Panther Analysis Tool, Splunk's security_content repository, and Microsoft Sentinel Repositories. Sigma adds a vendor-neutral YAML format that sigma-cli converts into each SIEM's query language.
Is detection as code the same as Sigma?
Detection as code is not the same as Sigma. Detection as code is the workflow of Git, tests, review, and automated deployment, while Sigma is one vendor-neutral rule format you can manage in that workflow. Many teams practice detection as code entirely in their SIEM's native language and never use Sigma.
Is detection as code going away?
Detection as code is not going away for teams that share rules across SIEMs, need a named approver for each change, or already run mature Git pipelines. What is changing is where versioning and review happen. Some platforms, Cotool among them, version detections inside the detection system and let an agent publish changes that pass an automated backtest gate, with humans reviewing the exceptions.

Sources

  1. What Is Detection as Code (DaC)? Benefits, Tools, and Real-World Use Cases (Splunk) · accessed November 16, 2026
  2. How to create a code-based detection (Panther) · accessed November 16, 2026
  3. Modernize Detection Engineering with Detection-as-Code (Panther) · accessed November 16, 2026
  4. Automating detection-as-code (Tines) · accessed November 16, 2026
  5. elastic/detection-rules (GitHub) · accessed November 16, 2026
  6. SigmaHQ/sigma main rule repository (GitHub) · accessed November 16, 2026
  7. splunk/security_content (GitHub) · accessed November 16, 2026
  8. splunk/contentctl (GitHub) · accessed November 16, 2026
  9. Writing Python detections (Panther docs) · accessed November 16, 2026
  10. Panther detections repo (Panther docs) · accessed November 16, 2026
  11. AI Detection Engineering: Write, Test & Tune Rules (Panther) · accessed November 16, 2026
  12. Manage custom content with Microsoft Sentinel repositories (Microsoft Learn) · accessed November 16, 2026
  13. Palantir Alerting and Detection Strategy Framework · accessed November 16, 2026
  14. karpathy/autoresearch (GitHub) · accessed November 16, 2026
  15. AI-Native SIEM Built for Agents - Cotool · accessed November 16, 2026
  16. AI Detection Engineering with Python Detections - Cotool · accessed November 16, 2026
  17. MCP Server - Cotool Documentation · accessed November 16, 2026
  18. AI SOC Without the Black Box - Cotool · accessed November 16, 2026
  19. How Ramp Scaled Detection & Response With Cotool · accessed November 16, 2026

Keep reading

Attackers are scaling with tokens

Cotool helps defenders operate at machine speed. See how security teams are scaling Detection & Response.

Book a demo