Skip to main content
A tripwire strung between two wooden stakes, standing for a hunt turned into a detection that fires on its own

When should a threat hunt become a detection?

A decision rule for turning a threat hunt into a detection: repeatability, backtested false positives, data latency, run cost, and severity, plus when to keep it a hunt.

Guide
Updated 11 min read

Summary

A threat hunt should become a detection when the behavior can recur, the query finds it without a hunter's judgment, a backtest over at least one to two weeks of history returns the known true positives with a false positive volume your team will triage at that severity, and every data source the logic needs arrives within the rule's lookback window. If the query only works with a person reading the results, or the data comes in batches, keep it as a scheduled hunt, report, or dashboard and revisit it later.

Allowed query interval and lookback for a Microsoft Sentinel scheduled rule
5 min-14 days
Historical range Elastic suggests for previewing a new rule
7-14 days
Hunting teams that formally measure hunt outcomes (SANS, 2026)
40%

Key takeaways

  1. Promote a hunt to a rule only when it passes five tests: repeatable, backtested with tolerable false positives, data on time, cheaper to run than to re-hunt, and severe enough to page someone.
  2. When a hunt fails the false positive test, ship it lower in PEAK's hierarchy as a scheduled report or dashboard instead of forcing a noisy rule.
  3. Backtest against the same window the hunt covered, with alerting off, and confirm the hunt's true positives still fire.
  4. Add each source's ingestion delay to the rule's lookback, or the rule will miss late events that the hunt saw.
  5. Track detections created and gaps closed per quarter, and revisit every report and dashboard to see if it can climb to a rule.

Before you start

  • A completed hunt with its queries and findings written down
  • Read access to enough history to backtest, ideally the same window the hunt covered
  • A place to keep detections under version control with review before deploy

Formulas

Expected alerts per day

backtest hits ÷ days in the backtest window

Count every hit, including true positives, then compare it with triage capacity for that severity.

Backtest precision

true positive hits ÷ all backtest hits

Label each hit by hand or from triage verdicts before computing it.

Weekly triage load in hours

expected alerts per day × 7 × minutes per triage ÷ 60

Use your measured triage time for similar alerts, not a guess.

The short version

Turn a hunt into a detection when it passes all five of these tests. If it fails one, the table in the next section says what to ship instead.

  1. Repeatable. The behavior can happen again, and the query finds it without a hunter interpreting the results.
  2. Fidelity. A backtest over the hunt's window, and at least one to two weeks, returns the hunt's true positives and an alert volume your team will triage at that severity.
  3. Data on time. Every source the logic joins arrives within the rule's lookback window, after accounting for ingestion delay.
  4. Run cost. Running the query continuously costs less than re-hunting on the schedule you would otherwise need.
  5. Severity. When it fires, someone should act within hours, not next month.

This is the step David Bianco's Hunting Maturity Model puts at the top level: at HMM4, "any successful hunting process will be operationalized and turned into automated detection." Teams that skip it hit the problem Bianco describes at HMM3, where the pile of hunt procedures grows faster than the analysts available to rerun them. In the SANS 2026 Threat Hunting Survey, the most common workflow already ends with a post-hunt step in which confirmed findings "are either converted into detections or assigned to platform owners for remediation."

The decision rule

Splunk's PEAK framework ranks detection outputs by how much human attention they need, from reports at the bottom to dashboards, analytics in code, and signatures and rules at the top. Its test for each hunt is one question: "When I have a result, would I be confident enough to put it in front of a SOC analyst as an alert?" If yes, pick a rule or analytic; if no, a report or dashboard. Then "choose the highest level that you can feasibly implement." The five tests above make that question concrete:

If the hunt failsShip this insteadRevisit when
Repeatable (one-off campaign or indicator)A report, then close the huntA related campaign appears
Fidelity (too many benign hits)A scheduled report or dashboardTuning or enrichment cuts noise
Data on time (batch or slow source)A scheduled hunt over the batchThe source streams
Run cost (heavy joins, long lookback)A scheduled hunt or daily analyticCost or query design changes
Severity (low impact)A dashboardThe asset or threat changes
None, but needs logic a query can't expressAnalytics in codeYour rule language can express it

Use the formulas on this page to size the fidelity test. Expected alerts per day is backtest hits divided by days in the window, and weekly triage load turns that into analyst hours. For example, Cotool's published example of a detection for AWS access keys created outside Terraform returned 2 hits and 0 false positives in a 30-day test. That works out to a precision of 100% and about 0.07 alerts per day, roughly one every two weeks. Even at the 30 minutes per alert that Ramp reports spending before automation, the weekly triage load is about 14 minutes, so the rule passes the fidelity test easily.

A high-severity behavior, such as credential theft on a domain controller, can justify more false positives than a low-severity one, because a missed true positive costs more.

Step by step: from hunt to deployed detection

  1. Extract the candidate rule. Take the query that separated malicious from benign during the hunt, and write down what the hunter checked by hand. Each manual check either becomes logic in the rule or a reason it can't be one yet. In Microsoft Sentinel's hunting workflow, you can select New alert rule > Create Microsoft Sentinel alert from a hunting query's results to start the analytics rule wizard.

  2. Document it. Palantir's Alerting and Detection Strategy (ADS) framework lists the sections to cover: goal, categorization, strategy abstract, technical context, blind spots and assumptions, false positives, validation, priority, and response. Writing the blind spots and false positives down is what lets the next person tune the rule without rerunning the hunt.

  3. Backtest over history. Run the rule against at least the hunt's window and confirm the hunt's true positives still fire. Elastic suggests 7 to 14 days of history to capture weekday and weekend patterns, and asks you to check for expected true positives, obvious false positives, and missing detections. Google SecOps retrohunts default to one week, and Google warns that a retrohunt on a rule with alerting enabled "creates a SOAR alert for every detection found," which can flood case management; run it with alerting off. Cotool runs the same check as a gate: its coding-agent example tests a new detection over a 30-day window and reports 2 hits and 0 false positives before deploying it.

  4. Tune. Add exclusions for each benign pattern, named and narrow. Exclude one CI role from one IP range, not the whole role, so the rule still fires if that role is abused from somewhere else. Then re-run the backtest. Triage verdicts are the best tuning input: in Ramp's published case study, a Cotool agent asks why a rule fired on benign activity and proposes exactly this kind of narrow exclusion for the CI runner role.

  5. Set the schedule against the data. In Sentinel, a scheduled rule's query interval and lookback each range from 5 minutes to 14 days. If a source arrives late, a rule that looks back only five minutes misses events that land after the run. Microsoft's fix is to widen the lookback by the delay and filter on ingestion time so overlapping windows don't create duplicate alerts:

    let ingestion_delay = 2min;
    let rule_look_back = 5min;
    CommonSecurityLog
    | where TimeGenerated >= ago(ingestion_delay + rule_look_back)
    | where ingestion_time() > ago(rule_look_back)
    
  6. Deploy through review. Keep the rule in version control and merge it through a pull request, as with any detection as code. Shadow-run it, or route its first alerts to the hunter, before it pages anyone. Cotool's GitOps example puts a 30-day backtest in CI, so each pull request shows how alert volume changes and whether any known detections would be missed before a reviewer approves it.

  7. Monitor and retire. Watch the live alert rate against the backtest estimate. A rule that drifts noisy feeds alert fatigue; a rule that never fires may have lost its data source. PEAK counts both new and updated detections as hunt outcomes, alongside gaps identified and closed.

When a hunt should stay a scheduled hunt

Some hunts should never become alerts, and forcing them creates noise. Keep a hunt on a schedule when:

  • A person's judgment does the finding. Baseline hunts that look for outliers often return mostly benign deviations. PEAK recommends a scheduled report when finding true malicious intent "may require analyst experience and insight," and says to revisit reports and dashboards every so often to see if they can climb.
  • The data arrives in batches. If a source lands once a day or once a week, a near-real-time rule adds nothing over a hunt run after each batch. When ingestion is the only blocker, querying the source tool directly is an alternative; Cotool's federated detections do this on a schedule against SaaS APIs.
  • The query is expensive. Joins across several large sources over long windows can cost more to run every few minutes than the behavior's severity justifies. A weekly or monthly run often catches the same thing.
  • The behavior is rare and low severity, such as a periodic review of persistence entries on servers.

Write down why the hunt stayed a hunt, and the condition that would change the answer, so the decision gets revisited instead of forgotten.

How Cotool closes the loop

Cotool's hunting and detection products are built around this hand-off. On the hunt page, agents form hypotheses from a threat model of the environment and from threat intel, and "when a hunt returns findings, the agent backtests and ships a new detection." The example hunt, a stolen GitHub token used to clone private repositories from an unmanaged device, ends with a detection for mass repository cloning mapped to ATT&CK T1213 and marked for review. The detection page describes the output as detection suggestions "ready for your review."

Each test in the decision rule has a matching mechanism:

  • Fidelity: detections are Python, and every alert's triage verdict feeds back into the detection that fired it, with cotool backtest --days 30 run in CI before a change merges.
  • Data on time: federated detections run "directly against the tools where your data already lives," so a source doesn't have to be ingested before a rule can cover it. The example joins Rippling and Okta to find terminated employees still active.
  • Repeatability and gaps: Cotool maps the detection library onto MITRE ATT&CK and prioritizes uncovered techniques, so hunts can target gaps first.
  • Review: in Ramp's published case study, each false positive becomes a pull request against Ramp's detection repository, and "an engineer reviews and merges." Ramp reports that alerts grew from about 4,000 to more than 40,000 a year while triage dropped from about 30 minutes per alert to under a minute.

Cotool also exposes these steps over MCP, so a coding agent such as Claude Code can search logs, create a detection, test it, and deploy it to run every 15 minutes. The same loop, seen from the hunter's side, is in AI threat hunting.

Common mistakes

  • Shipping the hunt query unchanged. Hunt queries are written to be read by a person. Without the hunter's manual checks encoded, they become noisy rules.
  • Backtesting with alerting on. In Google SecOps, that creates a SOAR alert for every historical match.
  • Ignoring ingestion delay. A rule that saw the event in a hunt can still miss it live if the lookback is shorter than the source's delay.
  • Broad exclusions. Excluding a whole account or host to cut noise hides the attack the rule was written for.
  • Counting hunts, not outcomes. PEAK's example: "We put twelve new automated detections into production this quarter" says more than a count of hunts. Only 40% of SANS 2026 respondents formally measure hunt outcomes at all.
  • Forcing every hunt into a rule. A good dashboard beats a rule nobody trusts.

Checklist

  • The hunt's query finds the behavior without manual interpretation
  • Manual checks from the hunt are encoded or listed as blind spots
  • An ADS-style write-up covers goal, blind spots, false positives, and response
  • The backtest covers at least the hunt's window, with alerting off
  • The hunt's true positives fire in the backtest
  • Expected alerts per day fits triage capacity at this severity
  • Lookback covers each source's ingestion delay
  • The rule merged through review and shadow-ran before paging
  • Hunts that stayed hunts have a schedule and a revisit condition
  • You report detections created and gaps closed each quarter

FAQ

How do you know when a threat hunt should become a detection?
A threat hunt should become a detection when the query can find the behavior again without a hunter interpreting the results, and a backtest shows an alert volume your team will triage. Splunk's PEAK framework asks it as one question: would you be confident enough to put the result in front of a SOC analyst as an alert? If yes, write a rule or analytic; if no, a report or dashboard fits better.
How long should you backtest a detection rule?
Backtest a detection rule over at least the window your hunt covered, and no less than one to two weeks. Elastic suggests 7 to 14 days to capture weekday and weekend patterns, and Google SecOps retrohunts default to one week. Longer windows, such as 30 days, catch monthly jobs and give a steadier false positive rate.
When should a threat hunt stay a hunt instead of a detection?
A threat hunt should stay a hunt when finding the behavior depends on a person's judgment, the data arrives in batches or with long delays, the query is too expensive to run every few minutes, or the behavior is rare and low severity. Run it on a schedule, such as monthly, and recheck whether better data or tuning lets it become a rule.
What is the difference between a threat hunt and a detection?
A threat hunt is a time-boxed search, led by a person or an agent, for activity no existing rule covers, while a detection is a rule or analytic that runs continuously and raises an alert on a match. A good hunt often ends by producing a detection, so the same behavior raises an alert next time without anyone hunting for it.

Sources

  1. Turning Hunts Into Detections with PEAK - Splunk · accessed November 16, 2026
  2. Measuring Hunting Success with PEAK - Splunk · accessed November 16, 2026
  3. Key Threat Hunting Deliverables with PEAK - Splunk · accessed November 16, 2026
  4. A Simple Hunting Maturity Model - David J. Bianco · accessed November 16, 2026
  5. Alerting and Detection Strategy Framework - Palantir · accessed November 16, 2026
  6. Create a scheduled analytics rule - Microsoft Sentinel · accessed November 16, 2026
  7. Handle ingestion delay in scheduled analytics rules - Microsoft Sentinel · accessed November 16, 2026
  8. Threat hunting in Microsoft Sentinel · accessed November 16, 2026
  9. Validate and test rules - Elastic Security · accessed November 16, 2026
  10. Run a rule against historical data - Google Security Operations · accessed November 16, 2026
  11. SANS 2026 Threat Hunting Survey: The Evolution of Threat Hunting (full report PDF) · accessed November 16, 2026
  12. AI Threat Hunting and Actionable Threat Intel - Cotool · accessed November 16, 2026
  13. AI Detection Engineering with Python Detections - Cotool · accessed November 16, 2026
  14. AI-Native SIEM - Cotool · accessed November 16, 2026
  15. How Ramp scaled detection and response 10x with Cotool · accessed November 16, 2026

Keep reading

Attackers are scaling with tokens

Cotool helps defenders operate at machine speed. See how security teams are scaling Detection & Response.

Book a demo