Skip to main content
A fire alarm pull station on a brick wall above an open notebook and pen, drawn in dots

AI incident response: what agents do in an investigation

What AI agents do in incident response, phase by phase under NIST SP 800-61r3, with BlueBench scores, failure modes, costs, and the calls people keep.

Guide
Updated 12 min read

Summary

In AI incident response, agents do the evidence work of an investigation: validating the alert, scoping affected hosts and accounts, ordering events into a timeline, drafting containment options, and writing the report. People still decide the containment strategy, legal and notification questions, and when recovery is done. On BlueBench's incident response track, the best models scored 86 to 88 out of 100, and six benchmarks produced six different leaders.

Top BlueBench incident response track score (GPT-6 Astra, Oct 2026)
88.0
Best incident report score on a real Windows intrusion
83.5%
BlueBench benchmarks with a different incident response leader
6 of 6
Range of average cost per BlueBench task on two benchmarks
$0.07–$19.22

Key takeaways

  1. Give agents the evidence work (validation, scoping, timeline, report draft) and keep containment strategy, notification, and recovery sign-off with a named incident lead.
  2. Require an approval step on every state-changing tool, and default containment to "prepare and ask" rather than "act".
  3. Pick the model per task from scores on cases like yours: no model led incident response on more than one of BlueBench's six benchmarks.
  4. Grade agent reports the way BlueBench does: every finding cites evidence, and calling a benign lookalike malicious costs more than missing it.
  5. Plan for refusals before an incident by testing your models on real attack artifacts and keeping a fallback.

The short version

An incident response agent is a model with read access to your logs and tools, a loop that lets it run queries until it has an answer, and a set of actions it may take or propose. Given an alert, it does the work an analyst does in the first hours: confirm the alert is real, find every host and account involved, order the events, and write up what happened and what to do next.

That work is measurable. BlueBench is Cotool's public benchmark for AI in defensive security, run in environments built around real and simulated intrusions; Cotool built it because capture-the-flag benchmarks mostly test offensive skills. It scores incident response by having an agent "investigate an alert end to end and write the incident report: scope, affected hosts and accounts, attacker tradecraft, impact, and the response actions warranted," graded against a hidden rubric. As of its October 2026 update, the top incident response track scores were GPT-6 Astra at 88.0 and Claude Opus 5.5 at 87.8, with Gemini 4 Argon at 86.8 and GPT-6.1 Sol at 86.4.

Three results matter more than the leaderboard order:

  • No model wins everywhere. Across BlueBench's six benchmarks, incident response was led by six different models.
  • Real intrusions are harder than simulations. The best incident report on a real Windows enterprise intrusion scored 83.5%; simulated cases topped out between 95.1% and 99.3%.
  • Decisions stay with people. NIST's incident response guidance assigns choices like containment strategy, management escalation, and legal review to humans, and nothing in the benchmark results argues for changing that.

How NIST SP 800-61r3 frames the work

NIST SP 800-61 Revision 3 is the current revision. It was finalized in April 2025, supersedes Revision 2 from August 2012, and is still listed as current on NIST's incident response project page. It drops the old four-phase life cycle (preparation; detection and analysis; containment, eradication, and recovery; post-incident activity) and maps incident response onto the six Functions of the Cybersecurity Framework (CSF) 2.0.

Govern, Identify, and Protect are preparation, which NIST describes as "not part of the incident response itself." Detect, Respond, and Recover are the response. Improvement, under Identify (ID.IM), feeds lessons from every incident back into all six. NIST also notes how much the job has grown: in the Rev. 2 era, incidents were "usually completed within a day or two," while today recovery "often takes weeks or months."

The CSF categories give a cleaner map of where agents help than a phase diagram does:

CSF 2.0 categoryWhat NIST asks forAgent role
DE.AE Adverse event analysisFilter large event sets for human reviewDo it
RS.MA Incident managementTriage, validate, categorize, prioritizeDraft; person approves
RS.AN Incident analysisRoot cause, timeline, magnitudeDo it, with evidence
RS.CO Reporting and communicationNotify, coordinate, shareDraft only
RS.MI Incident mitigationContain and eradicatePrepare; person decides
RC.RP Recovery plan executionRestore, verify backups, after-action reportDraft and check

Step by step: what agents do in an investigation

1. Triage and validation

NIST's RS.MA-02 asks responders to triage and validate incident reports, and its incident management guidance says incidents "should not be handled on a first-come, first-served basis." Prioritization weighs asset criticality, functional impact, data impact, attack stage, threat actor, and recoverability. An agent can fill in most of those fields from asset inventory, identity, and endpoint tools before a person reads the alert.

The BlueBench simulation cases test this directly: each starts from a single weak alert surrounded by several thousand raw events, and the system prompt tells the agent to "treat the incoming alert as a lead, not a conclusion." That framing matters for alert fatigue, since an agent that validates every alert removes the queue that fatigue comes from.

2. Scoping

Scoping is NIST's RS.AN-08: estimating the incident's magnitude, which the document calls "often one of the most challenging aspects" of response. Underestimating it lets the incident continue. Agents are suited to the breadth part, pivoting from one host to every host that shares a credential, a hash, or a destination. That only works across the tools the agent can query. Cotool's agents investigate alerts from any connected source, whether or not Cotool stores the logs, through over 100 built-in integrations plus custom MCP and CLI tools, and facts they learn along the way, such as "Okta is the source of truth for employee status," go into a global memory every agent reads.

The Windows Enterprise Intrusion benchmark shows how hard real scoping is. Agents get 3.9M+ events across 40 log tables and five hosts, and the attack chain runs from an RDP password spray through a remote access trojan, a hidden admin account, Mimikatz, tunneled command-and-control disguised as a service, WMI lateral movement, and domain controller replication. On its incident report task, the best model was Gemini 4 Argon at 83.5%.

3. Timeline and root cause

RS.AN-03 asks for the sequence of events and the root cause; RS.AN-06 asks responders to record their actions and preserve the integrity and provenance of what they collect; RS.AN-07 covers evidence collection and chain of custody. Agents are good at the first: ordering thousands of events and naming the step where the attacker got in. The second and third are partly free, because an agent's query log is a record of every action taken. Cotool's audit log, for example, records every agent execution and each tool call with its parameters, exportable by API to a SIEM, and its agents keep working files such as an IOC list and hand back deliverables like incident-report.pdf and timeline.xlsx from a persistent workspace. Whether that record meets your chain-of-custody standard is a question for your legal team, not the model.

4. Containment recommendations

NIST's RS.MI-01 suggests considering automatic containment, such as quarantining malware or isolating an endpoint, while letting handlers choose actions manually. RS.MA-03 adds the trade-off people must weigh: observing an attacker to learn more versus recovering quickly. And redirecting an attacker into a sandbox should involve the legal department first.

In BlueBench, containment is graded. The AWS Cloud Intrusion rubric includes containment as its own category, and the Impact & Exfiltration benchmark's incident response output includes response-action choices. The AWS task prompt also tells agents to "be precise about what the evidence does and does not establish — do not overstate impact or success," which is the same discipline a containment recommendation needs.

5. Report writing

The incident report is where agents save the most time, because the evidence is already in the agent's context. BlueBench's Windows task asks for "a complete incident response report: scope, attack chain, affected accounts and hosts, and containment recommendations." NIST's RC.RP-06 asks for an after-action report at the end of recovery, and RS.CO asks for updates to senior leadership along the way. An agent can draft each; a person should sign them.

Where people must decide

NIST's guidance leaves several decisions with named people, and the benchmark results give no reason to move them:

  • Who leads. RS.MA-01 suggests designating an incident lead. That person owns the decisions below.
  • Escalation and elevation. RS.MA-04 separates escalation (more resources) from elevation (management attention). An agent can recommend both; a person should trigger elevation.
  • Containment strategy. Watch or evict, isolate now or wait for business hours, revoke a token for four users or forty. These trade business risk against evidence.
  • Legal and notification. Sandboxing an attacker needs legal review first, insider activity means notifying HR, and regulator or customer notification has deadlines an agent shouldn't own.
  • When recovery is done. RS.MA-05 sets criteria for ending response, and RC.RP-03 asks responders to verify backup integrity before restoring. Signing off on both is a judgment about risk.

A practical rule: the agent may read anything it's been granted and write to the case, but any action that changes state outside the case (isolating a host, suspending a user, revoking a grant) goes through an approval.

Common mistakes

Trusting a confident report. BlueBench's scoring rules show how reports fail: a correct finding without evidence earns reduced credit, and asserting that a benign lookalike is malicious applies a multiplicative penalty. The Impact & Exfiltration benchmark plants benign lookalikes and a decoy data-export thread for this reason. A report that names every suspicious process as attacker activity reads well and scores badly.

Picking one model for everything. Incident response leadership split six ways: Kimi K3 led the macOS intrusion (96.3%), GPT-5.6 Sol the AWS synthesis and containment score (97.6%), Gemini 4 Argon the Windows report (83.5%), GPT-6 Astra the initial-access simulation (95.5%), Claude Sonnet 5.5 the identity simulation (99.3%), and Claude Opus 5.5 the impact and exfiltration simulation (95.1%). Spread also matters. On the Windows benchmark, Gemini 4 Argon's scores spread 100.0 points across runs. Which AI models are best for security work compares the models across all four tracks.

Not planning for refusals. Hugging Face's July 2026 incident disclosure describes analyzing more than 17,000 recorded attacker events and finding that hosted frontier models blocked the requests, because their guardrails "cannot distinguish an incident responder from an attacker." It ran the forensic work on the open-weight GLM-5.2 on its own infrastructure. BlueBench flags Claude Opus 5 and Claude Sonnet 5 as "cyber refused," and refused attempts score whatever they earn. Cotool handles this with Router, which moves a refused or failed request to another provider or an open-weight model.

Giving the agent write access by default. An agent that can isolate hosts will eventually isolate the wrong one. Grant read tools broadly and state-changing tools narrowly, each behind an approval.

Measuring speed only. Mean time to respond falls quickly with agents. Track the accuracy of agent verdicts alongside it, as covered in SOC metrics beyond MTTD and MTTR.

What AI incident response costs

Model usage is cheap next to analyst time, but the range is wide. BlueBench reports average cost per task at list token prices with prompt caching, excluding the judge:

  • Windows Enterprise Intrusion: GPT-5.6 Sol scored 80.7% at $1.88 per task in 3m 51s; Claude Opus 5.5 scored 80.6% at $3.41 in 21m 27s; GPT-6.1 Sol scored 79.6% at $0.93; GLM-5.3 Flash was cheapest at $0.07 for 69.7%; GPT-6 Astra cost $9.43 for 76.2%.
  • Impact & Exfiltration: Claude Opus 5.5 scored 89.8% at $5.17; GPT-6 Astra 88.0% at $12.10; Claude Sonnet 5.5 83.4% at $1.29; Gemini 4 Argon 79.3% at $19.22; GLM-5.3 Flash 69.6% at $0.16.

Across all six benchmarks, Claude Sonnet 5.5 scored 77.5 overall at $1.01 per task against Claude Opus 5.5's 83.2 at $3.54, and DeepSeek V4.1 Flash scored 71.5 at $0.17. For a team, the useful question is which alerts justify the expensive model. One option is a cheap model for first-pass triage and a frontier model once an alert looks real. Cotool sets the model per agent for this reason, with "frontier models for hard cases, cheaper ones for volume"; its own example pairs GLM-5.3 at low effort for triage with a Claude model at high effort for investigation. The bigger cost is the human review each report still needs. At Ramp, principal security engineer Antoinette Stevens describes reviewing an agent-investigated alert and merging its proposed detection fix "in five minutes."

A worked example: an OAuth grant responder

Cotool's response agents show the shape described above in one workflow. Its OAuth Grant Responder starts from a suspicious third-party app grant in Google Workspace. The agent lists the app's scopes and grants, checks the publisher domain in VirusTotal, traces what the token read and changed, and then asks #security before acting. The approval request reads "Revoke app pdf-convert-pro for all 4 users?"

The containment rule is written into the agent rather than left to the model's judgment. Cotool's Endpoint Alert Responder system prompt says to "prepare containment (network quarantine, session revocation) for approval; never auto-isolate developer endpoints without human sign-off," and to "propose a scoped exclusion instead of a broad suppression." On the Cotool platform, state-changing actions such as isolating a host in Defender or suspending a user in Okta can require approval in the app or Slack, and one click makes an agent read-only.

Ramp's security team runs a version of this split. Its agent writes a disposition into the ticket and Slack and escalates true positives to a person; when the answer is a false positive, the agent opens a pull request against the detection repository, and an engineer reviews and merges it. Ramp reports triage dropped from about 30 minutes to under a minute per alert while alert volume grew tenfold. For running that approval loop in chat, see running an AI SOC from Slack and Teams.

Checklist

  • Name an incident lead for every incident the agent escalates, per NIST RS.MA-01.
  • Map each agent task to a CSF 2.0 category, and mark which ones the agent does, drafts, or only prepares.
  • Grant read tools broadly; put every state-changing tool behind an approval.
  • Write containment rules into the agent's instructions, including which assets are never contained automatically.
  • Require every finding in an agent report to cite the query or event behind it.
  • Test your models on your own closed incidents, including ones with benign lookalikes, before trusting them.
  • Run a refusal test with real attack artifacts, and keep a fallback model ready.
  • Track verdict accuracy and review time alongside time to respond.
  • Keep the agent's action log with the case, and ask legal whether it meets your evidence standard.
  • Feed after-action findings back into detections and agent instructions (NIST ID.IM).

FAQ

Can AI handle incident response on its own?
AI can handle much of incident response's investigation work on its own, but not the decisions. On BlueBench, top models scored 86 to 88 out of 100 on the incident response track, and the best report on a real Windows intrusion scored 83.5%. NIST SP 800-61r3 also places choices such as containment strategy, escalation to management, and legal involvement with people, so agents should prepare those actions and ask.
What does NIST SP 800-61 Rev. 3 change for incident response?
NIST SP 800-61 Rev. 3, finalized in April 2025, replaces the old four-phase incident response life cycle with the six Functions of the Cybersecurity Framework 2.0. Govern, Identify, and Protect cover preparation; Detect, Respond, and Recover cover the response itself. It supersedes Rev. 2 from 2012 and notes that recovery today often takes weeks or months.
Which AI model is best for incident response?
No single AI model is best for incident response across case types. On BlueBench's incident response track, GPT-6 Astra (88.0) and Claude Opus 5.5 (87.8) were within a point, and each of the six benchmarks had a different leader, including Kimi K3 on a macOS intrusion and Gemini 4 Argon on a Windows one. Test the models on cases that look like yours before choosing.
How much does an AI incident investigation cost?
An AI incident investigation costs cents to tens of dollars in model usage, depending on the model. On BlueBench's Windows intrusion benchmark, average cost per task ran from $0.07 to $9.43, and on the impact and exfiltration benchmark from $0.16 to $19.22. Those figures exclude platform fees and the analyst time spent reviewing the output.
Why do AI models refuse incident response work?
AI models refuse incident response work because attack commands, exploit payloads, and command-and-control artifacts look the same whether a responder or an attacker submits them. Hugging Face reported in July 2026 that hosted frontier models blocked its analysis of more than 17,000 attacker events, so it ran the work on an open-weight model instead. Two Claude models on BlueBench are flagged for cyber refusals.

Sources

  1. SP 800-61 Rev. 3, Incident Response Recommendations and Considerations for Cybersecurity Risk Management: A CSF 2.0 Community Profile | CSRC · accessed November 16, 2026
  2. NIST SP 800-61 Rev. 3 (PDF) · accessed November 16, 2026
  3. Incident Response | CSRC · accessed November 16, 2026
  4. BlueBench — AI Benchmark for Defensive Security · accessed November 16, 2026
  5. BlueBench Methodology · accessed November 16, 2026
  6. BlueBench Cost Efficiency · accessed November 16, 2026
  7. Windows Enterprise Intrusion · accessed November 16, 2026
  8. AWS Cloud Intrusion · accessed November 16, 2026
  9. macOS Threat Investigation · accessed November 16, 2026
  10. Initial Access & Command-and-Control · accessed November 16, 2026
  11. Identity & Active Directory Attacks · accessed November 16, 2026
  12. Impact & Exfiltration · accessed November 16, 2026
  13. Security incident disclosure — July 2026 · accessed November 16, 2026
  14. Introducing Cotool Router · accessed November 16, 2026
  15. AI Alert Triage and Incident Response Agents · accessed November 16, 2026
  16. AI SOC Without the Black Box · accessed November 16, 2026
  17. Beyond CTFs: Evaluating AI Agents on Real Intrusion Data · accessed November 16, 2026
  18. Audit Logs - Cotool Documentation · accessed November 16, 2026
  19. How Ramp Scaled Detection & Response With Cotool · accessed November 16, 2026

Keep reading

Attackers are scaling with tokens

Cotool helps defenders operate at machine speed. See how security teams are scaling Detection & Response.

Book a demo