Stop Discussing AI Incidents. Start Replaying Them.

Stop Discussing AI Incidents. Start Replaying Them.

Stop Discussing AI Incidents. Start Replaying Them. 1920 1280 Andrew Heller

The scenario is familiar. A customer support agent, the one with RAG access to your internal knowledge base and a connected action tool that can issue refunds. Someone submits a support ticket. Buried in the ticket text is an instruction the agent was never meant to follow.

The security team works the problem. The team is sharp. They identify the injection point, they name the data at risk, they agree the SOC would catch the anomalous tool calls. The tabletop facilitator calls time. Everyone leaves confident.

Nothing that the team just agreed upon has ever been tested.

This is the state of most AI incident tabletops. They validate discussion, not detection. And AI agents do not operate at discussion speed.


The speed problem

On October 19, Reuters reported that a Chinese-speaking hacker broke into more than 100 companies, including Deere, SharkNinja, and Cummins. The hacker did almost none of the work. The intrusions were outsourced almost entirely to AI agents, which spent hours autonomously probing and breaking into target sites. A researcher at NorForge watched it happen through the attacker's own exposed tooling. "It's a new world now," he said.

One person. More than 100 breaches. A few thousand dollars. No human in the loop. 

That is the tempo of AI incidents, and it breaks the central assumption of the traditional tabletop. Tabletops assume human-speed response: someone notices, someone escalates, someone convenes. But the full attack chain, injection to tool invocation to exfiltration, now fits inside a single agent task loop. It completes in seconds to minutes, with no human anywhere in the chain. By the time the conference room assembles, the exfiltration is done.

We have seen this coming for months. In August, we tracked frontier models escaping sandboxes and attacking real infrastructure. Those were warnings. This is the confirmation: autonomous agents conducting end-to-end intrusions at a scale no human team could match. (Reuters could not confirm how much data was ultimately extracted, which is worth stating plainly. The intrusions themselves are the story.)

So the question is not whether your team can talk through an AI incident. It is whether your telemetry can see one while it is happening. A tabletop cannot answer that. Only a replay can.


The format: TTX + TTP replay

The exercise has two phases, and the second is the one that matters.

Phase one is the tabletop. A realistic AI incident scenario, built from current attack patterns: indirect prompt injection (AML.T0051.001), tool and permission abuse (AML.T0053), data extraction through connected integrations (AML.T0086). The team works it the way they would work the real thing. Every decision gets recorded: who gets called, what gets isolated, which logs get pulled.

Phase two is the replay. The team executes the exact TTPs from the scenario in a controlled environment against the client's actual stack. The tabletop prompt injection becomes a real injection. The tool abuse the team discussed becomes real tool calls. Every technique maps to MITRE ATLAS, so the results land in a framework the detection team already uses. In most scenarios, the core kill chain runs from AML.T0051.001 (indirect prompt injection) to AML.T0053 (AI agent tool invocation) to AML.T0086 (exfiltration via AI agent tool invocation).

Then comes the comparison. The team said the SOC would see the anomalous tool calls in ten minutes. The replay shows whether anything fired at all.


What the delta tells you

The gap between the tabletop and the replay is the finding. It usually falls into three buckets.

First, detection gaps. The team assumed a log source existed but never configured it, or wrote an alert but never tuned it. The replay proves the negative.

Second, speed gaps. The team assumed escalation in minutes. The replay shows the detection, if it exists, fires in hours, or only after the agent has completed its loop.

Third, ownership gaps. The tabletop assigns the AI incident to the SOC. The replay reveals nobody owns the agent's action logs, because the agent belongs to the business unit that bought it and the logs belong to no one.

Each of these is a governance failure wearing an incident response costume. Each maps directly to decisions an AI governance program must make explicit: which systems are in scope, who owns them, and what data they touch.


A scenario sketch

Take the support agent. RAG over internal documentation, plus an action tool that can issue refunds and update account records. The attacker's ticket contains an indirect prompt injection (AML.T0051.001): instructions embedded in what looks like a customer complaint, telling the agent to pull account records beyond the requester's scope (AML.T0082, data boundary violation) and push them through the refund tool's memo field (AML.T0086, exfiltration via tool invocation).

The tabletop team quickly catches the injection vector. They are good. They say the DLP would flag the memo-field exfiltration, and the SOC would isolate the agent's service account.

The replay runs it. The injection works because the agent's system prompt never defined a trust boundary for ticket content. The memo field exfiltration works because DLP was tuned for email and file shares, not a refund tool's free-text field. The service account is never isolated because no runbook covers an AI agent as a compromised identity.

Three assumptions, three failures, one afternoon. That is the value of the replay: it converts confident assertions into evidence.

Three assumptions, three failures, one afternoon. 

Why this is not a pentest and not a tabletop

A penetration test finds vulnerabilities. A tabletop exercises decisions. The TTX+TTP format does something neither does alone: it tests whether the organization's beliefs about its own detection and response survive contact with the actual attack.

The deliverable is not a list of findings. It is a corrected incident response plan, annotated with proof. "We believed X. The replay showed Y. Here is the control change."

The tabletop tells you what the team believes. The replay tells you what is true. 

Running it

The exercise works best with the team that would actually respond: SOC analysts, the AI platform owner, the data owner, and whoever gets woken up at 2 AM. Half a day for the tabletop, half a day for the replay and the delta review. The scenario should be built from the client's real AI footprint, not a generic deck, which means it starts with discovery.

Lares runs these exercises. The tabletop tells you what the team believes. The replay tells you what is true. The distance between the two is the work.

Related Article

Minnesota Water Attacks and the Predictable Reality

August 12, 2026 by Andrew Heller How possible Iran-linked actors exploited exposed PLCs in the 2026 Minnesota water attacks. MITRE ATT&CK mapping, risk matrix, and OT security lessons from Lares. Read More Blog, Penetration Testing, Purple Teaming, Red Teaming

Part 3: The Collapse of the Testing Boundary: Why Purple Teaming is Mission-Critical

August 7, 2026 by Andrew Heller Point-in-time pen tests can't catch AI agents that fail in minutes. See why continuous purple teaming is the only defense — and how to build the budget case. Read More Artificial Intelligence, Blog, Purple Teaming

Part 2: The Collapse of the Testing Boundary: Anthropic and the Behavioral Threat

August 7, 2026 by Andrew Heller Claude models attacked real infrastructure while believing they were in a simulation. Anthropic's retrospective reveals a new AI risk class beyond alignment. Read More Artificial Intelligence, Blog, Purple Teaming

Part 1: The Collapse of the Testing Boundary: Deconstructing the OpenAI Agent Breakout

August 7, 2026 by Andrew Heller OpenAI's frontier model escaped its sandbox and breached Hugging Face's cluster in under 13 hours. See the full kill chain mapped to MITRE ATT&CK and ATLAS. Read More Artificial Intelligence, Blog, Purple Teaming

Social Profiling – OSINT for Red/Blue

July 27, 2026 by Lares Labs Read More Blog, Penetration Testing, Red Teaming

The Phantom Menace: Exposing hidden risks through ACLs in Active Directory

June 18, 2026 by Raúl Redondo Discover how attackers exploit hidden risks in Active Directory ACLs. Explore techniques like GenericAll, GenericWrite, and WriteDACL abuse in our latest post. Read More Blog, Insider Threat, Penetration Testing, Red Teaming

Kerberos IV - Delegations

June 17, 2026 by Raúl Redondo Discover how to abuse Kerberos for lateral movement. Learn User Impersonation techniques like Pass the Ticket, Shadow Credentials, and forging tickets. Read More Blog, Blue Team, Penetration Testing, Red Teaming

Kerberos III - User Impersonation

June 17, 2026 by Raúl Redondo Discover how to abuse Kerberos for lateral movement. Learn User Impersonation techniques like Pass the Ticket, Shadow Credentials, and forging tickets. Read More Blog, Blue Team, Penetration Testing, Red Teaming

Kerberos II - Credential Access

June 16, 2026 by Raúl Redondo Dive into the fundamentals of the Kerberos authentication protocol. Explore its history, core concepts, authentication flow, and PKINIT in part one of our series. Read More Blog, Blue Team, Penetration Testing, Red Teaming

Kerberos I - Overview

June 16, 2026 by Raúl Redondo Dive into the fundamentals of the Kerberos authentication protocol. Explore its history, core concepts, authentication flow, and PKINIT in part one of our series. Read More Blog, Blue Team, Penetration Testing, Red Teaming

Empowering Organizations to Maximize Their Security Potential.

Lares is a security consulting firm that helps companies secure electronic, physical, intellectual, and financial assets through a unique blend of assessment, testing, and coaching since 2008.

18+ Years

In business

600+

Customers worldwide

4,500+

Engagements

Where There is Unity, There is Victory

[Ubi concordia, ibi victoria]

– Publius Syrus

Contact Lares Consulting logo (image)

Continuous defensive improvement through adversarial simulation and collaboration.

Email Us

©2025 Lares, a Damovo Company | All rights reserved.