# How to test whether your AI agent repeats an action after a crash, with a runnable test > Kill an agent at each tool call, restart it, and count what happened twice. A test that needs no model, no keys and no changes to your framework, with a runnable agent that fails and a fixed one that passes. Source: https://getargentic.com/docs/guides/test-an-agent-for-repeated-actions/ Most agents are never tested for the one failure that costs money: the process dies after a refund goes out and before the agent notes it. This guide kills an agent at every tool call and counts what happened twice. ## What the test checks A crash test answers one question. If this agent is killed at any point and started again, does any action reach the outside world more than it did in a normal run? The test runs your agent once to learn what it does. Then it runs it again for each action and kills it at that action, twice: once before the agent hears back, and once right after. Each time it restarts the same command and counts what arrived. Anything that arrived more often than in the clean run is a repeat. Anything that never arrived is a lost action. It works from the receiving side, so your framework and your language don't matter. The tool stands in for the services your agent calls (an HTTP API, a payments-style API, mail, files, a SQLite file) and only watches what lands there. Point the agent's tool calls at those stand-ins. A model-driven agent can take a different route each run, so measure it with `--samples`. ## An agent that fails it This agent refunds an order, emails the customer and closes the ticket. It has no checkpoint and no keys, which is how a first version usually looks. The one change from production code is that it calls `$CRASHTEST_SINK_URL` instead of the real services, so the test can see the calls. before.mjs ``` // A support agent with no safeguards: refund, then email, then close the ticket. const sink = process.env.CRASHTEST_SINK_URL; const post = (path, body, headers = {}) => fetch(`${sink}${path}`, { method: "POST", headers: { "content-type": "application/json", ...headers }, body: JSON.stringify(body) }); await post("/refund_order", { order: 1182, amount: 40 }); await post("/send_email", { to: "dana@example.com", subject: "Your refund" }); await post("/close_ticket", { ticket: "T-77" }); ``` Run it under the test. This needs Node 20 or newer and nothing else: terminal ``` npx @argentic-labs/crash-test -- node before.mjs ``` You should see ``` Clean run: 3 effects #1 POST /refund_order no key #2 POST /send_email no key #3 POST /close_ticket no key Killing at 6 points, restarting each time... By action Action Clean run Worst Repeated in Verdict POST /refund_order 1× 2× 6 of 6 REPEATED POST /send_email 1× 2× 4 of 6 REPEATED POST /close_ticket 1× 2× 2 of 6 REPEATED Result kill points exercised: 6 of 6 trials with a duplicated effect: 6 trials with a lost effect: 0 exit code: 1 ``` The refund repeated at all six kill points, because a restarted agent starts from the top and the first refund is still on the other side. The email repeated in four and the ticket close in two: an action only repeats when the kill comes at it or after it. Each repeat is a real customer-facing mistake: a second refund, a second email. ## The same agent, fixed The smallest fix is a key on each action, built from the thing the action is about. The order, not a random id and not the attempt number. Then a restart sends the same key, and a receiver that honors keys keeps the first and answers the repeat with it. after.mjs ``` // A support agent with no safeguards: refund, then email, then close the ticket. const sink = process.env.CRASHTEST_SINK_URL; const post = (path, body, key) => fetch(`${sink}${path}`, { method: "POST", headers: { "content-type": "application/json", "idempotency-key": key }, body: JSON.stringify(body) }); await post("/refund_order", { order: 1182, amount: 40 }, "refund:1182"); await post("/send_email", { to: "dana@example.com", subject: "Your refund" }, "email:1182"); await post("/close_ticket", { ticket: "T-77" }, "close:T-77"); ``` terminal ``` npx @argentic-labs/crash-test -- node after.mjs ``` You should see ``` Clean run: 3 effects #1 POST /refund_order key refund:1182 #2 POST /send_email key email:1182 #3 POST /close_ticket key close:T-77 By action Action Clean run Worst Repeated in Verdict POST /refund_order 1× 1× 0 of 6 ok (same-key replays) POST /send_email 1× 1× 0 of 6 ok (same-key replays) POST /close_ticket 1× 1× 0 of 6 ok (same-key replays) Result kill points exercised: 6 of 6 trials with a duplicated effect: 0 trials with a lost effect: 0 trials with an absorbed replay: 6 (same Idempotency-Key; safe if the receiver dedupes on it) exit code: 0 ``` Every action still arrives twice in some trials. The report passes them because each repeat carries a key the receiver can recognize, and its line about absorbed replays says so. They are harmless only if the service on the other end dedupes on the key. Payment APIs do. A mail provider or an internal service may not, so check yours. The test can show you the difference: add `--no-dedupe` and the same agent fails at every kill point, because the stand-in then treats a repeated key as a second effect. The [retry guide](https://getargentic.com/docs/guides/make-agent-tools-safe-to-retry/) covers choosing keys and keeping your own record for services that forget them. ## Run it on your agent - Point the agent's tool calls at `$CRASHTEST_SINK_URL`. A call to `https://api.example.com/refund` becomes `$CRASHTEST_SINK_URL/refund`. Mail goes to the SMTP address in `$CRASHTEST_SMTP_URL`. Files go to `$CRASHTEST_FILES_DIR` and rows to `$CRASHTEST_DB_PATH`. - Put your command after the `--`. Python, Go, a shell script or a Node entry point all work: `npx @argentic-labs/crash-test -- python3 agent.py`. - Make the agent exit 0 when its work is done, so the test can tell a finished restart from a broken one. - If the agent keeps state in its working directory, add `--copy-cwd` so each trial starts clean. If it keeps checkpoints, put them in `$CRASHTEST_STATE_DIR`, which survives the kill. - If the agent calls a model, route that call through `--upstream model=`. The test then also kills between a model reply and the tool call it asked for, which is the gap a normal run never hits. - A model-driven agent can take a different route each run. Use `--samples 5` to measure a repeat against the most any clean run did, and `--intent order_id` so "refund order 1182" counts as one action however the agent got there. ## Run it in CI The exit code is the verdict. 0 means every action landed as often as in the clean run. 1 means something landed twice or never. 2 means the setup is wrong, or a restarted agent crashed or timed out. `--json report.json` writes the details and `--md report.md` writes a version for a pull request comment. A fixed agent that stays at 0 is a regression test for the bug that is hardest to catch by reading code. The test runs the agent once per kill point, so use a scripted or cheap model in CI. The examples in the [crash-test repo](https://github.com/Argentic-Labs-AI/argentic-plugins/tree/main/crash-test) use a scripted model and make no paid calls. ## What this doesn't cover - It sees the effects that reach its stand-ins: HTTP requests of any method, a payments-style API, SMTP, appended files and inserted SQLite rows. SQLite updates and deletes, and other databases, are not seen. - Kill points follow the order of calls. An agent that makes calls in parallel can reach a different order on another run. - The output above is from version 0.1.0 on Node 22 and Linux. macOS works too, and Windows through WSL. - A pass means no repeat at the kill points it tried, with a receiver that honors keys. It doesn't test a real provider, and it can't promise an action happens once. If you would rather not write the keys and the resume logic yourself, durable-execution tools such as Temporal, Restate and DBOS resume from the last recorded step, though your side effects still need to tolerate a repeat. Argentic is one more option: it journals each tool call and resumes a killed run without repeating a finished one. The same test works on all of them, which is a fair way to compare. Our own numbers are on the [crash test study](https://getargentic.com/research/crash-test/).