11 hours 59 minutes on shift,and what it remembered.
One agent ran the back office of a small online shop for 11 hours 59 minutes of clock time, about 75 minutes of it with our test harness down, on Claude Haiku 5.5, handling 900 of 900 events while its worker was killed 31 times. It answered 81 of 91 callback questions right, including facts stated about 11 hours earlier. No kill made a finished action run again. It did refund 7 orders more than once, which no kill caused.
Neil Luo · · Finished
A good demo of memory is easy to stage and says little about a shift that lasts hours. So we gave one agent the back office of a small online shop and let it work for 11 hours 59 minutes, one event at a time: orders, refund requests, customer emails, deliveries, price changes and messages from the CEO. A supervisor killed its worker 31 times at random points.
Along the way, customers state preferences, the refund policy changes and the CEO sets rules, each once. Later the stream asks about one of those facts, from a few events to many hours after it was stated, with nothing to look it up in. If the agent can still do the step with the fact in view but not hours later, that is memory failing, not the model. Nobody gave it a hint, and we publish the curve whichever way it fell.
89% right
81 of 91 callback questions answered right, from facts stated 5 events earlier to facts stated 846 events (about 11 hours) earlier. No distance band scored below 83%. 900 events handled by Claude Haiku 5.5 in 11 hours 59 minutes, 31 kills, about $0.51 of model spend by the journal ($0.50 on the 952 calls with token counts). No kill made a finished action run again.
What went wrong, and no kill caused it: 7 orders were refunded more than once. The agent paid an earlier refund promise again when a later delivery arrived. And its running stock counts were exact on only 5 of 28 questions.
What it remembered, and how far back.
Every callback question was paired with a fact stated earlier in the stream, at a set distance. Control questions come within 15 events, so the fact is still in the agent's recent view and the step is easy for the model. The other bands are far past that, by up to 846 events. The questions come in four kinds: a customer's packaging and invoice preference, a refund promised until a delivery lands, a quoted price that changed, and a refund window that changed.
Long runone graded question per fact
89% of callback questions right
Questions answered right, by how long ago the fact arrived
| Kind | In view | Near | Mid | Far | Very far |
|---|---|---|---|---|---|
| Refund-window policy | 6/6 | 6/6 | 2/2 | n/a | n/a |
| Promised refunds | 1/4 | 2/4 | 2/4 | 3/4 | 1/1 |
| Price changes | 4/4 | 11/11 | 4/4 | 1/2 | 2/3 |
| Customer preferences | 4/4 | 8/8 | 8/8 | 8/8 | 8/8 |
The prompt stayed capped.
A plain loop adds every result to a growing transcript. Here the engine rebuilds the prompt from the journal, so what the model reads each call stays bounded. Across the 952 calls with recorded usage the median prompt was 22.1k tokens, the 90th percentile 30.8k and the largest 43.4k. The median of each tenth of the run stayed between 16.8k and 29.4k.
Long runops-ops12h · hosted engine
22.1k median prompt
Prompt size at every call
Long runthe last calls of the run, one point each
2 rebuilds in view
The last 120 calls, one point each
Every kill, and what came after.
Before each start the supervisor picked a point 40 to 80 executed actions ahead and one of four kinds of kill, then killed the worker there with the operating system's kill signal. It started the worker again as soon as it was gone. From a kill to the next finished call took a median of 92 seconds and at most 1056.
Long runevery event and every kill
900 of 900 events handled
Events handled over the run
| Kill | Where it landed | Kills |
|---|---|---|
| After a tool acted, before the engine heardThe tool had changed the files and saved its result. The worker died before the engine was told. | The tool had changed the files and saved its result. The worker died before the engine was told. | 10 |
| Just before a tool ranKilled after the model asked for a tool and before the tool did anything. | Killed after the model asked for a tool and before the tool did anything. | 8 |
| Just after an action finishedKilled as soon as an action had run, before the next one. | Killed as soon as an action had run, before the next one. | 8 |
| During the model callThe worker was killed while its request to the model was in flight. | The worker was killed while its request to the model was in flight. | 5 |
| All kills | 31 |
All 31 kills, one row each
| # | Call | Kind | Back after | Found already done |
|---|---|---|---|---|
| 1 | 43 | Just before a tool ran | 6 s | No |
| 2 | 64 | After a tool acted, before the engine heard | 4 s | Yes |
| 3 | 110 | Just after an action finished | 6 s | No |
| 4 | 145 | Just after an action finished | 48 s | No |
| 5 | 167 | During the model call | 93 s | No |
| 6 | 191 | Just before a tool ran | 93 s | No |
| 7 | 218 | After a tool acted, before the engine heard | 48 s | Yes |
| 8 | 247 | Just before a tool ran | 94 s | No |
| 9 | 279 | After a tool acted, before the engine heard | 94 s | Yes |
| 10 | 312 | During the model call | 93 s | No |
| 11 | 338 | After a tool acted, before the engine heard | 93 s | Yes |
| 12 | 369 | Just after an action finished | 94 s | No |
| 13 | 404 | Just before a tool ran | 94 s | No |
| 14 | 432 | Just after an action finished | 48 s | No |
| 15 | 455 | Just after an action finished | 94 s | No |
| 16 | 480 | After a tool acted, before the engine heard | 48 s | Yes |
| 17 | 523 | During the model call | 46 s | No |
| 18 | 563 | After a tool acted, before the engine heard | 5 s | Yes |
| 19 | 622 | Just before a tool ran | 94 s | No |
| 20 | 656 | Just after an action finished | 53 s | No |
| 21 | 694 | Just before a tool ran | 92 s | No |
| 22 | 724 | Just before a tool ran | 94 s | No |
| 23 | 755 | Just before a tool ran | 94 s | No |
| 24 | 780 | After a tool acted, before the engine heard | 92 s | Yes |
| 25 | 813 | During the model call | 45 s | No |
| 26 | 836 | During the model call | 1056 s | No |
| 27 | 857 | After a tool acted, before the engine heard | 3 s | Yes |
| 28 | 880 | Just after an action finished | 92 s | No |
| 29 | 908 | Just after an action finished | 92 s | No |
| 30 | 948 | After a tool acted, before the engine heard | 93 s | Yes |
| 31 | 969 | After a tool acted, before the engine heard | 48 s | Yes |
What we counted.
Counted from the run's own event log, one row per action in the downloads below. After a restart the worker walks back through the steps the engine already has, and an action recorded as finished is skipped instead of run. That happened 2,286 times.
Events handled900 of 900
Events missed0
Callback questions answered right81 of 91
CEO stock questions answered exactly5 of 28
Routine events answered as the scenario expects781 of 781
Orders shipped twice0
Orders refunded more than once (11 extra refund payments; two orders were paid four times)7
Refunds paid that nobody had promised0
Actions the agent ran1,842
A finished action ran again0
Actions applied twice under the same key, of 1,841 applied (separately, 11 extra refund payments under new keys)0
Cut-off actions found already done when retried under the same key10 of 10
Actions skipped on a restart because the journal already had them2,286
Provider errors0
Model spendabout $0.51 by the journal ($0.50 on the 952 calls with token counts)
Share of prompt tokens served from the provider's cache91%
Open a wake.
Each link is one call of this run, signed: the prompt the model was sent, what it did, each action's key, and the two checks as they ran when we shared it. The second check, running the engine again on the recorded replies, could not run on this agent's journal, because it is longer than that check accepts. The page says so on each one.
- A wake that was killed and carried on
The worker was killed in the middle of this wake, just before a tool ran. It picked up from the journal and finished. The page shows the two checks as they ran when we shared it.
- An ordinary wake
One event, one model call, from early in the shift. The page shows the two checks as they ran when we shared it. Proofs from later in the run can't show the re-run check, because the journal is over its 5,000-event limit.
What it doesn't show.
The scenario is generated. A seeded program wrote the event stream from templates a model drafted: orders, refund requests, emails, deliveries, price and policy changes, and CEO messages. Most events (781 of 900) are routine filler. The facts and the questions about them were placed by the program, and the same seed gives the same shift.
Grading is mechanical. A callback question, or a shipping, delivery, refund, price or stock check, counts as right when the agent's tool-call arguments equal the scenario's ground truth; a delivery also needs each promised refund paid exactly once, and not before it. The 363 answer-only events (directives, newsletters, customer messages, emails) pass if the agent sent any answer. No model judged anything. A right answer that was worded or called differently from what the grader expects counts as wrong.
Few questions per distance. There are 12 to 29 callback questions in each distance band, so a single band's score can move by several points on luck alone. The whiskers on the chart show that range.
The agent has no lookup tool. It cannot query customers, policies or balances, so a fact stated once lives in the run's journal and the prompt the engine rebuilds. The prompt also carries short checkpoint notes the agent writes itself, and those are part of what is being tested.
A stalled event would count as a miss. If the agent had gone 40 steps without answering an event, the harness would have skipped it and scored any question attached to it as wrong. That never happened in this run. The agent also had no human-written notes: everything it knew came from the stream and from what it wrote down itself.
Our test machine restarted twice. The container running our harness went down for about 75 minutes in total early in the shift (23:05 to 00:04 UTC and 00:09 to 00:26 UTC). The harness relaunched from its saved state and the hosted journal kept going, so the event stream resumed where it stopped, but those gaps are inside the clock time quoted here.
The re-run check only works on early wakes. The agent's journal ended at 7,252 events. The check that re-runs a wake through the engine refuses on a journal over 5,000 events, so the two signed wake proofs below were shared early in the run, while it still could.
Company pace, not a throughput test. Events were fed in at about one every 48 seconds, because the tenant's journal has a size cap. It shows memory over hours, not how fast the engine can go.
We killed the worker, not the engine. The kills hit the process that calls the model and runs the tools. The hosted engine and its journal kept running. We did not test a kill during a prompt rebuild.
No baseline. We did not run a plain transcript loop on this shift, so the page says nothing about whether the engine remembered more, or used fewer tokens, than another setup.
One model, one run. Claude Haiku 5.5, once, on one seed. A different model or scenario could behave differently.
30 calls have no token counts. A kill landed before the numbers for 30 of the 982 calls were written down. They are in the CSV without counts, and every token and cost figure here covers the other 952.
Method.
- Model
- Claude Haiku 5.5 (Anthropic), called through our hosted engine in production, on a fresh tenant. The agent was ops-ops12h.
- Compared against
- None. Recall is compared across distances within the run: questions with the fact in view against questions hours later. We did not run a plain loop on the same shift.
- Sample
- One run: 900 scenario events from one seed, 91 callback questions in five distance bands, 982 model calls (952 with token counts), 1,842 actions and 31 kills in 11 hours 59 minutes, from 2026-10-27 23:00 UTC.
- Source
- The run harness's own event, metrics and grading logs, archived with the run. The per-call, per-kill, per-action and per-event rows are downloadable below and recount with recount.mjs.
Every row.
One line per question or run, with what the model answered where the benchmark allows it and whether we counted it right. Recount the accuracy and crash-test counts above with recount.mjs, and read the column notes.
- Every wake of the long run, one row each982 rows · 118 KB · sha256 104695aec923
- Every kill of the long run, one row each31 rows · 2 KB · sha256 1490ec13558d
- Every action of the long run, one row each1,842 rows · 56 KB · sha256 96d997f16f71
- Every graded event of the long run, one row each900 rows · 31 KB · sha256 fcfac9adf235
Every claim on this site, what backs it and where we lost: Check our work.
Written up on the blog: Argentic and Temporal: two ways to make an agent survive a crash.
More research.
We killed it 504 times
504 kills across 422 agent sessions in the first run (496 in a rerun with downloadable rows), at every point in a step we could reach. No finished action ran again. A frozen worker, which is not a kill, did repeat a refund in 2 of 22 runs, and in 3 of 22 when we ran it again.
0finished actions repeated after 504 kills in the first run, and after 496 in the rerun
· Finished
Our scores next to the published ones
LoCoMo and LongMemEval: our runs (Gemma 4 31B, plus Claude Haiku 5.5 on LongMemEval_S) beside scores that memory vendors and paper authors report, with the answer model and the judge for each row, or "not stated" where none was given. On a fixed 400-question subset of LoCoMo, the Argentic engine scored 83.0%, and the same model reading the whole conversation scored 89.8%.
83.0%on 400 of the 1,540 LoCoMo questions, with a free open model
· Finished
Ten times the history, still 38 of 60
On LongMemEval_M, with histories of about 1.2 million tokens, the engine got 38 of 60 right with Gemma 4 31B and 29 of 60 with Claude Haiku 5.5. On Gemma 4 31B, a loop that rewrites notes got 17 of 60 and a loop that truncates got 5 of 60.
38 / 60correct for the Gemma 4 31B engine (both runs) at ~1.2M tokens of history
· Finished
Measured on someone else's tasks
Running the engine on τ-bench, a public benchmark. An early gap against the plain loop was most likely a bug in our runner, which counted failed model calls as the agent quitting. Rerun without rate limiting, the engine scored 13 against the loop's 12 on 18 pairs; the loop was not rerun and the engine is newer, so this is not a shown tie.
13 vs 12of 18 pairs, engine rerun against loop
· Ongoing
A fraction of the tokens, at a small accuracy cost
Our engine against a model reading a whole chat history, on LongMemEval and four model families. It was never ahead on accuracy and used 0.08 to 0.31 times the tokens. On all 500 questions with Claude Haiku 5.5 it scored 354 against the loop's 373, at about 1/24 of the cost.
0.08–0.31×the tokens of a loop reading the whole history, on LongMemEval_S (49 questions, four models). Never ahead on accuracy
· Finished
Two agents, one order
Settlement stopped 11 conflicting runs out of 50 with Claude Haiku 5.5 on permissive tools, and removed no bad outcomes where the tools already checked state. Majority vote over three branches was not significantly better than one agent; a verifier that checked the branches was, though on this task family a plain rule did just as well.
11 → 0bad runs of 50 with Claude Haiku 5.5 and permissive tools. On Nemotron, 32 of 36 runs stayed bad.
· Ongoing
Undo what a bad source caused
When a source is flagged after the run, a host that calls the engine's quarantine step undoes the calls the journal links to it, using undo functions the host supplies, and resumes the agent without it. It is not in the hosted API. Effects with no undo function are refused unless allowed. The agent finished without the source in 156 of 156 runs: 120 scripted, and 36 live, 24 of them with the injection forced.
156 / 156runs recovered, most of them scripted
· Finished
Rebuild the context, don't let it pile up
We rebuild an agent's context from a journal instead of keeping a growing transcript. What that buys, what it costs, and what we haven't tested yet.
97%of planted changes correct in the final documents, against 72% for a loop that hides old results and keeps notes (12 paired runs, DeepSeek V4.1 Flash)
· Ongoing