Rebuild the prompt,or let it pile up?
An early September test of rebuilding an agent's prompt from stored state against a growing transcript, on two long tasks and two models. The study reports the rebuilt arm produced more checked findings in all three cells, by less than we first claimed, and we retracted the claim that a growing context makes an agent worse.
Neil Luo · · Finished
This is a small early experiment from September 2026, run on a research harness and not on the Argentic engine. None of these numbers describe the engine as it works today.
Does rebuilding an agent's context from stored state beat letting a transcript pile up? This was our first, small test of that question, and part of what we first claimed did not hold up.
Both arms had the same model, tasks and tools, and on the site build the same 60-turn cap. One kept a growing transcript and cut it from the front at a ceiling. The other rebuilt its prompt at every task boundary from the goal, its decisions and its constraints. The study's own analysis says that in every cell it compared, the rebuilt arm produced more findings that checked out against the code. It also says the gap was much smaller once findings that named no file were left out, and the model mattered more than the context strategy.
3/3
Cells where the study reports the rebuilt arm produced more grounded findings per run than the transcript arm: the site build on GLM-4.7-Flash and on GLM-5.3-Flash, and the web app on GLM-5.3-Flash. On the first, which we recomputed from the public data, the rebuilt arm read about 0.40 million prompt tokens per run against 1.22 million for the transcript arm. The study reports 212 grounded findings against 68.
The study reports that first gap fell from 144 to 16 findings per run when only findings that named a file were counted, and that accuracy (how often a claim was right) did not separate on any cell. The claim that the transcript arm got worse past 30,000 tokens of context was retracted: on the public data the decline shows at a 30,000 cut and not at 25,000.
What it doesn't show.
Small. Nine runs per arm per cell, on two models from the same family. Some gaps are too wide to call either way.
The grounded counts do not reproduce from the public data. They need each run's written files, which the data export does not include. We recomputed the token and task counts from it, and the grounded counts come from the study's own published analysis.
A mechanical check. A finding counts as grounded when the function it names exists and sits in the file it names. That does not judge whether the reasoning was good.
Not the claim we set out to test. We could not show that a growing context makes an agent worse. The one test aimed at that failed, and we retracted it.
Not the engine. This ran in September on an early research harness. It is not how the Argentic engine works today, and we have not rerun it there.
Read it yourself.
The code, data and every retraction are public, and the plain-language write-up goes through what held up and what did not. Start with the repo's limitations file before you trust a number.
Method.
- Model
- GLM-4.7-Flash and GLM-5.3-Flash (Z.ai), one model per cell.
- Compared against
- A transcript that accumulates and is cut from the front at a token ceiling.
- Sample
- Nine runs per arm per cell. Three cells compared: the site build on both models and the web app on GLM-5.3-Flash. Findings counted as grounded by a mechanical check of the named function and file. Token and task counts recomputed from the public data; grounded counts are the study's own, because the export lacks the written files they need.
- Source
- The public study repository, github.com/Argentic-AI/reprojection-study (code Apache-2.0, data CC BY 4.0). The raw rows are in its data folder.
Every claim on this site, what backs it and where we lost: Check our work.
Written up on the blog: Does a growing transcript hurt a long agent run? Our early test, and the part we retracted.
More research.
The shop shift, rerun in November
We reran the 900-event shop shift in November on Claude Haiku 5.5, on engines 0.51.0, 0.63.0 and 0.70.1. Earlier facts, owed refunds and double refunds held in every run where the engine kept the state. Stock exact ranged from 10 to 22 of 28 with the configurations tried, and was 83 of 83 with clearer count wording, against 48 of 83 without it. One run per variant, one scenario.
83 of 83stock answers exact with clearer count wording, against 48 of 83 with the baseline wording; three seeds, one run each, engine 0.70.1
· Ongoing
11 hours 59 minutes on shift: what the agent remembered
One agent ran the back office of a small online shop for 11 hours 59 minutes of clock time, about 75 minutes of it with our test harness down, on Claude Haiku 5.5, handling 900 of 900 events while its worker was killed 31 times. It answered 81 of 91 callback questions right, including facts stated about 11 hours earlier. No kill made a finished action run again. It did refund 7 orders more than once, which no kill caused.
83% to 92%right with the fact in view, against the farthest facts
· Finished
HaluMem Long: our accuracy lead did not replicate
On 120 HaluMem Long questions our experimental memory pipeline answered 76.7% correctly against plain RAG's 62.5%. That lead did not replicate on two fresh question sets, and the run failed our own pre-registration: it used 1.7 times plain RAG's prompt tokens against a bar of 0.85.
FailedThe +14.2 point first-run lead over plain RAG did not replicate on two fresh question sets, and the run missed its own token bar.
· Finished
We killed it 504 times
504 kills across 422 agent sessions in the first run (496 in a rerun with downloadable rows), at every point in a step we could reach. No finished action ran again. A frozen worker, which is not a kill, did repeat a refund in 2 of 22 runs, and in 3 of 22 when we ran it again.
0finished actions repeated after 504 kills in the first run, and after 496 in the rerun
· Finished
Our scores next to the published ones
LoCoMo and LongMemEval: our runs (Gemma 4 31B, plus Claude Haiku 5.5 on LongMemEval_S) beside scores that memory vendors and paper authors report, with the answer model and the judge for each row, or "not stated" where none was given. On a fixed 400-question subset of LoCoMo, the Argentic engine scored 83.0%, and the same model reading the whole conversation scored 89.8%.
83.0%on 400 of the 1,540 LoCoMo questions, with a free open model
· Finished
Ten times the history, still 38 of 60
On LongMemEval_M, with histories of about 1.2 million tokens, the engine got 38 of 60 right with Gemma 4 31B and 29 of 60 with Claude Haiku 5.5. On Gemma 4 31B, a loop that rewrites notes got 17 of 60 and a loop that truncates got 5 of 60.
38 / 60correct for the Gemma 4 31B engine (both runs) at ~1.2M tokens of history
· Finished
Measured on someone else's tasks
Running the engine on τ-bench, a public benchmark. An early gap against the plain loop was most likely a bug in our runner, which counted failed model calls as the agent quitting. Rerun without rate limiting, the engine scored 13 against the loop's 12 on 18 pairs; the loop was not rerun and the engine is newer, so this is not a shown tie.
13 vs 12of 18 pairs, engine rerun against loop
· Ongoing
A fraction of the tokens, at a small accuracy cost
Our engine against a model reading a whole chat history, on LongMemEval and four model families. It was never ahead on accuracy and used 0.08 to 0.31 times the tokens. On all 500 questions with Claude Haiku 5.5 it scored 354 against the loop's 373, at about 1/24 of the cost.
0.08–0.31×the tokens of a loop reading the whole history, on LongMemEval_S (49 questions, four models). Never ahead on accuracy
· Finished
Two agents, one order
Settlement stopped 11 conflicting runs out of 50 with Claude Haiku 5.5 on permissive tools, and removed no bad outcomes where the tools already checked state. Majority vote over three branches was not significantly better than one agent; a verifier that checked the branches was, though on this task family a plain rule did just as well.
11 → 0bad runs of 50 with Claude Haiku 5.5 and permissive tools. On Nemotron, 32 of 36 runs stayed bad.
· Ongoing
Undo what a bad source caused
When a source is flagged after the run, a host that calls the engine's quarantine step undoes the calls linked to it, using undo functions the host supplies, and resumes the agent without it. It is not in the hosted API. Effects with no undo function are refused unless allowed. The agent finished without the source in 156 of 156 runs: 120 scripted, and 36 live, 24 of them with the injection forced.
156 / 156runs recovered, most of them scripted
· Finished
Rebuild the context, don't let it pile up
We rebuild an agent's context from a recorded history instead of keeping a growing transcript. What that buys, what it costs, and what we haven't tested yet.
97%of planted changes correct in the final documents, against 72% for a loop that hides old results and keeps notes (12 paired runs, DeepSeek V4.1 Flash)
· Ongoing