Chidori V3 is out. This is the biggest single release since the runtime moved off the prototype graph engine: a new content-addressed replay cache, an expanded Starlark standard library, and a serve loop that finally treats backpressure and HITL resume the way we always wanted it to.
Most of the work this cycle was unglamorous — squeezing tail latency out of cache lookups, plugging holes in determinism guarantees, cleaning up the surface area that early adopters kept tripping on. The headline is simple: replays are roughly 6× faster, and there are a lot fewer ways to write a non-deterministic agent by accident.
What's new
Content-addressed replay cache
The V2 cache keyed every host call by sequence number. That made replays trivially correct but rebuilt the entire trie on every cold start. V3 keys cache entries by a hash of (host_fn, args, parent) so identical subtrees across runs share storage on disk and in memory. The practical effect: cold-start replay of a 12k-span research session dropped from ~3.4s to ~520ms on our reference box.
Cache files are still plain JSON-Lines — open them in any editor, diff them in CI, ship them around as artifacts. Nothing about the on-disk shape requires the runtime.
Starlark standard library
We pulled a handful of things out of host functions and back into pure Starlark, where they belong:
std.json,std.url,std.text— parsing, encoding, slugifying, basic regex. No side effects.std.struct— typed, hashable records you can use as cache keys without writing a serializer.std.retry— deterministic retry combinators built on top ofpromptandtool. The retry policy is part of the call log, so a replayed run reproduces the exact same retry ladder.
Anything in std is guaranteed to be pure: no clocks, no randomness, no hidden I/O. If you can call it, you can replay it.
Serve loop: backpressure and resume
chidori serve now treats each session as a long-lived actor with a bounded inbox, instead of spawning a fresh task per request. Three things fall out of that:
- Bursty clients get a 429 with a
Retry-Afterinstead of silently queueing megabytes of pending events. input()-suspended sessions checkpoint to disk on shutdown and pick back up at the same sequence number when the process restarts. Rolling deploys no longer lose paused HITL flows.- The serve binary is ~38% smaller because we deleted the old per-request scheduler.
Breaking changes
Two, both small, both worth flagging:
- The
prompt(...)host function no longer accepts a bare string as its second positional argument. Passmodel=explicitly. The old form was ambiguous once we added structured output configs. - Cache files written by V2 are readable but not writable by V3. Run
chidori cache migrateonce per session directory; it's a no-op on already-migrated files.
Upgrading
$ cargo install chidori --version 3.0 $ chidori cache migrate ./sessions $ chidori serve agents/researcher.star --port 8080
Full notes are on the GitHub release. As always, the fastest way to flag a regression is to drop a minimal .star file plus its session JSON in Discord — replay means we can reproduce your run on our box without spending a token.