Skip to content
MorrowCache
Get started
2.0Agent Turn Cache is live

Skip the modelwhen the question isrephrased, retried, or the same.

A local OpenAI-compatible proxy. When a request means the same thing, or your agent retries the same job, MorrowCache returns the stored final answer. Tool calls never replay.

No account. Keys stay in .env. Full guide

GitHub stars
12
npm downloads, all time
1,172
Tests in repo
89
Miss vs hit, in ms
2817 → 423

The bill

Your agents ask the same thing in different words, retry the same job, and pay the model every single time.

Request 1

explain mutexes

Request 2

please explain mutexes

Hashes differ

X-Jevcache: MISS

01

Exact cache misses rephrases

Safe, but blind to wording. Every rephrase pays the model again.

Asked

latest node release?

Matched on shared words

node release notes

Shared words, different question

Wrong hit

02

Word overlap serves wrong answers

Fast and noisy. Offline Jaccard@0.35 false-positive rate: 0.49 (n=103).

Request 1

Explain mutexes simply please

Request 2

Please explain mutexes simply

Same intent, 0.94

X-Jevcache: HIT

03

MorrowCache checks the intent

A judge admits same intent. Freshness refuses. Unsure means miss.

Replay

Watch a paraphrase skip the model.

The first ask pays the full round trip. The rephrase is judged, admitted and served from the cache. The model never hears about it.

POST 127.0.0.1:8080/v1/chat/completions
01waiting

›

    02waiting

    ›

      Wall-clock, same answer

      Miss, model called0ms
      Hit, model skipped0ms

      6.7×

      One recorded paraphrase pair at intent 0.94. Real timings, played back at real speed. Not a benchmark, not an SLA.

      How Morrow decides

      Four checks between your agent and the bill.

      Exact cachesha-256 of request bytes

      incoming

      › explain mutexes simply please

      9f2c…a71eexplain mutexes simply pleasematch
      41d0…07bbwrite a regex for iso dates
      c3a8…5e19summarize this stack trace
      77be…d204what does EADDRINUSE mean
      upstreamnot calledX-Jevcache: HIT

      The exact repeat

      Same request bytes, same stored completion. The cheapest hit there is, and the model is never called.

      • 01Hash the request
      • 02Serve the stored completion
      • 03Upstream never sees it
      Adjudicatorshortlist proposes, judge admits

      incoming

      › Please explain mutexes simply

      shortlist
      • Explain mutexes simply pleaseadmit 0.94
      • Explain semaphores simplyreject
      • Mutex vs spinlock in Rust?reject
      similarity only proposes. it never admits alone.

      Same question, new words

      A shortlist proposes candidates. An adjudicator decides whether they really mean the same thing. Similarity never admits on its own.

      • 01Shortlist proposes
      • 02Judge admits or rejects
      • 03Cloud Jev or a local judge
      Agent Turn CacheX-Jevcache-Job-Id: refactor-17

      attempt 1

      system: repo context

      user: earlier turns

      user: add tests for parser.ts

      miss, model called

      attempt 2, a retry

      system: repo context

      user: earlier turns

      user: write tests for parser.ts

      hit, final text

      prior

      identical

      latest ask

      same intent

      stream

      replayed

      The agent retries the job

      Agent Turn Cache: same prior turns plus a same-intent latest ask returns the stored final text, including a later stream of that answer.

      • 01Same prior
      • 02Same-intent latest ask
      • 03Final text, streamed or not
      Decision logprefer a miss over a wrong hit
      • what's the latest node release?freshness question, refusedrefuse
      • response carries tool_callsnever stored, the tool step still runsmiss
      • adjudicator unreachablefails open, the real model answersmiss
      • please explain mutexes simplysame intent, admittedhit

      Refuses, or falls back to the model

      Freshness questions refuse. Tool-call responses are never stored, so tool steps still run. If the judge is down, it fails open.

      • 01Refuse stale answers
      • 02Never replay tool calls
      • 03Fail open, always

      Get started

      Running in minutes.

      Paste one prompt into Cursor, Claude Code or Codex. Your agent installs MorrowCache 2.0, writes .env, points baseURL at the proxy and proves a miss, then a hit. Your keys stay in your local .env.

      Full agent guide →
      1. 1

        Install and check

        Node 22.5+. The doctor checks your environment.

        npx @kushalicious/jevcache@latest doctor
      2. 2

        Point your client

        Keep your model. Change one line.

        baseURL: "http://127.0.0.1:8080/v1"
      3. 3

        Read the header

        Ask twice in different words.

        X-Jevcache: MISS → X-Jevcache: HIT

      Your stack

      Sits in front of what you already use.

      Upstream

      • OpenAI
      • OpenRouter
      • Cursor
      • Claude*
      • Local OpenAI-compat

      Judge

      • Jev (cloud)
      • Kev (local)
      • Laya (local)
      • Laya MLX
      • Any System One

      * Claude, when the client speaks an OpenAI-compatible HTTP API. Not a customer logo row.

      FAQ

      Common questions.

      Yes. Point the proxy at your upstream baseURL and your SDK at http://127.0.0.1:8080/v1. Upstream can be OpenAI, OpenRouter, or a local OpenAI-compatible server. Claude works when the client speaks that HTTP API. Chat Completions only: /v1/responses returns 501.

      Jev is the default same-intent judge and needs OPENROUTER_API_KEY. Or set ADJUDICATOR=kev|laya|laya-mlx|systemone for a local System One, and admits need no OpenRouter key. Without a judge ready, the proxy runs exact-only.

      Fail-open. The real model still runs. A miss is always preferred over a wrong hit.

      Yes, wrong hits can happen. Tune the threshold, respect freshness, and read the X-Jevcache header. Final text can hit, including a later stream of that answer. Tool-call responses are never stored. TURN_CACHE is on by default for agent retries; TURN_CACHE=off restores the 1.x stream/tools bypass.

      Stars and npm downloads are fetched live, with a fallback if the fetch fails. 89 is the test count in this repo, not users or a benchmark. 2817ms → 423ms is one recorded run at intent 0.94. The 0.49 false-positive rate is offline word-overlap Jaccard@0.35, n=103: that is the overlap cache, not a MorrowCache rate.

      Skip the
      model.

      Paste the prompt, ask the same thing twice, and check the X-Jevcache header.

      Star on GitHub