Agent Runs

Agent-trace-aware prompt caching. Tag every request in a tool-calling loop with one run_id and the run is scheduled as a single unit, pinned to the worker holding its KV cache. Each turn prefills only what changed since the last one, at the 90%-off cached-input rate. Untagged, every turn looks like an unrelated request. It lands on any free worker and re-prefills the whole conversation at full price. Tagged runs get:

Sticky placement. Every turn routes to the worker that already holds the run’s cache.
Priority resume. A run coming back from a tool call is admitted ahead of new arrivals.
Whole-run admission. Under load, whole runs pause instead of every run getting slow.

Program-aware scheduling, from the ThunderAgent paper. Adopting it is one field.

Available on Kimi K3 (morph-kimik3, morph-kimik3-fast). Requests without a run id are unaffected: they route exactly as they do today, on the same workers.

Quick Start

cURL
Python
TypeScript

curl -X POST "https://api.morphllm.com/v1/chat/completions" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "morph-kimik3",
    "messages": [{"role": "user", "content": "..."}],
    "run_id": "run-8f2c1a"
  }'

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.morphllm.com/v1",
)

# Non-standard fields go via extra_body
response = client.chat.completions.create(
    model="morph-kimik3",
    messages=[{"role": "user", "content": "..."}],
    extra_body={"run_id": "run-8f2c1a"},
)

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: "YOUR_API_KEY",
  baseURL: "https://api.morphllm.com/v1",
});

const response = await client.chat.completions.create({
  model: "morph-kimik3",
  messages: [{ role: "user", content: "..." }],
  // @ts-expect-error non-standard field
  run_id: "run-8f2c1a",
});

A multi-turn loop

Send the same run_id on every turn of one run, then release it when the run ends:

import uuid

run_id = f"run-{uuid.uuid4().hex[:8]}"
messages = [{"role": "system", "content": SYSTEM_PROMPT}, *task]

while True:
    response = client.chat.completions.create(
        model="morph-kimik3",
        messages=messages,
        tools=TOOLS,
        extra_body={"run_id": run_id},
    )
    choice = response.choices[0]
    messages.append(choice.message)

    if not choice.message.tool_calls:
        break
    for call in choice.message.tool_calls:
        messages.append(run_tool(call))   # the gap the scheduler holds cache across

# Release the run. Returns immediately, generates nothing.
client.chat.completions.create(
    model="morph-kimik3",
    messages=[{"role": "user", "content": ""}],
    extra_body={"run_id": run_id, "run_final": True},
)

The release call is not forwarded to the model. It drops the run from the scheduler’s table and returns an empty completion, so it generates no output tokens and is not billed for any. Send it even on the error path. Releasing frees the run’s slot immediately. A run that is never released is reclaimed on its own once it has gone 15 minutes without a request, so a crashed agent doesn’t leak capacity permanently, it just holds a slot until the sweep catches it. A run with a request in flight is never reclaimed.

Reference

Three top-level body fields, all sent the way any non-standard field is (extra_body in the Python SDK):

Field	Type	Required	Meaning
`run_id`	string	yes	Identifies the run. Every request sharing it is scheduled as one unit.
`parent_run_id`	string	no	The parent run, for a subagent spawned by another agent.
`run_final`	bool	no	Terminal marker. Releases the run without calling the model.

program_id, parent_program_id, and program_final are accepted as drop-in aliases, so clients written to the ThunderAgent convention (SkyRL, OpenHands) work unchanged. Ids must be printable ASCII, and are capped at 256 characters. An id that is empty, whitespace-only, or carries a control or non-ASCII character counts as absent: the request is served untagged rather than rejected, because a bad scheduler hint should cost you the optimization and never the request. A run_final request with no run_id has nothing to release and is served as an ordinary request.

Subagents

A fan-out gets one run per subagent, each naming the run that spawned it. Every branch is scheduled on its own, with its own sticky worker, so a subagent that takes several turns keeps its cache across them the same way a root run does. Give each branch a distinct id: reusing the parent’s id across concurrent branches makes the scheduler treat them as one unit and pin them all to one worker. parent_run_id records lineage. It is what ties a fan-out together in traces and attribution; it does not currently pull a child toward its parent’s worker.

import asyncio
from openai import AsyncOpenAI

client = AsyncOpenAI(api_key="YOUR_API_KEY", base_url="https://api.morphllm.com/v1")
root = "run-8f2c1a"

async def subagent(task: str, index: int):
    return await client.chat.completions.create(
        model="morph-kimik3",
        messages=[{"role": "system", "content": SYSTEM_PROMPT},
                  {"role": "user", "content": task}],
        extra_body={
            "run_id": f"{root}-sub-{index}",
            "parent_run_id": root,
        },
    )

results = await asyncio.gather(*(subagent(t, i) for i, t in enumerate(tasks)))

Best practices

One id per run, generated at the start. A UUID or your own trace id. Reuse it for every turn including retries of the same turn.
Never share an id across unrelated runs. The scheduler will pin them to one worker and account for them together, so they fight over the same cache.
Always send the release. One fire-and-forget call in whatever cleanup path your agent already has, ideally the finally rather than the happy path. The 15-minute reclaim is a backstop for crashes, not a substitute: until it fires, the run is still holding a slot.
Keep the prefix stable too. Stickiness gets your turns to the worker holding the cache; prefix caching is what makes those tokens cheap. On Kimi K3 cached input is 90% off, $0.29/1M against$ 2.90. A multi-turn loop with a stable prefix and a run id is the case both are built for.
Tag from the agent, not the gateway. The id has to follow one logical run. A per-process or per-user id collapses concurrent runs into one.

Pitfalls

cached_tokens didn't improve after tagging

Tagging keeps the stable prefix cached across turns; a changed prefix still misses the cache regardless of tagging

Tagging routes turns to the worker holding the cache; it does not make a changing prefix cacheable. Check prompt_tokens_details.cached_tokens on an untagged run first: if it was already near zero, the problem is prefix stability, not placement. See Prompt Caching.

Latency got worse under load

Under load the scheduler pauses whole runs at admission rather than slowing every worker down

Expected shape when runs are paused: the scheduler holds entire runs rather than degrading all of them. A paused run’s next turn waits for admission. If you need every request admitted immediately regardless of cache, don’t tag.

The release call returned an empty response

A run_final request short-circuits at the scheduler table and never reaches the model

That is the contract. run_final short-circuits before the model, so there are no choices and no completion tokens. Don’t parse it for content.

Get Started

Models

API Reference

Integrations

Quick Start

A multi-turn loop

Reference

Subagents

Best practices

Pitfalls

See Also

​Quick Start

​A multi-turn loop

​Reference

​Subagents

​Best practices

​Pitfalls

​See Also

Quick Start

A multi-turn loop

Reference

Subagents

Best practices

Pitfalls

See Also