Reflexes

Use Morph tracing + Reflexes — fast text classifiers at api.morphllm.com — to find and fix this agent's real failures.

STEP 1 — Find the prompts.
Find where this codebase stores its past conversations — a conversation table, trace store, or log files. Pull the user and assistant turns from the last 30 days. If there are more than 3,000 turns, take a uniform random sample of 3,000.

STEP 2 — Pick the Reflexes.
First understand what this product does, then pick the 2–4 Reflexes that would surface its most damaging failures. The defaults (model name — classes — what it catches):
- jailbreak — benign/jailbreak — prompt-injection and jailbreak attempts
- guardrail — true/false — harassment or NSFW content
- leaked-thinking — clean/leaked — agent leaking its internal thinking
- stuck-in-a-loop — progressing/looping — agent blocked, retrying the same thing
- incomplete-thought — complete/incomplete — user sent a truncated prompt
- user-frustrated — frustrated/not — user frustrated with the agent
- user-joy — joy/not_joy — user delighted with the agent
- ambiguity — low/med/high — how underspecified a prompt is
- difficulty — easy/medium/hard — prompt difficulty
- domain — general/summary/coding/design/data — topic (multi-label)
- health-emergency — emergency/non-emergency — message needs urgent attention

Match the Reflex to the use case and the turn role: to see if users are frustrated, run user-frustrated on user turns; a public-facing chatbot should run jailbreak and guardrail on user turns; a coding or tool-using agent should run stuck-in-a-loop and leaked-thinking on assistant turns; a healthcare or patient-facing product should run health-emergency on user turns. Caveat: each turn is classified on its own text, so cross-turn patterns (an agent repeating the same answer across turns) may not fire — treat stuck-in-a-loop hits as extra signal, not exhaustive.

STEP 3 — Replay the history as Morph traces.
Authenticate with the MORPH_API_KEY env var. If it isn't set, look for a Morph key in the repo's env files; otherwise ask me for one (keys are created at https://morphllm.com/dashboard/api-keys) — never hardcode it. Write a small script with the Morph SDK that replays each sampled turn as a traced interaction, with the step-2 Reflexes in evals — "user" Reflexes classify the user message, "assistant" Reflexes the agent's output. Install: npm install @morphllm/morphsdk, or pip install 'morphsdk[otel]':

import { morphTracing } from "@morphllm/morphsdk/tracing";
const morph = morphTracing({ apiKey: process.env.MORPH_API_KEY, disableBatching: true });

const replayed = [];
for (const t of turns) {
  const turn = morph.begin({
    userId: t.userId, convoId: t.convoId, event: "backfill",
    evals: { user: ["user-frustrated", "jailbreak"], assistant: ["stuck-in-a-loop"] },
  });
  turn.setInput(t.userText);
  await turn.finish({ output: t.assistantText });
  replayed.push({ eventId: turn.getEventId(), convoId: t.convoId, turnId: t.id });
}

(Python: from morphsdk.tracing import morph_tracing, then morph.begin({...}) / turn.set_input(...) / turn.finish({"output": ...}) — same fields, snake_case, "disable_batching": True.)

Keep each turn's real convo_id and user_id so conversations thread together in the dashboard. Record every replayed turn's event_id — it's the join key the labels attach to. Morph classifies each turn asynchronously after the span lands, off the request path, and stores the results on the trace.

STEP 4 — Poll until every turn is labeled.
Replayed turns are timestamped at replay time, so they're the newest rows. Page through them and check for labels:

curl "https://api.morphllm.com/v1/reflex/traces?limit=1000&offset=0" -H "Authorization: Bearer $MORPH_API_KEY"

Each row has convo_id, event_id, input_text, output_text, and reflex_results. Entries appear right away with "status": "pending", then flip to "completed" when the label lands (or "failed" — skip those). A turn is done when every Reflex you requested on it has a completed entry. Poll every 30s (don't hold a request open) until every recorded event_id is done — a small replay labels within a minute or two; a full 3,000-turn replay drains at a few rows per second and can take tens of minutes. The labeled turns are now stored on the account: browse them at https://morphllm.com/dashboard/traces, or pull them again any time from this same endpoint. Full reference: https://docs.morphllm.com/sdk/components/tracing#list-traced-turns

STEP 5 — Group the hits.
A hit is an entry whose label IS the failure class. Careful: selected names the winning class even when it's the benign one ("Not Frustrated", "false"), so non-empty selected is NOT a hit — match the label, case-insensitively, and expect the API's label strings to differ in case and wording from the class list above (user-frustrated returns "Frustrated" / "Not Frustrated"). Read the flagged turns before trusting them: discard obvious false positives, then group the real hits by label, then by what the user was trying to do.

STEP 6 — Root-cause with subagents.
For each group, spawn a subagent to read the full conversations around the flagged turns and trace the failure to a specific prompt, tool, or code path in this repo.

STEP 7 — Fix with subagents.
For each root cause fixable in this repo, spawn a subagent to implement the fix on its own branch and open a PR — one PR per root cause, citing the flagged conversations as evidence. If the repo has no git remote, prepare the branches and show me the diffs instead. Open one more PR that wires morphTracing + evals into this app's own turn loop (docs: https://docs.morphllm.com/guides/reflex-tracing), so every future turn is labeled as it happens and the next scan reads live traces instead of replaying.

Confirm with me before opening any PRs. Send conversation text only to the Morph API.