Batch API

Upload a JSONL file of chat-completion requests, create a batch over it, and collect the results when it finishes. Every completed request is billed at half the model’s synchronous rate. The API is the OpenAI Batch API, so the official OpenAI SDKs work unchanged once you point them at https://api.morphllm.com/v1. Available on morph-glm53flash, morph-dsv4flash, morph-glm53-744b, and morph-kimik3. Per-token rates are on the pricing page.

When to use it

Batch fits work where nobody is waiting on an individual response: eval runs, dataset generation, nightly summarization, re-indexing, backfills. You trade latency for price. Results land within 24 hours, usually much sooner, and there is no per-request rate limit to work around. For work that needs a response now, use the synchronous endpoint. For background traffic that still needs each answer within seconds, use Standby instead.

Quick Start

Write one request per line in a JSONL file. Each line names a custom_id you choose, the endpoint, and the same body you would send synchronously.

requests.jsonl

{"custom_id": "req-0001", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "morph-glm53flash", "messages": [{"role": "user", "content": "Summarize this diff in one sentence: ..."}], "max_tokens": 256}}
{"custom_id": "req-0002", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "morph-glm53flash", "messages": [{"role": "user", "content": "Summarize this diff in one sentence: ..."}], "max_tokens": 256}}

Then upload the file, create the batch, poll until it reaches a terminal status, and download the output and error files.

Python
TypeScript
cURL

import time
from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.morphllm.com/v1",
)

# 1. Upload the input file
input_file = client.files.create(
    file=open("requests.jsonl", "rb"),
    purpose="batch",
)

# 2. Create the batch
batch = client.batches.create(
    input_file_id=input_file.id,
    endpoint="/v1/chat/completions",
    completion_window="24h",
    metadata={"job": "nightly-summaries"},
)

# 3. Poll until terminal
TERMINAL = {"completed", "failed", "expired", "cancelled"}
while batch.status not in TERMINAL:
    time.sleep(60)
    batch = client.batches.retrieve(batch.id)
    print(batch.status, batch.request_counts)

# 4. Download results. Either file id can be null if nothing landed in it.
if batch.output_file_id:
    client.files.content(batch.output_file_id).write_to_file("output.jsonl")
if batch.error_file_id:
    client.files.content(batch.error_file_id).write_to_file("errors.jsonl")

import fs from "node:fs";
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: "YOUR_API_KEY",
  baseURL: "https://api.morphllm.com/v1",
});

// 1. Upload the input file
const inputFile = await client.files.create({
  file: fs.createReadStream("requests.jsonl"),
  purpose: "batch",
});

// 2. Create the batch
let batch = await client.batches.create({
  input_file_id: inputFile.id,
  endpoint: "/v1/chat/completions",
  completion_window: "24h",
  metadata: { job: "nightly-summaries" },
});

// 3. Poll until terminal
const TERMINAL = new Set(["completed", "failed", "expired", "cancelled"]);
while (!TERMINAL.has(batch.status)) {
  await new Promise((r) => setTimeout(r, 60_000));
  batch = await client.batches.retrieve(batch.id);
  console.log(batch.status, batch.request_counts);
}

// 4. Download results. Either file id can be null if nothing landed in it.
if (batch.output_file_id) {
  const output = await client.files.content(batch.output_file_id);
  fs.writeFileSync("output.jsonl", await output.text());
}
if (batch.error_file_id) {
  const errors = await client.files.content(batch.error_file_id);
  fs.writeFileSync("errors.jsonl", await errors.text());
}

# 1. Upload the input file
curl https://api.morphllm.com/v1/files \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -F purpose=batch \
  -F file=@requests.jsonl
# => {"id": "file_...", "object": "file", "purpose": "batch", ...}

# 2. Create the batch
curl https://api.morphllm.com/v1/batches \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input_file_id": "file_...",
    "endpoint": "/v1/chat/completions",
    "completion_window": "24h"
  }'
# => {"id": "batch_...", "status": "validating", ...}

# 3. Poll until status is completed, failed, expired, or cancelled
curl https://api.morphllm.com/v1/batches/batch_... \
  -H "Authorization: Bearer YOUR_API_KEY"

# 4. Download the output file (and error_file_id, if set)
curl https://api.morphllm.com/v1/files/file_.../content \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -o output.jsonl

Input file format

One JSON object per line. Blank lines are not allowed.

Field	Value
`custom_id`	Your key for the request. Unique within the batch, echoed on the matching output or error line. No length cap.
`method`	Always `"POST"`.
`url`	Always `"/v1/chat/completions"`, matching the batch’s `endpoint`.
`body`	A Chat Completions request body. Everything the synchronous endpoint accepts works here, except `stream: true`, which is rejected.

Every line in a batch must name the same model. A file that mixes models, repeats a custom_id, or targets another endpoint fails validation and the batch goes to failed with the reasons in errors.

Output file format

One line per request the model answered, in completion order rather than input order. Join back to your requests on custom_id.

{
  "id": "batch_req_1a2b3c4d-5e6f-4a7b-8c9d-0e1f2a3b4c5d",
  "custom_id": "req-0001",
  "response": {
    "status_code": 200,
    "request_id": "req_3f9a2c1b7d",
    "body": {
      "id": "chatcmpl-7d2f1c3a9b",
      "object": "chat.completion",
      "model": "morph-glm53flash",
      "choices": [{ "index": 0, "message": { "role": "assistant", "content": "..." }, "finish_reason": "stop" }],
      "usage": { "prompt_tokens": 412, "completion_tokens": 18, "total_tokens": 430 }
    }
  },
  "error": null
}

response.body is the full Chat Completions response, usage included. error is always null in this file.

Model errors are output lines, not error lines. When the model returns a non-2xx for a request (a 400 for a bad parameter, a 413 for an oversized prompt, a 500), the line is written to the output file with that status_code and the error envelope in response.body, and error stays null. Check status_code on every output line rather than assuming the file only holds successes. OpenAI routes these to the error file; Morph does not.

Error file format

One line per request that never got a model response. response is always null here.

{
  "id": "batch_req_9f8e7d6c-5b4a-4392-8170-6f5e4d3c2b1a",
  "custom_id": "req-0002",
  "response": null,
  "error": {
    "code": "batch_expired",
    "message": "This request could not be executed before the completion window expired."
  }
}

`error.code`	Meaning
`timeout`	The request was sent but no response arrived in time.
`batch_cancelled`	The batch was cancelled before this request ran.
`batch_expired`	The 24-hour window closed before this request ran.
`batch_failed`	The batch hit a system error before this request ran.
`model_not_found`	The `model` in this line is not available for batch.
`parse_error`	The line, or the model’s reply to it, was not valid JSON.

Lines in the error file are not billed.

Statuses

Status	Terminal	Meaning
`validating`		The input file is being checked: JSON per line, one model, unique `custom_id`, supported endpoint.
`in_progress`		Requests are running. `request_counts` advances.
`finalizing`		Every request has finished; the output and error files are being written.
`completed`	yes	Done. `output_file_id` and `error_file_id` are set where they have lines.
`failed`	yes	Validation failed or the batch hit a system error. Reasons are in `errors`; anything that completed first is still in the output file.
`expired`	yes	The 24-hour window closed. Completed requests are in the output file; the rest are in the error file as `batch_expired`.
`cancelling`		A cancel was requested; in-flight requests are draining.
`cancelled`	yes	Cancel finished. Completed requests are in the output file; the rest are in the error file as `batch_cancelled`.

Poll GET /v1/batches/{batch_id} every 30 to 60 seconds. There is no webhook.

Cancel and expiry

Cancel with POST /v1/batches/{batch_id}/cancel while the batch is validating or in_progress. It moves to cancelling, then cancelled once in-flight requests drain. Cancelling a terminal batch returns 400; cancelling one already cancelling is a no-op.
Expiry happens when the 24-hour completion_window closes. The batch moves to expired.
In both cases partial output is kept. Whatever completed is in the output file and billed. Whatever did not run is in the error file and not billed. OpenAI discards results on cancel; Morph keeps them.

Limits

Limit	Value
Input file size	100 MB
Lines per file	50,000
Models per batch	1
`endpoint`	`/v1/chat/completions` only
`completion_window`	`24h` only
`metadata`	Up to 16 pairs, keys up to 64 characters, values up to 512
`stream: true`	Rejected
`custom_id`	Unique within the batch; no length cap

Pricing

Every request that completes is billed at 50% of the model’s synchronous rate, input and output alike. Requests in the error file are free. Cancel and expiry do not refund requests that already completed. Rates per model are on the pricing page.

Retention

Input files, output files, and error files are deleted 30 days after they are written. Download what you need before then, or set a shorter window with expires_after on upload and output_expires_after on batch creation (1 hour to 30 days, anchored to when the file is created).
Zero-data-retention accounts: files and outputs are deleted after 24 hours. Poll and download promptly.
DELETE /v1/files/{file_id} removes a file immediately.

Pitfalls

Output file has fewer lines than my input

Check the error file too. The two files together cover every custom_id. If the batch is expired or cancelled, the missing lines are in the error file as batch_expired or batch_cancelled.

A line has status_code 400 in the output file

That is the model rejecting that one request (a bad parameter, an oversized prompt). Read response.body for the reason, fix the line, and resubmit it in a new batch. The rest of the batch is unaffected.

SDK auto-pagination fails on list endpoints

after on GET /v1/files and GET /v1/batches is an integer offset, not an object id, so the SDK’s auto-paginator (which passes the last id) gets a 400. Pass after and limit yourself and increment after by limit.

Output lines are in a different order than my input

Expected: lines are written as requests complete. Join on custom_id.

Get Started

Models

API Reference

Integrations

When to use it

Quick Start

Input file format

Output file format

Error file format

Statuses

Cancel and expiry

Limits

Pricing

Retention

Pitfalls

See Also

​When to use it

​Quick Start

​Input file format

​Output file format

​Error file format

​Statuses

​Cancel and expiry

​Limits

​Pricing

​Retention

​Pitfalls

​See Also

When to use it

Quick Start

Input file format

Output file format

Error file format

Statuses

Cancel and expiry

Limits

Pricing

Retention

Pitfalls

See Also