> ## Documentation Index
> Fetch the complete documentation index at: https://docs.morphllm.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Batch API
> Run thousands of chat completions offline at half price. OpenAI-compatible: upload a JSONL file, create a batch, download the results within 24 hours.
Upload a JSONL file of chat-completion requests, create a batch over it, and collect the results when it finishes. Every completed request is billed at **half the model's synchronous rate**. The API is the OpenAI Batch API, so the official OpenAI SDKs work unchanged once you point them at `https://api.morphllm.com/v1`.
Available on `morph-glm53flash`, `morph-dsv4flash`, `morph-glm53-744b`, and `morph-kimik3`. Per-token rates are on the [pricing page](https://www.morphllm.com/pricing).
## When to use it
Batch fits work where nobody is waiting on an individual response: eval runs, dataset generation, nightly summarization, re-indexing, backfills. You trade latency for price. Results land within 24 hours, usually much sooner, and there is no per-request rate limit to work around.
For work that needs a response now, use the synchronous endpoint. For background traffic that still needs each answer within seconds, use [Standby](/sdk/components/standby) instead.
## Quick Start
Write one request per line in a JSONL file. Each line names a `custom_id` you choose, the endpoint, and the same body you would send synchronously.
```jsonl requests.jsonl theme={null}
{"custom_id": "req-0001", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "morph-glm53flash", "messages": [{"role": "user", "content": "Summarize this diff in one sentence: ..."}], "max_tokens": 256}}
{"custom_id": "req-0002", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "morph-glm53flash", "messages": [{"role": "user", "content": "Summarize this diff in one sentence: ..."}], "max_tokens": 256}}
```
Then upload the file, create the batch, poll until it reaches a terminal status, and download the output and error files.
```python theme={null}
import time
from openai import OpenAI
client = OpenAI(
api_key="YOUR_API_KEY",
base_url="https://api.morphllm.com/v1",
)
# 1. Upload the input file
input_file = client.files.create(
file=open("requests.jsonl", "rb"),
purpose="batch",
)
# 2. Create the batch
batch = client.batches.create(
input_file_id=input_file.id,
endpoint="/v1/chat/completions",
completion_window="24h",
metadata={"job": "nightly-summaries"},
)
# 3. Poll until terminal
TERMINAL = {"completed", "failed", "expired", "cancelled"}
while batch.status not in TERMINAL:
time.sleep(60)
batch = client.batches.retrieve(batch.id)
print(batch.status, batch.request_counts)
# 4. Download results. Either file id can be null if nothing landed in it.
if batch.output_file_id:
client.files.content(batch.output_file_id).write_to_file("output.jsonl")
if batch.error_file_id:
client.files.content(batch.error_file_id).write_to_file("errors.jsonl")
```
```typescript theme={null}
import fs from "node:fs";
import OpenAI from "openai";
const client = new OpenAI({
apiKey: "YOUR_API_KEY",
baseURL: "https://api.morphllm.com/v1",
});
// 1. Upload the input file
const inputFile = await client.files.create({
file: fs.createReadStream("requests.jsonl"),
purpose: "batch",
});
// 2. Create the batch
let batch = await client.batches.create({
input_file_id: inputFile.id,
endpoint: "/v1/chat/completions",
completion_window: "24h",
metadata: { job: "nightly-summaries" },
});
// 3. Poll until terminal
const TERMINAL = new Set(["completed", "failed", "expired", "cancelled"]);
while (!TERMINAL.has(batch.status)) {
await new Promise((r) => setTimeout(r, 60_000));
batch = await client.batches.retrieve(batch.id);
console.log(batch.status, batch.request_counts);
}
// 4. Download results. Either file id can be null if nothing landed in it.
if (batch.output_file_id) {
const output = await client.files.content(batch.output_file_id);
fs.writeFileSync("output.jsonl", await output.text());
}
if (batch.error_file_id) {
const errors = await client.files.content(batch.error_file_id);
fs.writeFileSync("errors.jsonl", await errors.text());
}
```
```bash theme={null}
# 1. Upload the input file
curl https://api.morphllm.com/v1/files \
-H "Authorization: Bearer YOUR_API_KEY" \
-F purpose=batch \
-F file=@requests.jsonl
# => {"id": "file_...", "object": "file", "purpose": "batch", ...}
# 2. Create the batch
curl https://api.morphllm.com/v1/batches \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"input_file_id": "file_...",
"endpoint": "/v1/chat/completions",
"completion_window": "24h"
}'
# => {"id": "batch_...", "status": "validating", ...}
# 3. Poll until status is completed, failed, expired, or cancelled
curl https://api.morphllm.com/v1/batches/batch_... \
-H "Authorization: Bearer YOUR_API_KEY"
# 4. Download the output file (and error_file_id, if set)
curl https://api.morphllm.com/v1/files/file_.../content \
-H "Authorization: Bearer YOUR_API_KEY" \
-o output.jsonl
```
## Input file format
One JSON object per line. Blank lines are not allowed.
| Field | Value |
| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `custom_id` | Your key for the request. Unique within the batch, echoed on the matching output or error line. No length cap. |
| `method` | Always `"POST"`. |
| `url` | Always `"/v1/chat/completions"`, matching the batch's `endpoint`. |
| `body` | A [Chat Completions](/api-reference/endpoint/apply) request body. Everything the synchronous endpoint accepts works here, except `stream: true`, which is rejected. |
Every line in a batch must name the same `model`. A file that mixes models, repeats a `custom_id`, or targets another endpoint fails validation and the batch goes to `failed` with the reasons in `errors`.
## Output file format
One line per request the model answered, in completion order rather than input order. Join back to your requests on `custom_id`.
```json theme={null}
{
"id": "batch_req_1a2b3c4d-5e6f-4a7b-8c9d-0e1f2a3b4c5d",
"custom_id": "req-0001",
"response": {
"status_code": 200,
"request_id": "req_3f9a2c1b7d",
"body": {
"id": "chatcmpl-7d2f1c3a9b",
"object": "chat.completion",
"model": "morph-glm53flash",
"choices": [{ "index": 0, "message": { "role": "assistant", "content": "..." }, "finish_reason": "stop" }],
"usage": { "prompt_tokens": 412, "completion_tokens": 18, "total_tokens": 430 }
}
},
"error": null
}
```
`response.body` is the full Chat Completions response, `usage` included. `error` is always `null` in this file.
**Model errors are output lines, not error lines.** When the model returns a non-2xx for a request (a 400 for a bad parameter, a 413 for an oversized prompt, a 500), the line is written to the **output** file with that `status_code` and the error envelope in `response.body`, and `error` stays `null`. Check `status_code` on every output line rather than assuming the file only holds successes. OpenAI routes these to the error file; Morph does not.
## Error file format
One line per request that never got a model response. `response` is always `null` here.
```json theme={null}
{
"id": "batch_req_9f8e7d6c-5b4a-4392-8170-6f5e4d3c2b1a",
"custom_id": "req-0002",
"response": null,
"error": {
"code": "batch_expired",
"message": "This request could not be executed before the completion window expired."
}
}
```
| `error.code` | Meaning |
| ----------------- | --------------------------------------------------------- |
| `timeout` | The request was sent but no response arrived in time. |
| `batch_cancelled` | The batch was cancelled before this request ran. |
| `batch_expired` | The 24-hour window closed before this request ran. |
| `batch_failed` | The batch hit a system error before this request ran. |
| `model_not_found` | The `model` in this line is not available for batch. |
| `parse_error` | The line, or the model's reply to it, was not valid JSON. |
Lines in the error file are not billed.
## Statuses
| Status | Terminal | Meaning |
| ------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| `validating` | | The input file is being checked: JSON per line, one model, unique `custom_id`, supported endpoint. |
| `in_progress` | | Requests are running. `request_counts` advances. |
| `finalizing` | | Every request has finished; the output and error files are being written. |
| `completed` | yes | Done. `output_file_id` and `error_file_id` are set where they have lines. |
| `failed` | yes | Validation failed or the batch hit a system error. Reasons are in `errors`; anything that completed first is still in the output file. |
| `expired` | yes | The 24-hour window closed. Completed requests are in the output file; the rest are in the error file as `batch_expired`. |
| `cancelling` | | A cancel was requested; in-flight requests are draining. |
| `cancelled` | yes | Cancel finished. Completed requests are in the output file; the rest are in the error file as `batch_cancelled`. |
Poll [`GET /v1/batches/{batch_id}`](/api-reference/endpoint/batches-retrieve) every 30 to 60 seconds. There is no webhook.
## Cancel and expiry
* **Cancel** with [`POST /v1/batches/{batch_id}/cancel`](/api-reference/endpoint/batches-cancel) while the batch is `validating` or `in_progress`. It moves to `cancelling`, then `cancelled` once in-flight requests drain. Cancelling a terminal batch returns 400; cancelling one already `cancelling` is a no-op.
* **Expiry** happens when the 24-hour `completion_window` closes. The batch moves to `expired`.
* In both cases **partial output is kept**. Whatever completed is in the output file and billed. Whatever did not run is in the error file and not billed. OpenAI discards results on cancel; Morph keeps them.
## Limits
| Limit | Value |
| ------------------- | ---------------------------------------------------------- |
| Input file size | 100 MB |
| Lines per file | 50,000 |
| Models per batch | 1 |
| `endpoint` | `/v1/chat/completions` only |
| `completion_window` | `24h` only |
| `metadata` | Up to 16 pairs, keys up to 64 characters, values up to 512 |
| `stream: true` | Rejected |
| `custom_id` | Unique within the batch; no length cap |
## Pricing
Every request that completes is billed at **50% of the model's synchronous rate**, input and output alike. Requests in the error file are free. Cancel and expiry do not refund requests that already completed. Rates per model are on the [pricing page](https://www.morphllm.com/pricing).
## Retention
* Input files, output files, and error files are deleted **30 days** after they are written. Download what you need before then, or set a shorter window with `expires_after` on upload and `output_expires_after` on batch creation (1 hour to 30 days, anchored to when the file is created).
* **Zero-data-retention accounts:** files and outputs are deleted after **24 hours**. Poll and download promptly.
* [`DELETE /v1/files/{file_id}`](/api-reference/endpoint/files-delete) removes a file immediately.
## Pitfalls
Check the error file too. The two files together cover every `custom_id`. If the batch is `expired` or `cancelled`, the missing lines are in the error file as `batch_expired` or `batch_cancelled`.
That is the model rejecting that one request (a bad parameter, an oversized prompt). Read `response.body` for the reason, fix the line, and resubmit it in a new batch. The rest of the batch is unaffected.
`after` on `GET /v1/files` and `GET /v1/batches` is an integer offset, not an object id, so the SDK's auto-paginator (which passes the last id) gets a 400. Pass `after` and `limit` yourself and increment `after` by `limit`.
Expected: lines are written as requests complete. Join on `custom_id`.
## See Also
* [Standby Requests](/sdk/components/standby) for background traffic that still needs answers in seconds
* [Prompt Caching](/sdk/components/caching) for repeated prefixes across requests
* [Open Source Models](/sdk/components/fast-models) for model ids and context windows
* [API Reference: Batch](/api-reference/endpoint/files-upload) for every field on every endpoint