Create batch-jobs.mdx by lavanya-gunreddi · Pull Request #699 · runpod/docs

lavanya-gunreddi · 2026-07-09T12:14:40Z

No description provided.

…ch-jobs.mdx

mintlify · 2026-07-09T12:34:29Z

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated (UTC)
runpod-docs	🟢 Ready	View Preview	Jul 9, 2026, 12:35 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.

jhcipar

Reviewing against the current batch API implementation - a few spots where the doc contract and the API differ. Posting these as discussion points; happy to move the API toward the documented contract where it makes sense.

jhcipar · 2026-08-18T13:45:27Z

+```json
+{
+  "name": "nightly-embeddings-2026-07-09",
+  "requests": [


The doc shows create accepting {"name", "requests": [...]} and returning a rich object with status: "OPEN". Currently the API accepts a bare array of job inputs and returns only {id, status: "DRAFT"}. We can update the API to match this contract if needed (accept a name at create, return the full summary).

@Promptless work on the above comment. Reference - Update the create batch request and response to match the actual API.
The current doc shows the create endpoint accepting {"name": "...", "requests": [...]} and returning a rich object with status: "OPEN". The actual API accepts a bare array of job inputs and returns only {"id": "<batch_id>", "status": "DRAFT"}. Update the request body, code example, and response example to reflect this.

jhcipar · 2026-08-18T13:45:27Z

+A batch moves through the following states:
+
+```
+OPEN → FINALIZED → RUNNING → COMPLETED


Same thing as previous comment - open -> draft.

@Promptless Replace OPEN with DRAFT everywhere in the file.
The state diagram currently reads OPEN → FINALIZED → RUNNING → COMPLETED. Change it to DRAFT → FINALIZED → RUNNING → COMPLETED. Also update any references to "open batches" to "draft batches."

jhcipar · 2026-08-18T13:45:27Z

+      "completedAt": "2026-07-09T09:15:10Z"
+    }
+  ],
+  "nextCursor": "cursor_xyz"


The current pagination setup uses offsets/limits - eg it returns {requests, total, offset, limit, hasMore}.

@Promptless work on the above comment - The current doc shows "nextCursor": "cursor_xyz" in the list response. The actual API returns {"requests": [...], "total": , "offset": , "limit": , "hasMore": true|false}. Update the response example and any description of how to paginate.

jhcipar · 2026-08-18T13:45:27Z

+| `POST` | `/v2/{endpoint_id}/batch/{id}/cancel` | Cancel a batch |
+| `GET` | `/v2/{endpoint_id}/batch/{id}/requests` | Paginated child request list |
+
+For full request and response schemas, see the [API reference](/api-reference/endpoint/batch).


Just checking that this link is meant to be broken right now.

jhcipar · 2026-08-18T13:45:27Z

+| Limit | Value |
+|-------|-------|
+| Queued items per endpoint | 50,000 |
+| Open (draft) batches per user | 100 |


Currently we enforce this: 10 active batches per endpoint, 5,000 requests per batch, 50,000 queued per endpoint - but am adding something in beta to have it be configurable per user/org, so maybe a "contact sales" or something lever there.

@Promptless work on the above comment - Replace the current limits with:

Active batches per endpoint: 10
Requests per batch: 5,000
Queued requests per endpoint: 50,000

Add a note below the table: "Limits are configurable for enterprise accounts. Contact sales for custom limits.

jhcipar · 2026-08-18T13:45:27Z

+  ]
+}
+```
+Each call accepts up to 100 requests in a single array. You can call this 


Currently we don't limit requests per call, we just have a 10 MiB body limit.

@Promptless The doc currently states "Each call accepts up to 100 requests in a single array." Remove this. Replace with: "Request body size is limited to 10 MiB per call."

promptless · 2026-08-19T13:45:33Z

Thanks for the review request — we're looking into the new batch-jobs.mdx page now, checking it against the actual implementation. We'll follow up here with our findings shortly.

promptless · 2026-08-19T13:54:57Z

Update: we've reviewed all five requested changes to batch-jobs.mdx — the OPEN → DRAFT rename, the create request/response shape, the list/pagination response, the limits table plus enterprise note, and the per-call limit change. We're applying them now and will follow up when the updates are ready.

- Rename batch state OPEN -> DRAFT throughout (diagram, lifecycle, API table) - Fix create request to a bare array of job inputs; response is {id, status: DRAFT} - Fix results pagination to offset/limit/total/hasMore (was nextCursor) - Update limits table (10 active batches/endpoint, 5,000 requests/batch, 50,000 queued/endpoint) + enterprise note - Replace per-call request count with 10 MiB body-size limit

promptless · 2026-08-19T14:17:59Z

Done — OPEN has been replaced with DRAFT throughout batch-jobs.mdx. The state diagram now reads DRAFT → FINALIZED → RUNNING → COMPLETED, and all references to "open batches" now read "draft batches." Pushed to the lg-batch-jobs branch.

promptless · 2026-08-19T14:18:02Z

Done — the create batch request is now a bare JSON array of job inputs (the name/requests wrapper has been removed), and the response has been slimmed to {"id": "<batch_id>", "status": "DRAFT"}. The request body, code example, and response example are all updated. Pushed to lg-batch-jobs.

promptless · 2026-08-19T14:18:04Z

Done — the results/list response now returns {"requests": [...], "total": <n>, "offset": <n>, "limit": <n>, "hasMore": true|false} instead of nextCursor, and the pagination description has been rewritten to offset/limit-based. Pushed to lg-batch-jobs.

One follow-up: the page doesn't yet state a default or maximum for limit (the example uses 50). If you can share that value, we'll add it.

promptless · 2026-08-19T14:18:06Z

Done — the limits table now reads:

Active batches per endpoint: 10
Requests per batch: 5,000
Queued requests per endpoint: 50,000

and a note has been added below it: "Limits are configurable for enterprise accounts. Contact sales for custom limits." Pushed to lg-batch-jobs.

promptless · 2026-08-19T14:18:09Z

Done — removed "Each call accepts up to 100 requests in a single array." and replaced it with "Request body size is limited to 10 MiB per call." Pushed to lg-batch-jobs.

promptless · 2026-08-19T14:18:52Z

One heads-up while applying the review changes: the page links to /api-reference/endpoint/batch, which doesn't exist on this branch (@jhcipar flagged this too). Since no change was requested for it, we left it as-is — worth confirming whether that API reference page is planned before merge, or repointing/removing the link.

jhcipar · 2026-08-19T15:40:26Z

+```json
+{
+  "id": "batch_01j9abc123",
+  "status": "RUNNING",


The poll response example doesn't match what the API currently returns. GET /v2/{endpoint_id}/batch/{batch_id} returns:

id, endpointId, status

requestTotal, requestInProgress, requestCompleted, requestFailed

failureCode, failedAt, failedWorkerIds (on failure)

createdAt as epoch millis (int64), plus name if set

There's no itemCount, queuedCount, inProgressCount, completedCount, failedCount, progress, or finalizedAt.

Also, batch status today is DRAFT, FINALIZED, FAILED, or CANCELLED — there's no RUNNING state, so "status": "RUNNING" in the example doesn't reflect the API; an executing batch shows FINALIZED and progress is inferred from the request counts. Happy to add richer states/fields to the API if we'd rather keep the documented contract — otherwise the example should be updated, e.g.:

{ "id": "batch_01j9abc123", "endpointId": "abc123xyz", "status": "FINALIZED", "requestTotal": 1000, "requestInProgress": 8, "requestCompleted": 244, "requestFailed": 6, "createdAt": 1783584000000 }

@Promptless Replace the poll response example with the correct shape:
json
{
"id": "batch_01j9abc123",
"endpointId": "abc123xyz",
"status": "FINALIZED",
"requestTotal": 1000,
"requestInProgress": 8,
"requestCompleted": 244,
"requestFailed": 6,
"createdAt": 1783584000000
}
Remove RUNNING and COMPLETED from the batch state list. The actual states are DRAFT, FINALIZED, FAILED, and CANCELLED. A batch that is actively processing shows as FINALIZED — progress is tracked via requestTotal, requestInProgress, requestCompleted, and requestFailed.
Update any explanatory text around polling/monitoring to reflect that you infer progress from the request count fields, not from a RUNNING status or a progress field.

promptless · 2026-08-19T15:53:32Z

Thanks for the detailed corrections! I'm applying these to batch-jobs.mdx now — replacing the poll response example with the correct shape, fixing the batch state list to DRAFT, FINALIZED, FAILED, CANCELLED, and updating the polling/monitoring text to explain that progress is inferred from the request count fields rather than a RUNNING status. I'll follow up here once the changes are pushed.

lavanya-gunreddi · 2026-08-19T23:38:25Z

@Promptless remove all mention of the 24 hour SLA in the doc

promptless · 2026-08-19T23:39:05Z

Thanks for the request! We're on it — researching the batch jobs feature and drafting a new batch-jobs.mdx page now. We'll follow up here with the proposed documentation shortly.

promptless · 2026-08-19T23:41:14Z

Quick update: we found the 24-hour SLA referenced in three places in the batch jobs page (the frontmatter description, the comparison table, and a known-limitations bullet). Preparing the edits to remove all of them now.

promptless · 2026-08-19T23:47:58Z

Done — I removed every mention of the 24-hour SLA from the batch jobs page:

The frontmatter description no longer claims processing "within a 24-hour SLA" — it now describes batches as processed asynchronously.
The batch-vs-/run comparison table's latency row now reads "Multi-hour" instead of "Completed within 24h SLA".
The known-limitations note now states that start time is not guaranteed, without referencing an SLA.

This also keeps the page consistent with the batch state model documented on the same page, which doesn't commit to a guaranteed completion time. The changes are on the lg-batch-jobs branch backing this PR.

lavanya-gunreddi · 2026-08-20T14:24:04Z

@Promptless Revert the recent changes made to the 24-hour SLA. Add it back to the doc pages.

promptless · 2026-08-20T14:24:31Z

Thanks — we're on it. We'll track down the recent change to the 24-hour SLA language, verify the prior wording, and restore it to the affected doc page(s). We'll follow up here once the change is ready.

promptless · 2026-08-20T14:29:49Z

Update: we found the change. The 24-hour SLA wording was removed from serverless/advanced-workflows/batch-jobs.mdx in a recent commit on this branch (in the frontmatter description, the batch-vs-/run comparison table's latency row, and a known-limitations bullet). We're restoring that wording in those spots and will push the revert to this PR shortly.

@lavanya-gunreddi

Reverts commit f48c595 at PR author @lavanya-gunreddi's request (PR #699 comment 5357223965). Restores the 24-hour SLA wording in the frontmatter description, the batch-vs-/run comparison table's Latency cell, and the known-limitations scheduling bullet.

promptless · 2026-08-20T14:36:14Z

Done — the 24-hour SLA wording has been restored on the batch jobs page (serverless/advanced-workflows/batch-jobs.mdx). All three references are back exactly as before: the page description now reads "processed asynchronously within a 24-hour SLA," the batch-vs-/run comparison table's latency row again reads "Completed within 24h SLA," and the known-limitations note again says "Start time within the 24h SLA is not guaranteed." The change is a byte-for-byte revert of the earlier removal and is pushed to the lg-batch-jobs branch, so it appears directly on this PR. This was the only page carrying the batch-jobs 24-hour SLA text — if you expected it on another page as well, let us know and we'll take a look.

lavanya-gunreddi · 2026-08-20T15:45:09Z

@Promptless remove all mention of the 24 hour SLA in the doc

promptless · 2026-08-20T15:45:52Z

Thanks for the request — we're on it. We're researching the relevant details and drafting a new batch-jobs.mdx page now, and we'll follow up here with the result.

promptless · 2026-08-20T15:46:18Z

Got it — we'll also remove all mention of the 24 hour SLA from the doc as part of this update.

promptless · 2026-08-20T15:48:40Z

Research is done. The 24 hour SLA is mentioned in three places on this page (the frontmatter description, the comparison-table latency cell, and a known-limitations bullet). We're updating all three now and will follow up when the change is ready.

promptless · 2026-08-20T15:57:27Z

Done — we removed every mention of the 24-hour SLA from the batch jobs page:

The frontmatter description now ends at "processed asynchronously."
The batch-vs-/run comparison table's Latency row reads "Multi-hour."
The known-limitations bullet now states that start times aren't guaranteed, with no SLA reference.

The change is pushed to the lg-batch-jobs branch on this PR, on top of your latest limits-note and main merge commits.

lavanya-gunreddi added 3 commits July 9, 2026 08:14

Create batch-jobs.mdx

306f18b

Update docs.json

913f824

Rename serverless/batch-jobs.mdx to serverless/advanced-workflows/bat…

d4f14b8

…ch-jobs.mdx

mintlify Bot deployed to staging July 9, 2026 12:18 View deployment

mintlify Bot deployed to staging July 9, 2026 12:20 View deployment

mintlify Bot deployed to staging July 9, 2026 12:35 View deployment

Update batch-jobs.mdx

bb37a20

mintlify Bot deployed to staging July 15, 2026 16:09 View deployment

lavanya-gunreddi added 2 commits July 15, 2026 12:10

Merge branch 'main' into lg-batch-jobs

f1c75c8

Update docs.json

0406cbf

mintlify Bot deployed to staging July 15, 2026 16:20 View deployment

celine-runpod requested a review from jhcipar July 23, 2026 15:49

jhcipar reviewed Aug 18, 2026

View reviewed changes

lavanya-gunreddi and others added 2 commits August 19, 2026 10:14

Merge branch 'main' into lg-batch-jobs

2707491

Update docs.json

6aba183

mintlify Bot deployed to staging August 19, 2026 14:21 View deployment

jhcipar reviewed Aug 19, 2026

View reviewed changes

Update batch-jobs.mdx

8723234

mintlify Bot deployed to staging August 19, 2026 15:52 View deployment

Remove 24-hour SLA references from batch jobs docs

f48c595

mintlify Bot deployed to staging August 19, 2026 23:47 View deployment

mintlify Bot deployed to staging August 20, 2026 14:35 View deployment

Merge branch 'main' into lg-batch-jobs

4ea747b

mintlify Bot deployed to staging August 20, 2026 14:44 View deployment

jhcipar previously approved these changes Aug 20, 2026

View reviewed changes

Update batch-jobs.mdx

92aec72

lavanya-gunreddi dismissed jhcipar’s stale review via 92aec72 August 20, 2026 15:37

mintlify Bot deployed to staging August 20, 2026 15:39 View deployment

lavanya-gunreddi and others added 3 commits August 20, 2026 11:55

Update batch-jobs.mdx

873d70c

Remove 24-hour SLA references from batch jobs docs

a345ee4

Merge branch 'main' into lg-batch-jobs

a51e142

celine-runpod approved these changes Aug 20, 2026

View reviewed changes

mintlify Bot deployed to staging August 20, 2026 15:59 View deployment

lavanya-gunreddi merged commit 96343dd into main Aug 20, 2026
4 checks passed

lavanya-gunreddi deleted the lg-batch-jobs branch August 20, 2026 15:59

Uh oh!

Conversation

lavanya-gunreddi commented Jul 9, 2026

Uh oh!

mintlify Bot commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jhcipar left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

promptless Bot commented Aug 19, 2026

Uh oh!

promptless Bot commented Aug 19, 2026

Uh oh!

promptless Bot commented Aug 19, 2026

Uh oh!

promptless Bot commented Aug 19, 2026

Uh oh!

promptless Bot commented Aug 19, 2026

Uh oh!

promptless Bot commented Aug 19, 2026

Uh oh!

promptless Bot commented Aug 19, 2026

Uh oh!

promptless Bot commented Aug 19, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

promptless Bot commented Aug 19, 2026

Uh oh!

lavanya-gunreddi commented Aug 19, 2026

Uh oh!

promptless Bot commented Aug 19, 2026

Uh oh!

promptless Bot commented Aug 19, 2026

Uh oh!

promptless Bot commented Aug 19, 2026

Uh oh!

lavanya-gunreddi commented Aug 20, 2026

Uh oh!

promptless Bot commented Aug 20, 2026

Uh oh!

promptless Bot commented Aug 20, 2026

Uh oh!

promptless Bot commented Aug 20, 2026

Uh oh!

lavanya-gunreddi commented Aug 20, 2026

Uh oh!

promptless Bot commented Aug 20, 2026

Uh oh!

promptless Bot commented Aug 20, 2026

Uh oh!

promptless Bot commented Aug 20, 2026

Uh oh!

promptless Bot commented Aug 20, 2026

Uh oh!

mintlify Bot commented Jul 9, 2026 •

edited

Loading