Retrieval API endpoint
Reads committed outcomes in stable publication order. While running, `has_more=false` only means the job is currently drained; poll again with `next_cursor` to observe later outcomes. Each response is bounded to 1 MiB.
/web/crawl/{job_id}/resultsjob_id must identify an owned crawl job. cursor is an opaque owner-bound continuation token. limit is 1–100 and defaults to 50. The server may return fewer outcomes than requested to keep the JSON response at or below 1 MiB. The cursor resumes after the last returned publication sequence, so byte-bounded pages do not skip or duplicate outcomes.
While the job is running, has_more=false means the results are currently drained; poll again with next_cursor to observe later outcomes. Result order is the committed monotonic publication order, not URL discovery order. snapshot_sequence is the last sequence returned on this page. Each admitted URL eventually appears once as succeeded, failed, or skipped.
While the job is queued, running, or cancelling, progress and the operational counters in usage are read from the latest durable frontier checkpoint. After the job becomes terminal, its stored terminal progress and usage remain authoritative. The REST and MCP responses overlay credits_reserved, credits_charged, credits_released, and settlement_pending from the cumulative financial ledger while retaining the operational counters.
The response state values are queued, running, cancelling, succeeded, failed, and cancelled. Each result outcome is succeeded, failed, or skipped. A succeeded outcome has document and no failure; failed or skipped has failure and no document.
Failure reason values are invalid_url, unsupported_content_type, blocked, not_found, rate_limited, timeout, upstream_error, internal_error, out_of_scope, robots_disallowed, duplicate, limit_reached, unsupported_filter, unknown_source, idempotency_conflict, invalid_cursor, result_expired, full_text_unavailable, and budget_exhausted.
| Parameter | Type | Required | Default | Description | Example |
|---|---|---|---|---|---|
| job_id (path) | string | Yes | Owned crawl job ID | job-123 | |
| cursor | string | No | Opaque owner-bound cursor | ||
| limit | integer | No | Outcomes per page Minimum: 1. Maximum: 100. |
curl -X GET "https://api.crawlora.net/api/v1/web/crawl/job-123/results?limit=10" \ -H "x-api-key: $CRAWLORA_API_KEY"
{
"code": 200,
"msg": "OK",
"data": {
"job_id": "opaque-job-id",
"state": "running",
"progress": {
"discovered": 1,
"discovery_excluded": 0,
"admitted": 1,
"queued": 0,
"in_flight": 0,
"succeeded": 1,
"failed": 0,
"skipped": 0
},
"usage": {
"search_requests": 0,
"fetch_attempts": 1,
"fetched_urls": 1,
"browser_renders": 0,
"input_tokens": 0,
"output_tokens": 0,
"credits_reserved": 5,
"credits_charged": 5,
"credits_released": 0,
"settlement_pending": true
},
"items": [
{
"sequence": 1,
"url": "https://example.org/docs/",
"depth": 0,
"outcome": "succeeded",
"document": {
"document_id": "doc-1",
"source_url": "https://example.org/docs/",
"final_url": "https://example.org/docs/",
"content_type": "text/html",
"fetched_at": "2026-10-09T04:00:15Z",
"revision": "sha256:…",
"content_sha256": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"metadata": {},
"markdown": "# Documentation",
"truncated": false
}
}
],
"next_cursor": "opaque-cursor",
"has_more": false,
"snapshot_sequence": 1,
"partial": false
}
}Request schema
No body schema
Response schema
#/definitions/crawlretrieval.crawlResultsEnvelope
| Field | Type | Required | Enum | Bounds | Example | Description |
|---|---|---|---|---|---|---|
| code | integer | No | 200 | |||
| data | crawlretrieval.crawlResultsDoc | No | ||||
| data.expires_at | string · date-time | No | ||||
| data.has_more | boolean | No | ||||
| data.items | array | No | ||||
| data.items[].depth | integer | No | min 0 | |||
| data.items[].document | retrievaltypes.Document | No | Document record, 29 fields: content_sha256, content_type, document_id, fetched_at, final_url, links, markdown, metadata, passages, revision, source_url, truncated. Described under Search the public web API. | |||
| data.items[].failure | crawlretrieval.crawlFailureDoc | No | crawlFailureDoc record, 4 fields: message, reason, retry_after_seconds, retryable. Described under Cancel a web crawl API. | |||
| data.items[].outcome | string | No | succeeded, failed, skipped | |||
| data.items[].sequence | integer | No | min 1 | |||
| data.items[].url | string · uri | No | ||||
| data.job_id | string | No | ||||
| data.next_cursor | string | No | ||||
| data.partial | boolean | No | ||||
| data.progress | retrievaltypes.JobProgress | No | JobProgress record, 8 fields: admitted, discovered, discovery_excluded, failed, in_flight, queued, skipped, succeeded. Described under Cancel a web crawl API. | |||
| data.snapshot_sequence | integer | No | min 0 | |||
| data.state | string | No | queued, running, cancelling, succeeded, failed, cancelled | |||
| data.usage | retrievaltypes.Usage | No | Usage record, 10 fields: browser_renders, credits_charged, credits_released, credits_reserved, fetch_attempts, fetched_urls, input_tokens, output_tokens, search_requests, settlement_pending. Described under Map a public website API. | |||
| msg | string | No | OK |
| Status | Description | Schema |
|---|---|---|
| 400 | Invalid job ID, cursor, limit, or query parameter | #/definitions/app.Response |
| 401 | Authentication required | #/definitions/app.Response |
| 404 | Unknown, unowned, or non-crawl job | #/definitions/app.Response |
| 410 | Expired cursor or retained result | #/definitions/app.Response |
| 500 | Internal storage error | #/definitions/app.Response |
| 503 | Required backend unavailable | #/definitions/app.Response |
Use environment variables for secrets and keep Crawlora API keys server-side.
curl -X GET "https://api.crawlora.net/api/v1/web/crawl/job-123/results?limit=10" \
-H "x-api-key: $CRAWLORA_API_KEY"