Retrieval API endpoint
Searches indexed sources only; this operation never starts ingestion or contacts upstream sources. The current ready source is `go:docs` and it covers only the 18 selected verified pages documented by `developer-sources`. Search matches Unicode-aware lexical terms and punctuation-bearing identifiers such as `context.Context`; passages retain byte/line locators into the indexed document. `types` accepts `doc`, `readme`, `issue`, and `pull_request`, but the current source supports only `doc`; unsupported values return 400. The `version` filter is not supported by the current unversioned source and returns 400. Continuation bodies contain only `cursor` and optional `limit`; their pages come from the frozen snapshot and do not repeat the search.
/developer/searchSearches indexed documents only. It never fetches a source or starts ingestion. The current ready source is go:docs, covering only the selected 18 verified pages returned by /developer/sources.
q: required on the first page; 1–2,000 characters. Unicode-aware lexical matching preserves punctuation identifiers such as context.Context, io.EOF, and -flag. When other content terms remain, a small set of ordinary question words is ignored for ranking. Literal queries consisting only of those words remain searchable; stored index terms and frozen continuation order are unchanged.sources: optional, at most 20 unique source IDs. The only currently registered ID is go:docs.types: optional unique values from doc, readme, issue, and pull_request. The current source supports only doc; the other documented enum values are rejected for this source.version: optional string, but unsupported by the current unversioned Go documentation source and rejected with 400.limit: optional integer from 1–50; defaults to 10.passages_per_result: optional integer from 1–5; defaults to 2.cursor: optional opaque cursor. When present, the body may contain only cursor and optional limit.Scalar values must appear once. Unknown JSON fields are rejected. Search pages are frozen, owner-scoped snapshots; a continuation reads the frozen result payload and does not search a newer index. Public response data is kept below 960 KiB; if needed, a continuation returns a smaller page with its cursor advanced only through the last returned item.
items contains id, source_id, type, canonical url, revision, indexed_at, exact matched passages, and rank. Passage locators point into the retained normalized document bytes.coverage.status values are ok, failed, not_indexed, and unsupported.warnings.code values are source_failed, source_not_indexed, discovery_truncated, ranking_window_limit, imprecise_date_excluded, content_unavailable, and cached_source_stale.partial is true when a source is not indexed or the bounded 200-document ranking window is exhausted.usage.search_requests=1. Cursor-only continuation reports zero usage. credits_charged and settlement_pending reflect the actual billing receipt.Example request:
{
"q": "Go assembler",
"sources": ["go:docs"],
"types": ["doc"],
"limit": 10,
"passages_per_result": 2
}
Shared response schema enum values (the owning endpoint emits its own job kind): blocked, budget_exhausted, duplicate, full_text_unavailable, idempotency_conflict, internal_error, invalid_cursor, invalid_url, limit_reached, not_found, out_of_scope, rate_limited, result_expired, robots_disallowed, timeout, unknown_source, unsupported_content_type, unsupported_filter, upstream_error.
| Parameter | Type | Required | Default | Description | Example |
|---|---|---|---|---|---|
| Authorization (header) | string | No | Dashboard JWT in the form `Token <jwt>` (use this or x-api-key) | ||
| x-api-key (header) | string | No | API key (alternative to a dashboard JWT) | ||
| request (body) | object | Yes | Search query or cursor-only continuation |
curl -X POST "https://api.crawlora.net/api/v1/developer/search" \
-H "x-api-key: $CRAWLORA_API_KEY" \
-H "Content-Type: application/json" \
-d '{"cursor":"string","limit":1,"passages_per_result":1,"q":"string","sources":["string"],"types":["doc"],"version":"string"}'{
"cursor": "string",
"limit": 1,
"passages_per_result": 1,
"q": "string",
"sources": [
"string"
],
"types": [
"doc"
],
"version": "string"
}{
"code": 200,
"msg": "Request successful",
"data": {
"items": [
{
"id": "doc_<stable-id>",
"source_id": "go:docs",
"type": "doc",
"url": "https://go.dev/doc/asm",
"title": "A Quick Guide to Go's Assembler",
"revision": "<content-sha256>",
"indexed_at": "2026-10-09T00:00:00Z",
"passages": [
{
"passage_id": "passage_<stable-id>",
"text": "<matched passage from the indexed page>",
"locator": {
"heading": "Canceling in-progress operations",
"start_byte": 0,
"end_byte": 34
}
}
],
"rank": 1
}
],
"has_more": false,
"partial": false,
"coverage": [
{
"source_id": "go:docs",
"status": "ok",
"corpus_revision": "<sha256>"
}
],
"warnings": [],
"usage": {
"search_requests": 1,
"fetch_attempts": 0,
"fetched_urls": 0,
"browser_renders": 0,
"input_tokens": 0,
"output_tokens": 0,
"credits_reserved": 0,
"credits_charged": 0,
"credits_released": 0,
"settlement_pending": false
}
}
}Request schema
#/definitions/developerretrieval.DeveloperSearchRequest
| Field | Type | Required | Enum | Bounds | Example | Description |
|---|---|---|---|---|---|---|
| cursor | string | No | ||||
| limit | integer | No | ||||
| passages_per_result | integer | No | ||||
| q | string | No | ||||
| sources | array | No | ||||
| types | array | No | ||||
| version | string | No |
Response schema
#/definitions/developerretrieval.developerSearchResponseDoc
| Field | Type | Required | Enum | Bounds | Example | Description |
|---|---|---|---|---|---|---|
| code | integer | No | 200 | |||
| data | developerretrieval.developerSearchDataDoc | No | ||||
| data.coverage | array | No | ||||
| data.coverage[].corpus_revision | string | No | ||||
| data.coverage[].failure | developerretrieval.developerFailureDoc | No | ||||
| data.coverage[].failure.message | string | No | ||||
| data.coverage[].failure.reason | string | No | invalid_url, unsupported_content_type, blocked, not_found, rate_limited, timeout, upstream_error, internal_error, out_of_scope, robots_disallowed, duplicate, limit_reached, unsupported_filter, unknown_source, idempotency_conflict, invalid_cursor, result_expired, full_text_unavailable, budget_exhausted | |||
| data.coverage[].failure.retry_after_seconds | integer | No | ||||
| data.coverage[].failure.retryable | boolean | No | ||||
| data.coverage[].indexed_at | string | No | ||||
| data.coverage[].retrieved_at | string | No | ||||
| data.coverage[].source_id | string | No | ||||
| data.coverage[].status | string | No | ok, failed, not_indexed, unsupported | |||
| data.has_more | boolean | No | ||||
| data.items | array | No | ||||
| data.items[].id | string | No | ||||
| data.items[].indexed_at | string | No | ||||
| data.items[].passages | array | No | ||||
| data.items[].passages[].locator | retrievaltypes.PassageLocator | No | ||||
| data.items[].passages[].locator.anchor | string | No | ||||
| data.items[].passages[].locator.end_byte | integer | No | ||||
| data.items[].passages[].locator.end_line | integer | No | ||||
| data.items[].passages[].locator.heading | string | No | ||||
| data.items[].passages[].locator.page | integer | No | ||||
| data.items[].passages[].locator.start_byte | integer | No | ||||
| data.items[].passages[].locator.start_line | integer | No | ||||
| data.items[].passages[].passage_id | string | No | ||||
| data.items[].passages[].score | number | No | ||||
| data.items[].passages[].text | string | No | ||||
| data.items[].rank | integer | No | ||||
| data.items[].repository | string | No | ||||
| data.items[].revision | string | No | ||||
| data.items[].source_id | string | No | ||||
| data.items[].title | string | No | ||||
| data.items[].type | string | No | doc, readme, issue, pull_request | |||
| data.items[].updated_at | string | No | ||||
| data.items[].url | string | No | ||||
| data.items[].version | string | No | ||||
| data.next_cursor | string | No | ||||
| data.partial | boolean | No | ||||
| data.usage | retrievaltypes.Usage | No | Usage record, 10 fields: browser_renders, credits_charged, credits_released, credits_reserved, fetch_attempts, fetched_urls, input_tokens, output_tokens, search_requests, settlement_pending. Described under Map a public website API. | |||
| data.warnings | array | No | ||||
| data.warnings[].code | string | No | source_failed, source_not_indexed, discovery_truncated, ranking_window_limit, imprecise_date_excluded, content_unavailable, cached_source_stale | |||
| data.warnings[].message | string | No | ||||
| data.warnings[].source_id | string | No | ||||
| msg | string | No | Request successful |
| Status | Description | Schema |
|---|---|---|
| 400 | Invalid query, unsupported source/filter, or cursor | #/definitions/app.Response |
| 401 | Authentication required | #/definitions/app.Response |
| 410 | Frozen result cursor expired | #/definitions/app.Response |
| 500 | Snapshot or response configuration error | #/definitions/app.Response |
| 503 | Source registry or search index unavailable | #/definitions/app.Response |
Use environment variables for secrets and keep Crawlora API keys server-side.
curl -X POST "https://api.crawlora.net/api/v1/developer/search" \
-H "x-api-key: $CRAWLORA_API_KEY" \
-H "Content-Type: application/json" \
-d '{"cursor":"string","limit":1,"passages_per_result":1,"q":"string","sources":["string"],"types":["doc"],"version":"string"}'