Crawlora Docs
Handle invalid requests, authentication errors, rate limits, upstream failures, timeouts, challenged responses, and temporary execution issues.
Production public web data workflows need clear handling for invalid requests, auth errors, rate limits, upstream failures, timeouts, challenged responses, and temporary execution issues.
Exact error bodies can vary by endpoint. This shape is illustrative until a specific endpoint documents a more precise response schema.
{
"error": {
"code": "upstream_unavailable",
"message": "The upstream response could not be used.",
"request_id": "req_..."
}
}Check individual endpoint docs for exact response behavior. Generated endpoint docs also include failure responses where metadata is available.
| Status | Meaning | Retry? | Recommended action |
|---|---|---|---|
| 400 | Invalid request | No | Fix input, required fields, or schema mismatch. |
| 401 | Missing or invalid API key | No | Check x-api-key and server-side secret configuration. |
| 403 | Permission, plan, or policy issue | No | Check plan access and account state. |
| 404 | Endpoint or resource not found | No | Confirm endpoint path and path parameters. |
| 422 | Validation error when supported | No | Fix field values and enum choices. |
| 429 | Rate limit exceeded | After delay | Back off and respect rate-limit reset behavior. |
| 500 | Internal execution error | Yes | Retry with capped exponential backoff. |
| 502 | Upstream failure or unusable upstream response | Sometimes | Retry later, inspect endpoint notes, and log request context. |
| 503 | Temporary unavailable | Yes | Retry with backoff and reduce concurrency. |
| 504 | Timeout | Yes | Retry with backoff and review request size or endpoint notes. |
Some upstream responses may be blocked, challenged, empty, redirected, or otherwise unusable. Crawlora aims to return clear failure context for supported endpoints instead of silently returning bad data.
Use environment variables for secrets and keep Crawlora API keys server-side.
async function crawloraWithRetry(url: string, init: RequestInit, attempts = 3) {
for (let attempt = 1; attempt <= attempts; attempt += 1) {
const response = await fetch(url, init);
if (response.ok) return response;
if (![429, 500, 502, 503, 504].includes(response.status) || attempt === attempts) {
throw new Error(`Crawlora request failed: ${response.status} ${await response.text()}`);
}
const delayMs = Math.min(1000 * 2 ** (attempt - 1), 8000);
await new Promise((resolve) => setTimeout(resolve, delayMs));
}
throw new Error("Retry loop exited unexpectedly");
}Crawlora is designed for responsible structured public web data workflows. Customers are responsible for using Crawlora in compliance with applicable laws, third-party rights, target-platform rules, and Crawlora terms.
Read Crawlora termsUse the endpoint docs first. If no precise schema is documented, expect an HTTP status plus a JSON or text body with failure context when available.
Retry temporary 5xx, timeout, and some 429 responses with capped backoff. Do not retry invalid requests until the input is fixed.
The request exceeded a plan, endpoint, or throughput limit. Reduce concurrency and wait before retrying.
Crawlora detects challenged, blocked, or unusable upstream responses for supported endpoints and surfaces failure context when the response cannot be used.
Credit behavior should be checked on pricing and endpoint docs because billing policy can vary by successful response and endpoint metadata.
Check auth, endpoint path, input schema, plan limits, response body, request IDs, and Playground behavior.
The platform may have returned no matching data, paginated data, or an unusable upstream response. Review endpoint response notes and schema docs.
Pair endpoint-specific schema docs with rate-limit handling and capped retries before production.