Cursor pagination is a way to traverse a large result set by returning an opaque continuation value with each page. The client sends that cursor with the next request — commonly as after in GraphQL or as a next-page token in REST — and stops when the response says no next page remains.
A GraphQL connection commonly returns edges, each edge's cursor, and pageInfo fields such as endCursor and hasNextPage. The client requests a bounded first page, processes it, then passes endCursor as after for the next page. REST APIs use different names — next, cursor, continuation_token — but the loop is the same: follow the server-issued token rather than inventing the next position.
Treat the cursor as opaque. It may encode an offset, a record key, a timestamp, or internal state, and decoding it creates a dependency on an implementation the server can change. Preserve the same filters, sort order, page size, authentication, and other request context when reusing it, because a cursor from one query is not safely portable to another.
Offset pagination asks for positions such as page=7 or offset=300. It is easy to inspect and jump around, but inserts or deletions before that position can shift later pages and cause records to be skipped or repeated. A cursor gives the server a continuation point tied to its ordering, which often makes forward traversal more stable and efficient on changing datasets.
That advantage is not a snapshot guarantee. A cursor can itself encode an offset, and records can still move, appear, or disappear while a long scrape runs. Deduplicate by a stable record ID and define how updates to an already-seen record should be handled instead of assuming cursor pagination produces exactly-once delivery.
Commit the current page's records before saving its continuation cursor. If the process crashes after the data commit but before the cursor checkpoint, replaying the page is safe when writes are idempotent; saving the cursor first can skip an entire uncommitted page forever. Store the query context beside the checkpoint so a resumed run cannot accidentally apply the token to changed filters or ordering.
Stop when hasNextPage is false or the API omits its documented next token. Also guard against a repeated cursor, a claimed next page with no continuation value, and an excessive page count. Those conditions usually indicate a server bug or a client-context mismatch; looping blindly can burn quota while collecting the same page repeatedly.
How Crawlora handles this
Crawlora documents pagination on each platform endpoint that supports it. Follow that endpoint's response fields and request parameters rather than assuming one universal cursor name or pagination contract across every source.
Related reading
Glossary
FAQ
No. It is an opaque value whose format belongs to the server. It may encode an ID, offset, timestamp, or other state, but clients should store and replay it without depending on that encoding.
No. It often behaves better than offsets while data changes, but it is not automatically a consistent snapshot. Deduplicate on stable IDs and design the importer to tolerate records moving between pages.
After the page's records have been committed successfully. With idempotent writes, a crash before the checkpoint only replays data; checkpointing before the commit risks permanently skipping that page.
Beyond Cursor Pagination, Crawlora's own docs cover the rest of the stack — browse the APIs, test a request in Playground, and move from scraping infrastructure work to production data workflows.