Docs menu

Datasets API endpoint

Tech Stack Dataset Item API

Returns one site by its domain from dataset id enum value `techstack`, including every detected technology (name, categories, confidence, version, evidence) plus the CMS / e-commerce / CDN / web-server / server-language rollups. Also flags whether the domain looks like backend CDN/DNS/cloud-vendor infrastructure rather than a real, human-navigable site (see `is_infrastructure`). Returns 404 when the domain is not in the index.

GETapiKey1 credit/requestDatasetsdatasets.techstackItemResponseDoc/datasets/techstack/items/{domain}

Endpoint facts

Method: GET
API path: /datasets/techstack/items/{domain}
Auth: apiKey
Credits: 1 credit/request
Charged on: Successful 2xx responses only
Platform: Datasets
Execution: Stored dataset records. No live scraping, browser session or proxy routing.
Response schema: #/definitions/datasets.techstackItemResponseDoc

Stored structured dataset search API Try Tech Stack Dataset Item API in Playground Datasets API documentation

How this endpoint behaves

Returns one site by its domain from the website tech-stack index, including every detected technology and the platform rollups. Returns 404 when the domain is not in the index.

Dataset id enum:

techstack

Path parameters:

domain: the site domain, e.g. shopify.com. A leading scheme and www. are stripped, so https://www.shopify.com/ resolves to the same record.

Response behavior:

The item carries domain, tld, rank, is_infrastructure (true when the domain looks like backend CDN/DNS/cloud-vendor infrastructure — e.g. cloudfront.net, gtld-servers.net — rather than a real, human-navigable site; a best-effort heuristic classification, so false/absent is not a guarantee the domain is a real site), reachable, status (HTTP status of the last probe attempt, when one was made), technologies (each with name, categories, confidence, version, evidence), technology_names, categories, tech_count, the cms/ecommerce/cdn/web_server/server_language rollups, render_tier, detector_version, and the crawl provenance (run_id, crawled_at).
When the last probe attempt didn't fully pass, the item also carries failure_reason (a short classification, e.g. "blocked/challenge markers found", "bad status 503", "request failed"), probe_error (the first line of the underlying error, when there was one), and block_matched (which block/challenge marker(s) were found, when failure_reason is a block/challenge classification). This can appear even when reachable is true — a blocked page returning a non-5xx status still counts as reachable while carrying a failure reason. Absent for a site that fully passed.
When the crawl looked for extra non-homepage pages (opt-in per run), the item also carries secondary_pages_discovered (count of candidate pages found on the homepage nav, regardless of fetch outcome), secondary_pages_fetched (count that were actually fetched successfully and contributed content — always <= discovered), secondary_page_categories (which categories were successfully fetched and contributed, e.g. login, docs, cart), secondary_pages (the full per-page audit trail — every discovered candidate, each entry {category, url, fetched}; fetched: false means the candidate was found on the homepage nav but the fetch itself failed, e.g. blocked or timed out), and secondary_only_technology_names (technology names detected only via a fetched secondary page, not the homepage itself — a subset of technology_names). Absent when no secondary pages were discovered for that run.
Returns the standard not-found envelope when the domain is not in the index.
Does not trigger live scraping; for a live, on-demand fingerprint of any URL use /web/techstack.

Sampled response

Request, 14 Oct 2026: GET /datasets/techstack/items/shopify.com
Items in technologies: 4
Fields filled in every checked item: 4

Field	Filled in, of 4 checked	Values in the sample
name	4
categories	4	median 1 entry
confidence	4
evidence	4

Request parameters

Parameter	Type	Required	Default	Description	Example
domain (path)	string	Yes		Site domain, e.g. shopify.com (a scheme and www. are stripped)	shopify.com
x-api-key (header)	string	Yes		API key required

cURL example

curl -X GET "https://api.crawlora.net/api/v1/datasets/techstack/items/shopify.com" \
  -H "x-api-key: $CRAWLORA_API_KEY"

Example JSON response

{
  "code": 200,
  "msg": "OK",
  "data": {
    "domain": "example-store.com",
    "tld": "com",
    "rank": 5231,
    "reachable": true,
    "technologies": [
      {
        "name": "Shopify",
        "categories": [
          "Ecommerce"
        ],
        "confidence": "high",
        "evidence": "markup: cdn.shopify.com"
      },
      {
        "name": "Google Analytics",
        "categories": [
          "Analytics"
        ],
        "confidence": "high",
        "evidence": "script: googletagmanager.com/gtag/js"
      },
      {
        "name": "Cloudflare",
        "categories": [
          "CDN"
        ],
        "confidence": "high",
        "evidence": "header: cf-ray"
      }
    ],
    "technology_names": [
      "Shopify",
      "Google Analytics",
      "Cloudflare",
      "Afterpay"
    ],
    "categories": [
      "Analytics",
      "CDN",
      "Ecommerce",
      "Payment"
    ],
    "tech_count": 4,
    "ecommerce": "Shopify",
    "cdn": "Cloudflare",
    "secondary_pages_discovered": 2,
    "secondary_pages_fetched": 1,
    "secondary_page_categories": [
      "cart"
    ],
    "secondary_pages": [
      {
        "category": "cart",
        "url": "https://example-store.com/cart",
        "fetched": true
      },
      {
        "category": "docs",
        "url": "https://example-store.com/docs",
        "fetched": false
      }
    ],
    "secondary_only_technology_names": [
      "Afterpay"
    ],
    "render_tier": "http",
    "detector_version": "2026.07.4",
    "run_id": "top1m-2026-07-08"
  }
}

Request schema

No body schema

Response schema

#/definitions/datasets.techstackItemResponseDoc

Field	Type	Required	Example	Description
code	integer	No	200
data	es.TechstackRecord	No
data.analytics	array	No		Analytics, tag manager and tracking pixel technologies detected on the site.
data.block_matched	array	No		block/challenge marker strings that matched, when FailureReason is a block/challenge classification
data.categories	array	No		distinct, sorted; category facets
data.category	string	No		optional seed category (usually empty)
data.cdn	string	No		Main CDN, hosting or platform provider detected on the site.
data.cms	string	No		single-value rollups (denormalized from Technologies for cheap facets/filters)
data.detector_version	string	No		Version of the technology detection rules used for this record.
data.domain	string	No		identity
data.ecommerce	string	No		Ecommerce platform detected on the site.
data.failure_reason	string	No		failure diagnostics: populated whenever a probe attempt didn't fully pass, even if the site still counts as reachable (e.g. a blocked page returning a non-5xx status). Unset for a fully passing site.
data.final_url	string	No		URL that produced the adopted body (after www. fallback / redirects)
data.has_captcha	boolean	No		Whether a CAPTCHA technology was detected on the site.
data.is_infrastructure	boolean	No		IsInfrastructure flags a backend CDN/DNS/cloud-vendor/registry hostname (e.g. cloudfront.net, gtld-servers.net) rather than a real, human-navigable website. Best-effort heuristic: a false negative (unflagged) is not proof a domain is a real site.
data.method_version	string	No		Version of the scan procedure used to collect this record.
data.probe_error	string	No		raw transport error's first line (e.g. "context deadline exceeded", "no such host"), when there was one
data.probed_at	string	No		RFC3339 — when last probed
data.rank	integer	No		Tranco rank when seeded from the top-1M list
data.reachable	boolean	No		Whether the site answered the probe.
data.render_tier	string	No		provenance / reproducibility
data.run_date	string	No		Time the scan run started, in UTC (RFC 3339).
data.run_id	string	No		Identifier of the scan run that produced this record.
data.scan_method	string	No		Method used to scan the site, for example `http_scan`.
data.schema_version	integer	No		Version of the record format.
data.scheme	string	No		URL scheme the site was reached with, `https` or `http`.
data.secondary_only_technology_names	array	No		technology names detected ONLY via a fetched secondary page, not the homepage alone; a subset of TechnologyNames
data.secondary_page_categories	array	No		which categories were successfully fetched and contributed content, sorted (careers\|login\|blog\|status\|signup\|docs\|contact\|trust\|cart)
data.secondary_pages	array	No		full audit trail: every discovered candidate, fetched or not; detail view, stored not indexed
data.secondary_pages[].category	string	No		Kind of page found on the homepage navigation: `careers`, `login`, `blog`, `status`, `signup`, `docs`, `contact`, `trust` or `cart`.
data.secondary_pages[].fetched	boolean	No		Whether the page was loaded successfully. False when it was blocked or timed out.
data.secondary_pages[].url	string	No		Address of the page found on the homepage navigation.
data.secondary_pages_discovered	integer	No		secondary-page telemetry: populated when the crawl looked for extra non-homepage pages (opt-in per run) and discovered at least one on the homepage nav for this domain. "Discovered" and "fetched" are deliberately distinct: a candidate can be found on the homepage nav but still fail to fetch (e.g. blocked or timed out) — that failure is recorded in SecondaryPages (Fetched=false), not silently dropped.
data.secondary_pages_fetched	integer	No		count that were ACTUALLY fetched successfully and contributed content; <= SecondaryPagesDiscovered
data.seed_source	string	No		tranco \| internal-harvest
data.server_language	string	No		Server-side language or web framework detected on the site.
data.source_url	string	No		the exact seed URL probed
data.status	integer	No		HTTP status code of the last probe attempt.
data.tech_count	integer	No		Number of distinct technologies detected on the site.
data.technologies	array	No		detections (the payload)
data.technologies[].categories	array	No		Categories the technology belongs to, such as `CDN` or `Analytics`.
data.technologies[].confidence	string	No		How sure the detection is: `high`, `medium` or `low`.
data.technologies[].evidence	string	No		Signal the technology was recognised from, such as `header: cf-ray`.
data.technologies[].name	string	No		Technology name, such as `Shopify` or `Cloudflare`.
data.technologies[].version	string	No		Version of the technology, when the site exposes it.
data.technology_names	array	No		flattened for AND/NOT term filters + facets
data.tld	string	No		Top-level domain of the site, such as `com`.
data.web_server	string	No		Web server software detected on the site.
msg	string	No	OK

Failure responses

Status	Description	Schema
400	Bad Request	#/definitions/app.Response
404	Not Found	#/definitions/app.Response
429	Too Many Requests	#/definitions/app.Response
500	Internal Server Error	#/definitions/app.Response

Code examples

Use environment variables for secrets and keep Crawlora API keys server-side.

curl -X GET "https://api.crawlora.net/api/v1/datasets/techstack/items/shopify.com" \
  -H "x-api-key: $CRAWLORA_API_KEY"

Shared guides

Authentication Errors and retries Rate limits and credits Pricing Acceptable use

More Datasets endpoint docs Stored structured dataset search API Stored Datasets List API documentation Airbnb Markets Dataset Facets API documentation Airbnb Markets Dataset Country API documentation Nearby Airbnb Markets Dataset API documentation Airbnb Markets Dataset Search API documentation Facet Apple Podcasts shows dataset API documentation

Open in Playground Back to group

Tech Stack Dataset Item API

Endpoint facts

Method

GET

API path

/datasets/techstack/items/{domain}

Auth

apiKey

Credits

1 credit/request

Charged on

Successful 2xx responses only

Platform

Datasets

Execution

Stored dataset records. No live scraping, browser session or proxy routing.

Response schema

#/definitions/datasets.techstackItemResponseDoc

How this endpoint behaves

Returns one site by its domain from the website tech-stack index, including every detected technology and the platform rollups. Returns 404 when the domain is not in the index.

Dataset id enum:

techstack

Path parameters:

domain: the site domain, e.g. shopify.com. A leading scheme and www. are stripped, so https://www.shopify.com/ resolves to the same record.

Response behavior:

The item carries domain, tld, rank, is_infrastructure (true when the domain looks like backend CDN/DNS/cloud-vendor infrastructure — e.g. cloudfront.net, gtld-servers.net — rather than a real, human-navigable site; a best-effort heuristic classification, so false/absent is not a guarantee the domain is a real site), reachable, status (HTTP status of the last probe attempt, when one was made), technologies (each with name, categories, confidence, version, evidence), technology_names, categories, tech_count, the cms/ecommerce/cdn/web_server/server_language rollups, render_tier, detector_version, and the crawl provenance (run_id, crawled_at).

When the last probe attempt didn't fully pass, the item also carries failure_reason (a short classification, e.g. "blocked/challenge markers found", "bad status 503", "request failed"), probe_error (the first line of the underlying error, when there was one), and block_matched (which block/challenge marker(s) were found, when failure_reason is a block/challenge classification). This can appear even when reachable is true — a blocked page returning a non-5xx status still counts as reachable while carrying a failure reason. Absent for a site that fully passed.

When the crawl looked for extra non-homepage pages (opt-in per run), the item also carries secondary_pages_discovered (count of candidate pages found on the homepage nav, regardless of fetch outcome), secondary_pages_fetched (count that were actually fetched successfully and contributed content — always <= discovered), secondary_page_categories (which categories were successfully fetched and contributed, e.g. login, docs, cart), secondary_pages (the full per-page audit trail — every discovered candidate, each entry {category, url, fetched}; fetched: false means the candidate was found on the homepage nav but the fetch itself failed, e.g. blocked or timed out), and secondary_only_technology_names (technology names detected only via a fetched secondary page, not the homepage itself — a subset of technology_names). Absent when no secondary pages were discovered for that run.

Returns the standard not-found envelope when the domain is not in the index.

Does not trigger live scraping; for a live, on-demand fingerprint of any URL use /web/techstack.

Field

Filled in, of 4 checked

Values in the sample

name

categories

median 1 entry

confidence

evidence

Parameter

Type

Required

Default

Description

Example

domain (path)

string

Yes

Site domain, e.g. shopify.com (a scheme and www. are stripped)

shopify.com

x-api-key (header)

string

Yes

API key required

{ "code": 200, "msg": "OK", "data": { "domain": "example-store.com", "tld": "com", "rank": 5231, "reachable": true, "technologies": [ { "name": "Shopify", "categories": [ "Ecommerce" ], "confidence": "high", "evidence": "markup: cdn.shopify.com" }, { "name": "Google Analytics", "categories": [ "Analytics" ], "confidence": "high", "evidence": "script: googletagmanager.com/gtag/js" }, { "name": "Cloudflare", "categories": [ "CDN" ], "confidence": "high", "evidence": "header: cf-ray" } ], "technology_names": [ "Shopify", "Google Analytics", "Cloudflare", "Afterpay" ], "categories": [ "Analytics", "CDN", "Ecommerce", "Payment" ], "tech_count": 4, "ecommerce": "Shopify", "cdn": "Cloudflare", "secondary_pages_discovered": 2, "secondary_pages_fetched": 1, "secondary_page_categories": [ "cart" ], "secondary_pages": [ { "category": "cart", "url": "https://example-store.com/cart", "fetched": true }, { "category": "docs", "url": "https://example-store.com/docs", "fetched": false } ], "secondary_only_technology_names": [ "Afterpay" ], "render_tier": "http", "detector_version": "2026.07.4", "run_id": "top1m-2026-07-08" } }

Field

Type

Required

Enum

Bounds

Example

Description

code

integer

200

data

es.TechstackRecord

data.analytics

array

Analytics, tag manager and tracking pixel technologies detected on the site.

data.block_matched

array

block/challenge marker strings that matched, when FailureReason is a block/challenge classification

data.categories

array

distinct, sorted; category facets

data.category

string

optional seed category (usually empty)

data.cdn

string

Main CDN, hosting or platform provider detected on the site.

data.cms

string

single-value rollups (denormalized from Technologies for cheap facets/filters)

data.detector_version

string

Version of the technology detection rules used for this record.

data.domain

string

identity

data.ecommerce

string

Ecommerce platform detected on the site.

data.failure_reason

string

failure diagnostics: populated whenever a probe attempt didn't fully pass, even if the site still counts as reachable (e.g. a blocked page returning a non-5xx status). Unset for a fully passing site.

data.final_url

string

URL that produced the adopted body (after www. fallback / redirects)

data.has_captcha

boolean

Whether a CAPTCHA technology was detected on the site.

data.is_infrastructure

boolean

IsInfrastructure flags a backend CDN/DNS/cloud-vendor/registry hostname (e.g. cloudfront.net, gtld-servers.net) rather than a real, human-navigable website. Best-effort heuristic: a false negative (unflagged) is not proof a domain is a real site.

data.method_version

string

Version of the scan procedure used to collect this record.

data.probe_error

string

raw transport error's first line (e.g. "context deadline exceeded", "no such host"), when there was one

data.probed_at

string

RFC3339 — when last probed

data.rank

integer

Tranco rank when seeded from the top-1M list

data.reachable

boolean

Whether the site answered the probe.

data.render_tier

string

provenance / reproducibility

data.run_date

string

Time the scan run started, in UTC (RFC 3339).

data.run_id

string

Identifier of the scan run that produced this record.

data.scan_method

string

Method used to scan the site, for example `http_scan`.

data.schema_version

integer

Version of the record format.

data.scheme

string

URL scheme the site was reached with, `https` or `http`.

data.secondary_only_technology_names

array

technology names detected ONLY via a fetched secondary page, not the homepage alone; a subset of TechnologyNames

data.secondary_page_categories

array

data.secondary_pages

array

full audit trail: every discovered candidate, fetched or not; detail view, stored not indexed

data.secondary_pages[].category

string

Kind of page found on the homepage navigation: `careers`, `login`, `blog`, `status`, `signup`, `docs`, `contact`, `trust` or `cart`.

data.secondary_pages[].fetched

boolean

Whether the page was loaded successfully. False when it was blocked or timed out.

data.secondary_pages[].url

string

Address of the page found on the homepage navigation.

data.secondary_pages_discovered

integer

secondary-page telemetry: populated when the crawl looked for extra non-homepage pages (opt-in per run) and discovered at least one on the homepage nav for this domain. "Discovered" and "fetched" are deliberately distinct: a candidate can be found on the homepage nav but still fail to fetch (e.g. blocked or timed out) — that failure is recorded in SecondaryPages (Fetched=false), not silently dropped.

data.secondary_pages_fetched

integer

count that were ACTUALLY fetched successfully and contributed content; <= SecondaryPagesDiscovered

data.seed_source

string

tranco | internal-harvest

data.server_language

string

Server-side language or web framework detected on the site.

data.source_url

string

the exact seed URL probed

data.status

integer

HTTP status code of the last probe attempt.

data.tech_count

integer

Number of distinct technologies detected on the site.

data.technologies

array

detections (the payload)

data.technologies[].categories

array

Categories the technology belongs to, such as `CDN` or `Analytics`.

data.technologies[].confidence

string

How sure the detection is: `high`, `medium` or `low`.

data.technologies[].evidence

string

Signal the technology was recognised from, such as `header: cf-ray`.

data.technologies[].name

string

Technology name, such as `Shopify` or `Cloudflare`.

data.technologies[].version

string

Version of the technology, when the site exposes it.

data.technology_names

array

flattened for AND/NOT term filters + facets

data.tld

string

Top-level domain of the site, such as `com`.

data.web_server

string

Web server software detected on the site.

msg

string

Status

Description

Schema

400

Bad Request

#/definitions/app.Response

404

Not Found

#/definitions/app.Response

429

Too Many Requests

#/definitions/app.Response

500

Internal Server Error

#/definitions/app.Response

Tech Stack Dataset Item API

Endpoint facts

How this endpoint behaves

Sampled response

Request parameters

cURL example

Example JSON response

Failure responses

Code examples

Shared guides

Related

Tech Stack Dataset Item API

Endpoint facts

How this endpoint behaves

Sampled response

Request parameters

cURL example

Example JSON response

Failure responses

Code examples

Shared guides

Related