Crawlora Changelog

Published a landing page for the stored Google Maps business index, with browsable coverage aggregates alongside the existing search, facets, item-lookup, and radius endpoints.

• Headline coverage figures — listing total and the share carrying a website, phone number, and geocoordinates — are derived from the dataset's own snapshot at render time rather than typed in by hand.
• Country, category, and US-state breakdowns render as real HTML tables, so answer engines and scripts can read them without executing the charts.
• Rating and review-volume distributions show how much of the index is actively reviewed, and how much is uncategorized.
• Every figure on the page carries the snapshot date it was derived from.
• Links straight through to the dataset endpoints for search, facets, item lookup, and radius queries.

Clamped deep pagination on the Anti-Bot Index route so an out-of-range page parameter can no longer fail the request.

• The requested page is clamped to the last available page before the search query runs, rather than after the query has already been rejected.
• Search engines that had picked up deep, out-of-range paginated URLs now receive a normal page instead of an error.

Added a US state choropleth to the State of Local Business on the Web study, plus a dedicated county page with an interactive county map, the complete county table, and a data download.

• The state choropleth colours every US state by the share of its listings that carry a website.
• A separate county page maps 3,131 counties covering roughly 19.5 million listings, colouring only counties with enough listings to be meaningful and greying out the rest.
• The complete county table ships as a real HTML table in the page, so the numbers are readable without running the map.
• County aggregates are downloadable as JSON, alongside the boundary geometry the map joins against.
• Aggregates only — counts and shares per county, never individual business records.
• Each county row sums every spelling variant the source data uses for that county, so places listed under more than one name are not split in two.

Added Apple Maps as a new platform: place search and detail, reverse geocoding, directions and travel-time estimates, live transit departures, indoor venue directories, and Apple Guides.

• Keyword and category place search around a coordinate, typeahead suggestions, and full place detail — singly or twenty at a time — with the complete photo set.
• Reverse geocoding, plus driving, walking, and cycling directions, with a cheaper travel-time-only variant when you do not need the route itself.
• Live transit departures for a stop, and indoor venue directories for malls and airports.
• The whole Apple Guides editorial layer, browsable by city, by publisher, or by coordinate.
• Generates matching methods across the six SDKs and is covered by the open-source MCP tool catalog.

Added four delivery marketplaces: Swiggy and Zomato in India, Wolt across its European markets, and 7NOW for 7-Eleven convenience delivery.

• Swiggy: restaurant and dish search around a latitude/longitude with veg, rating, offer, and sort filters, the curated collections its own homepage surfaces, restaurant detail, and full priced menus.
• Zomato: city-wide listings with popularity, rating, and cost sorting and cursor pagination, in-city keyword search, curated editorial collections, and restaurant detail with menus.
• Wolt: a live directory of every city Wolt operates in, coordinate venue search with category and store-type filters, venue menus, and live availability with delivery and pickup estimates, fees, order minimums, and current discount campaigns.
• 7NOW: address-to-store resolution, per-store search and a browsable department and aisle catalog with real pricing and live stock, discount listings that name the promotion behind each price, and product reference records with nutrition, ingredients, and allergens.
• All four generate matching methods across the six SDKs and are covered by the open-source MCP tool catalog.

Added Five Guys, Pizza Hut, and Zaxby's as new platforms, each with a store locator and menus carrying that restaurant's own prices.

• Five Guys: free-text and coordinate locator search with in-store hours, a separate delivery schedule and order links, a three-level state/city/restaurant directory, the published marketing menu plus the ordering platform's real per-restaurant priced menu, a large customer FAQ corpus, and a live-resolved link to the current nutrition and allergen guide.
• Pizza Hut: restaurant search by city, state, postal code, name, franchise code, or store number with per-channel hours, one restaurant's full priced menu, and the build-your-own tree down to per-topping weight and left/right/whole placement with each option's real price delta.
• Pizza Hut also exposes the choice-builder behind a deal or combo, and a checkout-time delivery serviceability and fee estimate.
• Zaxby's: coordinate proximity search read anonymously with no key or cookie, per-store detail with per-channel hours and fulfillment flags, and the complete national reference menu with a recursive size, protein, and sauce customization tree carrying its own price deltas.
• All three generate matching methods across the six SDKs and are covered by the open-source MCP tool catalog.

Added GOAT, with faceted resale catalog search and live per-size pricing, and Kroger, with faceted grocery search, bulk product detail, and digital coupons.

• GOAT: faceted search across sneakers, apparel, accessories, bags, and collectibles by brand, colorway, gender, condition, release year, and price band.
• GOAT product detail combines descriptive metadata with live per-size lowest-ask, Instant Ship, last-sold, and highest-offer pricing, plus its own curated editorial collections and the recommendations its product pages surface.
• Kroger: keyword search and category browsing returning priced product cards with unit price, size, SNAP EBT eligibility, and low-stock flags, filterable on brand, dietary, flavour, scent, savings, and price.
• Kroger bulk product detail resolves up to fifty items at a time with nutrition facts, ingredients, allergens, store-specific price and stock, and the full rating histogram, alongside reviews, refinement chips, digital coupons, and store detail.
• Both generate matching methods across the six SDKs and are covered by the open-source MCP tool catalog.

Added Arby's, Culver's, Jimmy John's, Papa John's, Raising Cane's, Sonic Drive-In, and Whataburger as new platforms.

• Arby's: a state/city browse of the whole US chain, coordinate proximity search with distance, direct store lookup, and menu items carrying a full per-serving nutrition panel plus real per-store pricing when a store id is supplied.
• Papa John's: per-restaurant priced menus and current promotions across fourteen markets, national nutrition-facts and allergen reference data, ingredient catalogs, and separate priced menus for India, Colombia, Peru, El Salvador, Poland, and Russia.
• Culver's: a directory of roughly 1,121 restaurants, national menu categories, item detail with size and topping modifier trees, and per-restaurant detail including today's Flavor of the Day.
• Jimmy John's and Whataburger: store indexes read from each chain's own published sitemap — 9,700-plus store URLs for Jimmy John's with coordinate proximity search and menus down to per-ingredient extra/regular/light/none choices, and roughly 3,950 typed URLs for Whataburger with full per-store detail.
• Raising Cane's and Sonic Drive-In: directory trees and coordinate proximity search, full per-store detail with per-channel hours, the complete national menus, and each chain's current promotions with their published official rules.
• All seven generate matching methods across the six SDKs and are covered by the open-source MCP tool catalog.

Added Bonhams for auction and lot data including realized prices, Rover for pet sitters, walkers, and trainers, and Tes for the UK education marketplace.

• Bonhams: auction search across upcoming, past, or all sales, one auction's normalized facts and its full lot listing, and lot search across every auction with department, country, and estimate-range filters.
• Each Bonhams lot carries its estimate, its hammer and realized price once sold, and the sale date — so historical results are queryable, not just live catalogs.
• Rover: sitter and dog-walker search by location and service type, trainer search, and normalized provider profiles with star rating, review count, repeat-client count, and years of experience.
• Tes: teaching-job search with the site's own live faceted breakdown, job and employer detail, teaching-resource search and detail with attachment metadata and reviews, author shops, and the Schools Directory.
• All three generate matching methods across the six SDKs and are covered by the open-source MCP tool catalog.

Added five delivery marketplaces, each with location search, partner detail, and full menus carrying that partner's real prices.

• Deliveroo: restaurant, grocery, and shopping-store search by coordinate, each partner's full priced catalog with resolved customization options, live sort and filter catalogs with per-location result counts, and delivery and pickup scheduling windows.
• Grubhub: restaurant search and detail, menus whose delivery and pickup pricing genuinely differ, ordering availability across up to twenty restaurants at once, promotional offers, reviews with rating histograms, and the full future orderable time-slot calendar.
• Just Eat: full restaurant listings for a UK postcode in Just Eat's own default sort order, restaurant profiles, and priced menus with customization option groups.
• foodpanda: restaurant search by coordinate and cuisine across its Asia-Pacific markets, full menus with original and discounted pricing, restaurant profiles, and a curated sample of customer reviews.
• pandamart: darkstore search by coordinate, per-store detail with delivery and pickup availability and minimums, the store's full category tree, curated shelves, product detail by ID, and keyword search within one darkstore's catalog.
• All five generate matching methods across the six SDKs and are covered by the open-source MCP tool catalog.

Added Chick-fil-A, KFC, Shake Shack, Wendy's, and Wingstop as new platforms, each with a store surface and menus carrying real per-restaurant prices.

• Wendy's: a country-to-state-to-city-to-store directory across the US and Canada, coordinate and address proximity search, the national day-part menu, one restaurant's own priced item list with combo bundle prices, per-component allergen detail, full nutrition panels, and mobile-order time slots.
• Chick-fil-A: a 3,467-restaurant directory with address and hours, a 567-item menu catalog with taxonomy filters, a 522-entry customer FAQ corpus, and the full editorial family of press releases, stories, legal documents, and press-kit assets.
• KFC: restaurant search and lookup by store number, per-channel priced menus, delivery serviceability and fee estimates for a real address, and public promotions by restaurant or redemption code.
• Shake Shack: a store index parsed from the site's own map and text-list sections, coordinate search against the live ordering API with real-time fulfillment-mode availability, per-store detail, and menus with customization trees and calorie deltas.
• Wingstop: a country-to-store directory browse, coordinate search, delivery-address-to-store resolution, per-channel hours, each store's priced carryout or delivery menu, and the complete sauce and dry-rub flavour catalog.
• All five generate matching methods across the six SDKs and are covered by the open-source MCP tool catalog.

The site's error boundary now recognises a stale-script failure and does a full page reload instead of offering a manual retry that could not have fixed it.

• Only stale-script failures trigger the automatic reload; every other error still renders the normal error screen with its retry action.
• A short per-session cooldown prevents a reload loop in the event that the newly published build is itself failing.
• The retry that used to be offered could never succeed on its own, because the browser had already cached the failed script request as permanently rejected.

Added McDonald's, Chipotle, Taco Bell, Dunkin', and Subway as new platforms, covering store directories, menu catalogs, and store-scoped prices.

• McDonald's: restaurant locations across ten markets with hours and amenities, menu categories and items across eight markets, one US restaurant's real eat-in, pickup, and delivery prices, and per-item nutrition detail for up to twenty items in a single call.
• Chipotle: restaurant locations with capability flags for Chipotlane, catering, and curbside, the national menu catalog and preset meals, restaurant-specific live pricing for both, and the full ingredient-sourcing catalog.
• Taco Bell: US restaurant locations with capability flags, national and store-scoped menus carrying price and calorie data together, per-item customization matrices, and full nutrition-facts panels.
• Dunkin': a directory browse across all forty-four US states it operates in, coordinate proximity search with distance and amenity labels, per-store detail including the third-party delivery platforms each store is live on, and the national menu catalog.
• Subway: over 22,700 store URLs read from its own published index across a country/region/city/street tree, full per-store detail, and each store's complete priced menu with nutrition panels and allergen disclosures per size.
• All five generate matching methods across the six SDKs and are covered by the open-source MCP tool catalog.

Added Burger King, Domino's, Panera Bread, and Popeyes as new platforms, each with a store surface, priced menus, and their current promotions.

• Burger King: restaurant locations with per-channel amenity flags, live per-channel open/closed status, restaurant-specific priced menus, and full item and combo customization trees.
• Domino's: a store locator by address, city and state, or postal code, store profiles with per-service-method hours and wait windows, structured menus with variant pricing, each store's active coupons, and the full build-your-own catalog of sizes, crusts, toppings, and sides.
• Panera Bread: the full US cafe locator tree, per-cafe menus and catering menus with genuinely per-cafe pricing and nutrition, item customization options, order time-slots, quantity limits, retired and seasonal items, upsell suggestions, and the separate Panera at Work workplace-delivery program.
• Popeyes: restaurant locations with amenity flags and published hours, direct store lookup, per-restaurant menus with real pricing where Popeyes reports it, the current catalog of named value deals and coupons, and the published customer-support FAQ.
• Popeyes also exposes the full Popeyes Rewards loyalty catalog — bonus-points promotions, multi-step quests, and points-redemption rewards.
• All four generate matching methods across the six SDKs and are covered by the open-source MCP tool catalog.

Added CVS as a new platform covering store locations and the over-the-counter retail catalog, deliberately scoped away from pharmacy and patient data.

• Store locations by postal code or coordinate, with service indicators per store.
• The full brand and category taxonomy, plus keyword search and category browsing across the non-prescription retail catalog.
• Per-product detail with ingredient and label statements.
• Scoped to general store information and retail products only — no pharmacy hours and no patient-specific data of any kind.
• Requests are served from US and US-territory egress, matching where the catalog is published.
• Generates matching methods across the six SDKs and is covered by the open-source MCP tool catalog.

Added OpenSea, Rightmove, and Starbucks as new platforms, each returned as normalized JSON.

• OpenSea: collection floor prices and order books, individual NFT sale history and ownership, wallet activity and holdings, and marketplace-wide rankings and drops.
• Rightmove: UK location autocomplete, for-sale, to-let, and sold search with price and bedroom filters, dedicated commercial, new-homes, and student-accommodation searches, and estate-agent branch listings.
• Rightmove property detail includes floor plans, EPC rating, and nearby schools.
• Starbucks: store locations by coordinate, the full public menu, and per-product nutrition that recalculates for customizations like milk, size, and number of shots.
• All three generate matching methods across the six SDKs and are covered by the open-source MCP tool catalog.

Added Boots, the UK's largest pharmacy, health, and beauty retailer, and Cricinfo, ESPN's cricket scores and scorecard service.

• Boots: keyword and category search with dynamic facets, price and stock filtering, and search-box suggestions.
• Cricinfo: live and recent scores, full match detail, and ball-by-ball commentary.
• Cricinfo also covers series and venue schedules, squads and grounds, team and player statistics from Statsguru, and the records index.
• Editorial feeds round it out — news, stories, photos, videos, and RSS.
• Both generate matching methods across the six SDKs and are covered by the open-source MCP tool catalog.

Added Lazada, Otto, and Tokopedia as new platforms, each with category browsing, keyword search, and full product detail.

• Lazada: the category directory, homepage feed, keyword search, category listings, and product detail across six country storefronts.
• Otto: keyword search, the top-level category tree, and full product detail including pricing and availability for Germany's largest online retailer after Amazon.
• Tokopedia: keyword search with sorting and filters, category and homepage browsing, full product detail, review-filter metadata, and search-box autocomplete.
• All three generate matching methods across the six SDKs and are covered by the open-source MCP tool catalog.

Added BBC News, CNN, and The Guardian for article and headline extraction, and CourtListener for public court-opinion research.

• BBC News: full article body and metadata from a canonical article URL, plus fresh section headlines from its public RSS feeds.
• CNN: full article content with author and publish time, current section headline streams, and chronological updates from a CNN live story.
• The Guardian: full article content and metadata, section headlines from its public RSS feeds, and paginated topic and category archives.
• CourtListener: full-text search across published court opinions with normalized result metadata, and a browsable directory of US courts.
• All four generate matching methods across the six SDKs and are covered by the open-source MCP tool catalog.

Added Bilibili, Roblox, and Patreon as new platforms, covering curated shelves, rankings, and public creator surfaces.

• Bilibili: anime, Chinese-animation, and vertical homepage sections, curated must-watch and weekly picks, the current all-site ranking, the popular feed, and search-box suggestions — all readable without an account.
• Roblox: experience search, public game detail with aggregate votes and icon, cursor-paginated badge listings, and curated ranking shelves such as top-trending and top-playing-now.
• Patreon: creator identity and public earnings snapshot, published membership tiers and their benefits, curated creator shelves by topic, and canonical public podcast RSS feeds.
• All three generate matching methods across the six SDKs and are covered by the open-source MCP tool catalog.

Added Hotels.com for date-bound hotel search and rates, Leboncoin for France's largest classifieds marketplace, and SparkFun for its electronics storefront.

• Hotels.com: date-bound hotel search with per-night and per-stay prices, one property's live rates and room offers, the guest-review overview and a paginated review archive, and static property metadata — no login needed.
• Leboncoin: normalized public listing cards from a location search, and full public detail for one listing.
• SparkFun: category browsing with dynamic facet filters, product search, and per-SKU detail with price, stock status, images, and a cleaned description.
• All three generate matching methods across the six SDKs and are covered by the open-source MCP tool catalog.

Added Accor for hotel-group property metadata, and BigCommerce as a storefront-generic platform that works from just a storefront URL.

• Accor: typeahead and paginated search across the group's properties, destination and brand directories, a geographic catalog lookup, and an amenity reference catalog.
• Accor property records carry brand, city and country, amenities, and published check-in and check-out times — deliberately scoped to static public data, with no booking, rates, rooms, or review text.
• BigCommerce: full-catalog search, paginated category product grids, and per-product detail with pricing, availability, brand, description, and images, for any public Stencil storefront.
• Both generate matching methods across the six SDKs and are covered by the open-source MCP tool catalog.

Added TicketWeb as a new platform: event and venue search, plus event and venue detail with ticket availability.

• Search the public event catalog by artist, event, or venue.
• Resolve an event into a normalized record with dates, delivery methods, and per-tier ticket pricing.
• Resolve a venue into its own detail record.
• Generates matching methods across the six SDKs and is covered by the open-source MCP tool catalog.

Launched the Monitors API: create a monitor on a page URL or a sitemap URL, choose a check cadence from 5 minutes to 7 days, and receive signed webhook notifications when content changes or a check completes.

• Page mode does an exact content fingerprint diff on every check; sitemap mode tracks a sitemap's listed URLs for additions and removals, with optional include/exclude glob patterns and a max-URL cap.
• Webhook deliveries are signed with an HMAC-SHA256 signature header, so receivers can verify authenticity and reject stale or replayed requests.
• Subscribe to change.detected (fires only when a check finds a difference) and/or run.completed (a heartbeat on every check, useful for a 'last checked' UI).
• Full CRUD is available over both the REST API and as MCP tools, so an agent can create and manage its own monitors with its own API key with no dashboard interaction required.
• A new /app/monitors dashboard and /docs/monitors guide cover setup end to end alongside the API reference.

Launched Logo Link on its own host: a public image endpoint that turns a domain name into that company's logo, gated by a client id and a referrer allowlist instead of an API key.

• Works from an <img> element, a CSS background, or a React component; the response content type reflects what was actually served — PNG, SVG, JPEG, or WebP.
• Size, format, theme (auto, light, or dark), and greyscale parameters are all supported, and SVG sources are rasterized whenever an image parameter is present.
• When no logo can be extracted it renders a monogram fallback, or returns a 404 instead if you ask for one.
• Access is gated by a public client id plus a referrer allowlist, with an optional signed mode for server-side use; one account can hold several independently scoped client ids, each with its own allowlist and telemetry.
• Logo Link has its own monthly request quota per client id, kept separate from the core credit pool, so a heavy scraping month cannot break logos already embedded on your site — and rejected requests never count against it.
• A dashboard page and a docs guide cover setup end to end, and served logos come back with a week-long cache lifetime.

Added a device-approval page and the matching endpoints, so agent clients without a browser of their own can be connected to the hosted MCP server.

• The client displays a short user code; you open the approval page, sign in if you are not already, and confirm.
• A code passed in the approval URL is prefilled, so it does not have to be retyped.
• Sits alongside the existing browser connector flow — clients already connected are unaffected.

Added IKEA as a new platform: product search and detail, category browsing, real-time stock availability, highlighted customer reviews, search-box typeahead, and a physical store locator.

• Product search and full product detail, including pricing and category placement.
• Real-time item stock availability and highlighted customer reviews.
• Category browsing and search-box typeahead suggestions.
• A store locator covering IKEA's physical store network, plus per-store detail.
• Generates matching methods across the six SDKs and is covered by the open-source MCP tool catalog.

Added Audible, J.Crew, and Quince as new platforms, and a vehicle-listings dataset with search, facets, item lookup, and price-history endpoints.

• Audible: search and batch product lookup, full audiobook detail with narrators and series, customer reviews, related-title recommendations, category browsing, charts, and editorial lists — most calls read Audible's own public catalog API without requiring an Audible or Amazon account.
• J.Crew: category and search browsing across both the J.Crew and J.Crew Factory storefronts, full product detail with every color/size SKU, size charts with real body measurements, customer reviews, and a distance-sortable store locator.
• Quince: Algolia-backed category taxonomy and curated navigation, filterable product search and typeahead, product detail with color/size variant pricing and stock, reviews, product FAQ content, and sitemap URL discovery.
• Vehicle listings dataset: search, facets, item lookup, and price-history endpoints over a stored dataset of vehicle listings, queryable without triggering live scraping.
• All three platforms generate matching methods across the six SDKs and are covered by the open-source MCP tool catalog.

Added BBB, Chewy, GDELT, and Kickstarter as new platforms, spanning business trust ratings, pet-product retail, global news/TV coverage, and crowdfunding campaigns.

• BBB: business search and profile, complaint history, full rating reasons, customer reviews, category browsing, and Scam Tracker report search with state/province aggregate stats.
• Chewy: category taxonomy and listings, product search with typeahead suggestions, product detail and batch lookups, and GTIN/UPC barcode resolution.
• GDELT: search and sentence-level context search across global news coverage, coverage-volume and sentiment timelines, plus a Television 2.0 AI suite covering US TV transcripts, captions, stations, and word clouds.
• Kickstarter: campaign discovery and search, full campaign detail, comments, and the updates feed.
• All four platforms generate matching methods across the six SDKs and are covered by the open-source MCP tool catalog.

Added six brand-pinned storefront APIs covering product search, collection browsing, product detail with every size and color variant, recommendations, and sitemap discovery.

• Brooklinen: bedding and home textile catalog with full product detail and collection browsing.
• Rothy's: sustainable footwear catalog with per-size availability and collection browsing.
• The Body Shop: cosmetics and skincare catalog with product detail and recommendations.
• Allbirds: sustainable footwear catalog with per-size availability and predictive search suggestions.
• Kylie Cosmetics: beauty product catalog with product detail, collections, and recommendations.
• Gymshark: fitness apparel catalog served from its headless storefront, including live per-size inventory and filter facets.

Added six brand-pinned storefront APIs covering product search, collection browsing, product detail with every size and color variant, recommendations, and sitemap discovery.

• Fashion Nova: fast-fashion catalog with product detail, collections, and predictive search suggestions.
• SKIMS: shapewear and loungewear catalog with per-size availability and product recommendations.
• Steve Madden: footwear and accessories catalog with full product detail and collection browsing.
• Oh Polly: occasionwear catalog with product detail, collections, and predictive search suggestions.
• Everlane: apparel basics catalog with product detail, collections, and sitemap discovery.
• Cole Haan: footwear and leather accessories catalog with full product detail and collection browsing.

Added Adidas, SHEIN, and Walgreens, covering product search, category browsing, store locators, and trending keywords.

• Adidas: keyword search or category browsing with pagination and sort, per-SKU product detail with every purchasable size, and a store locator with per-store hours and services.
• SHEIN: keyword product search, product detail with sizes/stock and color variants, category listings and filter facets, search autocomplete, and trending search keywords.
• Walgreens: nearby store lookup by coordinates or ZIP code, with hours, phone, and in-store service indicators such as pharmacy and immunizations.
• All three generate matching methods across the SDKs and are covered by the open-source MCP tool catalog.

Added the Home Depot platform, covering department discovery, category and brand browsing, product detail, and product-question data.

• Department taxonomy from Home Depot's own top-level navigation.
• Category or brand page browsing, and product detail with name, brand, model, and pricing.
• Customer questions and answers by product id, with a total Q&A count.
• Generates matching methods across the SDKs and is covered by the open-source MCP tool catalog.

Added Best Buy and Sephora, covering category and keyword search, product detail, reviews, and related products.

• Best Buy: category or keyword search with pagination, product detail with price and availability, plus customer reviews, Q&A, and related products by SKU.
• Sephora: keyword search with pagination, and product detail with every color/shade variant's own price, availability, rating, and a sample of recent reviews.
• Both platforms generate matching methods across the SDKs and are covered by the open-source MCP tool catalog.

Added the DraftKings Sportsbook platform, covering league and event discovery, event context, and moneyline, spread, total, and futures odds.

• Discover leagues and events via leagues, featured-leagues, and quick-links endpoints, then resolve a league id into scheduled and live events.
• Event context returns participants, status, start time, sport, and league context for a single event.
• Odds and futures endpoints monitor moneyline, spread, total, and futures markets by league, with full event markets by event id.
• Generates matching methods across the SDKs and is covered by the open-source MCP tool catalog.

Added Google Patents and USPTO Patent Public Search, covering full-text search, patent bibliographic detail, classification lookup, and recent-publication browsing.

• Google Patents: full-text search with inventor, assignee, office, status, type, language, and date filters; single-patent bibliographic detail, abstract, claims, description, citations, and family members; CPC classification lookup; recent weekly publications; and inventor/assignee autocomplete.
• USPTO Patent Public Search: full-text search across granted patents and published applications with normalized bibliographic results, plus full document detail — abstract, description, and claims text.
• Both platforms generate matching methods across the SDKs and are covered by the open-source MCP tool catalog.

Added Twitch and SoundCloud, covering channel profiles, live streams, clips, and videos plus track, playlist, and profile search.

• Twitch: channel lookup, live stream browsing by game or category, clips, videos, search, top games, and team rosters.
• SoundCloud: track search and detail, playlist detail, user profiles, and a user's own uploaded tracks.
• Both platforms generate matching methods across the six SDKs and are covered by the open-source MCP tool catalog.

Added five more retail and beauty platforms, covering product search, category browsing, per-color and per-size pricing, customer reviews, and store locations.

• H&M: keyword search, category listings, and product detail with per-color live per-size stock and reviews.
• Macy's: full product detail with sale-aware pricing and every color variant's own price.
• Ulta Beauty: keyword search and product detail with pricing, ratings, reviews, and every color/shade variant.
• Sam's Club: warehouse-club product detail with pricing, availability, and aggregate rating.
• lululemon: category navigation, category listings, product detail, outfit recommendations, and the physical store directory.
• All five platforms generate matching methods across the six SDKs and are covered by the open-source MCP tool catalog.

Added six more Yahoo verticals, covering homepage and section editorial story feeds plus full article content, with retailer store deals on Yahoo Shopping.

• Yahoo Shopping: homepage and section story feeds, full article content, and retailer store deals.
• Yahoo Tech, Yahoo Health, Yahoo Autos, and Yahoo Entertainment: homepage and section editorial story feeds plus full article content.
• Yahoo Life: homepage story feed and full article content.
• All six platforms generate matching methods across the six SDKs and are covered by the open-source MCP tool catalog.

Added Yahoo News and Yahoo Sports, covering homepage and section story streams, full article content, and live scoreboards, standings, rosters, and game boxscores across NFL, NBA, MLB, NHL, golf, MMA, motorsports, tennis, and the Olympics.

• Yahoo News: homepage and section story streams, full article content, autocomplete suggestions, and article comments.
• Yahoo Sports: scoreboards, standings, teams, rosters, players, and game boxscores across major US leagues.
• Yahoo Sports also covers golf tournament leaderboards, MMA fight cards, motorsports races, tennis rankings, and Olympic medal counts.
• Both platforms generate matching methods across the six SDKs and are covered by the open-source MCP tool catalog.

Added the Kohl's platform, covering category and campaign page product grids with filter facets.

• Pass a category or curated campaign slug for that page's normalized product grid.
• Returns facets for discovering further category values.
• Generates matching methods across the SDKs and is covered by the open-source MCP tool catalog.

Added the Wayfair platform, covering category browsing and per-product detail with pricing and ratings.

• Category browsing returns a paginated, normalized product grid with name, brand, pricing, and image.
• Product detail includes stock status, an aggregate rating with a star breakdown, and Wayfair's own site-selected feature highlights.
• Generates matching methods across the SDKs and is covered by the open-source MCP tool catalog.

Added five apparel, footwear, and marketplace retailers, covering keyword search, category browsing, and per-color/per-size product detail.

• Nike: keyword product search and the full Men/Women/Kids/Jordan category taxonomy, with per-color-variant pricing and sizing.
• Old Navy: covers the Gap Inc. storefront family — Old Navy, Gap, Banana Republic, and Athleta — selectable per brand, with category browsing, product detail, and a store locator.
• Zara: keyword search, full category listings, and per-color product detail with real marketing copy and image galleries.
• Wish: keyword search and per-product detail with pricing, aggregate rating, and every purchasable size/color/merchant variation.
• Zappos: keyword search and per-product detail with pricing, an aggregate rating breakdown, and featured customer reviews.
• All five generate matching methods across the SDKs and are covered by the open-source MCP tool catalog.

Added Agoda and Trip.com, covering hotel, activity, home, and flight search and detail for travel research and price monitoring.

• Agoda: hotel search and detail, activity search and detail, home search, and flight search including itinerary amenities and search locations.
• Trip.com: hotel search and detail across international storefronts.
• Both platforms generate matching methods in the Go, TypeScript, Python, Ruby, Java, and PHP SDKs and are covered by the open-source MCP tool catalog.

Added six resale and live-shopping marketplaces, covering listing search, seller and closet profiles, and sneaker and streetwear pricing.

• Depop, Mercari, Poshmark, and Vinted: listing search, listing detail, and seller or closet profiles.
• Whatnot: live-shopping category browsing and live show search.
• StockX: sneaker and streetwear product search, detail, and release calendars.
• All six platforms generate matching methods across the Go, TypeScript, Python, Ruby, Java, and PHP SDKs and are covered by the open-source MCP tool catalog.

Added CarMax, Cars.com, Costco, and Instacart, covering vehicle search and dealer inventory, warehouse locations, product search and reviews, and nearby grocery stores.

• CarMax: vehicle search, vehicle detail, store locations, and shop-by-brand browsing.
• Cars.com: vehicle search and detail across dealer listings.
• Costco: warehouse locations, product search, product detail, availability, and reviews.
• Instacart: nearby store discovery, product detail and pricing, and keyword product search.
• All four platforms generate matching methods across the six SDKs and are covered by the open-source MCP tool catalog.

Added Fiverr and Upwork, covering gig and job search plus seller and freelancer profiles.

• Fiverr: gig search, gig detail, and seller profiles.
• Upwork: job search, job detail, and freelancer search.
• Both platforms generate matching methods across the six SDKs and are covered by the open-source MCP tool catalog.

Added Bluesky, Pinterest, DuckDuckGo, and Yahoo Search, covering social profiles and posts plus multi-engine web, image, and news search.

• Bluesky: profile lookup, author feeds, post threads, and follower and following lists.
• Pinterest: pin, board, and user search and detail.
• DuckDuckGo and Yahoo Search: web, image, news, and video search.
• All four platforms generate matching methods across the six SDKs and are covered by the open-source MCP tool catalog.

Added Congress and Strava, covering bills, votes, and member stock-disclosure filings plus routes, clubs, and challenges.

• Congress: bill and report lookup and member stock-disclosure filings.
• Strava: route detail, club information, and challenges.
• Both platforms generate matching methods across the six SDKs and are covered by the open-source MCP tool catalog.

Added five new platforms — Etsy, Autotrader, ImportYeti, Indeed, and Zalando — covering marketplace search, listing detail, and shop, dealer, and company profiles, all with hosted MCP tools.

• Etsy: listing search, listing detail and reviews, and shop profiles, search, and reviews.
• Autotrader: vehicle listing search, vehicle detail, and dealer profiles with current inventory.
• ImportYeti: company and supplier search plus US customs shipment-volume reports.
• Indeed: job search, job detail, and location autocomplete.
• Zalando: product search, category browsing, product detail, supported country storefronts, and search autocomplete.

Added the OpenTable platform, covering location-based restaurant search, live reservation timeslots, reviews, and menus.

• Search by free-text term near a latitude/longitude, or look up a known restaurant id.
• Live bookable timeslots for a given date, time, and party size, plus diner reviews with per-category ratings and full menus.
• Generates matching methods across the SDKs and is covered by the open-source MCP tool catalog.

Added Booking.com and Expedia, covering hotel search and detail with reviews, flight search, and things-to-do/activity search.

• Booking.com: hotel search and detail with rooms, photos, and guest reviews, flight search, and attractions search.
• Expedia: destination and property search with pricing and reviews, flight search, and Things To Do activity search.
• Both platforms generate matching methods across the SDKs and are covered by the open-source MCP tool catalog.

Added the DoorDash platform, covering discovery feeds, location and dish search, full store detail with menus, reviews, and fulfillment options.

• Discovery feeds and location search with autocomplete, plus keyword dish search.
• Store detail returns menus with prices, customer ratings and reviews, and delivery or pickup fulfillment schedules.
• Generates matching methods across the SDKs and is covered by the open-source MCP tool catalog.

Added the Ticketmaster platform, covering event search, category and city discovery, and event, attraction, and venue detail.

• Search by keyword and location, or list every supported discovery category and browse Ticketmaster city pages.
• Page through category, city, attraction, venue, or search results for deeper coverage.
• Resolve event, attraction, and venue identifiers into structured detail records and related event listings.
• Generates matching methods across the SDKs and is covered by the open-source MCP tool catalog.

Added five single-company careers-platform scrapers plus a Facebook Pages, Groups, and Marketplace endpoint, cross-linked to the Jobs dataset.

• Amazon Jobs, Apple Jobs, Google Careers, Meta Jobs, and Tesla Jobs: search each company's own careers site by keyword, category, team, or location, and fetch a single posting by id.
• Each careers endpoint is cross-linked to the bulk Jobs dataset for hiring-trend research.
• Facebook: look up a public Page by id, vanity name, or URL for name, followers, category, hours, and public contact details; look up a public Group; or search Marketplace listings by location, category, price, and condition.
• All six generate matching methods across the SDKs and are covered by the open-source MCP tool catalog.

Added the optional `include_metrics=true` mode to Reddit post and comments endpoints, with updated API docs, playground controls, and generated client support.

• The default feed-based response remains 1 credit; metrics mode costs 3 credits and fetches the anonymous Reddit page directly in a single request.
• Post metrics include net score, upvote ratio, comment count, award count, and approximate upvote and downvote estimates.
• Comment results include the public net score and award count available for each returned comment, alongside the post-level metrics.
• Exact downvote totals and Reddit's private view, share, and repost counters are reported as unavailable rather than inferred.

Added the Apple Books platform, covering search, catalog detail, paginated reviews, similar titles, author bibliographies, series listings, and charts.

• Search across ebooks and audiobooks, or look up a title, author, or series by id.
• Catalog detail includes title, author, price, rating, and description; reviews return reviewer name, rating, and text.
• Chart endpoints return current ranking movement for catalog and review-intelligence workflows.
• Generates matching methods across the SDKs and is covered by the open-source MCP tool catalog.

Added the UberEats platform, covering location and keyword search, full store detail with menus, and a reviews snapshot.

• Search by delivery coordinates with an optional keyword for restaurant name, rating, cuisine tags, and delivery estimate.
• Store detail returns the full menu with item prices plus a reviews snapshot.
• Generates matching methods across the SDKs and is covered by the open-source MCP tool catalog.

Added iCIMS and Eightfold AI board listings and single-job endpoints to the Jobs platform, plus both providers to the hiring-signals aggregate.

• iCIMS board and single-job endpoints for tenants' public Career Site postings.
• Eightfold AI board and single-job endpoints, covering both of its frontend generations.
• Both providers included in the normalized job shape and the hiring-signals aggregate.
• Available as hosted MCP tools for agent workflows, alongside the REST endpoints.

Added Capterra and Numbeo, covering software search and reviews plus city and country index data.

• Capterra: search by category or product name, resolve a product profile, then page through public reviews with ratings, pros, and cons.
• Numbeo: city or country detail and current or historical rankings across cost of living, quality of life, crime, health care, pollution, traffic, and property investment.
• Both platforms generate matching methods across the SDKs and are covered by the open-source MCP tool catalog.

Added seven media-catalog platforms, covering search, detail, ratings, reviews, and rankings across anime, manga, music, books, film, and critic-review data.

• Anime and Manga: AniList-backed search and title detail with scores, popularity, genres, and MyAnimeList cross-reference ids, plus rankings, characters, staff, and airing schedules.
• TMDB and Metacritic: movie, TV, and (for Metacritic) game detail with release dates, Metascores, and critic and user reviews.
• Goodreads and Letterboxd: book ratings, reviews, and author bibliographies; film ratings, rating histograms, reviews, and similar-film recommendations.
• Discogs: release, master, artist, and label search and full discographies from the public music database.
• All seven generate matching methods across the SDKs and are covered by the open-source MCP tool catalog.

Added the Walmart platform, covering item detail and keyword search across the Walmart catalog.

• Item detail includes price, availability, brand, images, rating, seller, and specifications.
• Keyword search returns normalized results across the Walmart catalog.
• Generates matching methods across the SDKs and is covered by the open-source MCP tool catalog.

Added the PlayStation Store platform, covering product and concept detail, store search, category browsing, deals shelves, and latest releases.

• Product and concept (game hub) detail with pricing, editions, and release dates.
• Store search, category and all-games browse grids, deals shelves, and latest-release listings.
• Generates matching methods across the SDKs and is covered by the open-source MCP tool catalog.

Added the Jobs platform (11 endpoints): per-provider board listings and single postings across Greenhouse, Lever, Ashby, Workday, and SmartRecruiters, a cross-provider company search, and a hiring-signals aggregate.

• One normalized job shape across five ATS providers — title, company, department, location, remote, compensation, and description — so postings are comparable regardless of the underlying careers platform.
• Hiring signals: total open roles with department, location, and top-title breakdowns, remote share, and how many roles were posted in the last 7 and 30 days — a leading indicator of company growth.
• Cross-provider company search finds which ATS a company uses from its careers slug.
• Available as hosted MCP tools for agent workflows, alongside the REST endpoints.

Added the Steam platform, covering store detail and pricing, review scores and histograms, live and peak player counts, achievements, news, and SteamSpy ownership estimates.

• Store detail and pricing, review scores with rating histograms, and live concurrent player counts for any app id.
• Achievements, news, featured listings, and SteamSpy ownership estimates.
• Store search by keyword for game-market dashboards and research pipelines.
• Generates matching methods across the SDKs and is covered by the open-source MCP tool catalog.

Added the SEC EDGAR platform (10 endpoints): company search, filings, single-filing detail and section extraction, full-text search, XBRL frames, normalized financial statements with computed ratios, insider transactions, 13F holdings, and a company-intelligence overview.

• Normalized income statement, balance sheet, and cash flow for any US filer under one stable schema across EDGAR inconsistent XBRL tags, with computed margins, revenue growth, current ratio, debt-to-equity, and free cash flow.
• Extract 10-K, 10-Q, and 8-K item sections such as Risk Factors and MD&A as clean text, plus full-text search across filings.
• Insider transactions (Forms 3, 4, and 5), 13F institutional holdings, company filings, and a single company-intelligence overview.
• Available as hosted MCP tools for agent workflows, alongside the REST endpoints.

Added four new platforms — Chrome Web Store, SofaScore, PitchBook, and TrustMRR — plus a new tech-stack dataset, all returning normalized JSON with hosted MCP tools.

• Chrome Web Store: extension details, search, similar extensions, reviews, permissions, privacy practices, developer profiles, categories, collections, and charts.
• SofaScore: live scores, events, statistics, lineups, odds, standings, teams, players, and head-to-head data across sports.
• PitchBook: company, investor, and fund profiles for private-market research.
• TrustMRR: verified startup revenue leaderboards, startup profiles, categories, and marketplace listings.
• Tech-stack dataset: query the technologies a website runs on across a large web census.

Added the ESPN platform, covering scoreboards, standings, teams, rosters, athletes, game summaries, and league news.

• Every endpoint takes a sport and league, such as NFL, NBA, MLB, or NHL.
• Scoreboard returns games with scores, schedule, status, and odds when available.
• Game summary, team detail, and roster endpoints open a single event or team; standings and poll rankings add season context.
• Generates matching methods across the SDKs and is covered by the open-source MCP tool catalog.

Added two new platforms — GitHub (repos, orgs, users, search, and trending) and X (public posts and profiles) — returning structured JSON with hosted MCP tools.

• GitHub: search repositories and users, track trending repos and developers, and fetch detail for users, organizations, and repositories including contributors, stargazers, forks, languages, releases, events, and pinned items.
• X: fetch a public post by ID, a profile by username, and a profile's first page of posts.
• Both platforms are available as hosted MCP tools for agent workflows, alongside the existing REST endpoints and SDKs.

Added stored-dataset query surfaces for TikTok creators, GitHub users, and app intelligence (apps, reviews, and charts), each returning structured records without live scraping.

• TikTok creators dataset: search creators by handle, niche, country, verification, and follower count, sorted by reach, engagement, or relevance.
• GitHub users dataset: search, item lookup, facet aggregations, and nearby location queries over stored public GitHub user profiles.
• App intelligence datasets: query stored App Store and Google Play apps, reviews, and ranking charts.

Enhanced the Reddit, Airbnb, and Yahoo Finance APIs with opt-in Reddit vote scores, Airbnb nightly pricing and host signals, and Yahoo Finance full-history support.

• Reddit: opt-in vote scores (with_scores) across search, subreddit posts, subreddit about and comments, multi-subreddit, domain, user posts and comments, and trends.
• Airbnb: derived nightly price per night on search results, host id and superhost status on room detail, and day-level availability from the calendar endpoint.
• Yahoo Finance: history with period set to max now returns full daily history back to a ticker's IPO.

Launched a suite of nine no-signup tools at /tools, each backed by the live Crawlora API with a one-click path from a free sample to a ready-to-run API call.

• Free Web Scraper turns any URL into clean Markdown, with a "Works with" wall of 180+ verified sites and clear blocked-vs-unreachable errors.
• "Can I Scrape This Site?" anti-bot checker reports the protection a domain uses and how hard it is to collect, framed around authorized public data.
• Free SERP Checker runs multi-engine searches with shareable, cached result permalinks.
• Email & Contact Finder, Keyword Research, YouTube Transcript Extractor, App Review Analyzer, TikTok Ad & Trend Finder, and Google Trends Dashboard round out the set.
• Every result shows a "Get this via the API" snippet prefilled with your inputs, plus a soft rate-limit upgrade prompt, so the jump from sample to integration is one copy-paste.
• Results are shareable via per-result permalinks with their own social preview cards.

Added result permalinks with per-result Open Graph cards across the free tools, plus a share bar on blog posts.

• Every free-tool run can be shared as a replayable permalink that re-renders the same result.
• Per-result Open Graph cards are generated server-side, so shared links unfurl with a result-specific preview image.
• SERP-checker results are cached and shareable, with a visit counter on shared links.
• Blog posts gained a footer share bar with an "Explore with AI" option.

Added the Redfin platform, covering region search, property detail, Redfin Estimate, and market trends.

• Search a region with filters, look up a single property, or pull the Redfin Estimate and comparables.
• Property detail includes price, beds, baths, square footage, status, and price history.
• Regional market-trend data supports monitoring price cuts, new listings, and sold prices over time.
• Generates matching methods across the SDKs and is covered by the open-source MCP tool catalog.

Added six new platforms — Polymarket, Kalshi, Metaculus, IMDb, Rotten Tomatoes, and Box Office Mojo — plus an expanded Reddit API, all returning normalized JSON.

• Polymarket: events, markets, outcome prices, order books, midpoints, spreads, price history, tags, sports, leaderboards, and trade activity.
• Kalshi: events, markets, series, recent trades, order books, exchange schedule, and historical market data.
• Metaculus: forecasting questions, community forecasts and history, metadata, options, and comment feeds.
• IMDb: title and name detail, credits, awards, episodes, reviews, keywords, trivia, and technical specs.
• Rotten Tomatoes scores and reviews for movies, series, seasons, and episodes; Box Office Mojo lifetime grosses, yearly and weekend charts, releases, franchises, and genres.
• Expanded the Reddit API with subreddit about and comments, multi-subreddit and domain posts, user posts and comments, and trends.

Regenerated all six official SDKs from the public API contract to cover the new platforms — now 491 public operations — and published v1.7.0-sdk.1 to every language registry.

• Added grouped helpers for Polymarket, Kalshi, Metaculus, IMDb, Rotten Tomatoes, and Box Office Mojo, plus the expanded Reddit API.
• Published v1.7.0-sdk.1 to npm, PyPI, RubyGems, Maven Central, Packagist, and the Go module proxy.
• The public contract grew to 491 operations, and the hosted MCP server now exposes 439 tools.

Published official beta SDKs for Java, PHP, and Ruby across Maven Central, Packagist, and RubyGems, and documented all six SDKs on the SDK and examples pages.

• Added an official Java SDK to Maven Central as net.crawlora:crawlora-sdk — a dependency-free client built on java.net.http for Java 17+.
• Added an official PHP SDK to Packagist as crawlora/sdk for PHP 8.1+, installable with composer require crawlora/sdk:^1.5@dev.
• Added an official Ruby SDK to RubyGems as the crawlora gem for Ruby 3.0+, installable with gem install crawlora --pre.
• All three share grouped endpoint helpers, API-key and JWT auth, automatic retries with backoff, numeric and cursor pagination, middleware hooks, and client-side rate limiting.
• Promoted every official SDK to the v1.5.0-sdk.3 tag and refreshed the SDK hub, guide pages, and developer examples to cover all six languages.

Shipped a new Brand Data API and surfaced it across the public catalog, platform discovery, docs, Playground, and a use case.

• Added the GET /brand/retrieve endpoint, which fetches a domain's homepage and returns a normalized brand profile as structured JSON.
• Extracted fields include title, description, slogan, brand colors, logos and icons, social profiles, footer links, and schema.org organization data such as address and stock ticker.
• Added a Brand Data API platform landing page with docs, Playground, and related discovery links.
• Supported query options for domain input, a faster maxSpeed mode, a cache-freshness window, and an upstream fetch timeout.
• Wired the Brand Data API into the company data enrichment use case alongside LinkedIn, SimilarWeb, and Product Hunt.

Expanded the hosted MCP server to expose the public endpoint catalog as callable tools, with API-key authentication and enforced REST/MCP parity.

• Exposed the public endpoint catalog as hosted MCP tools so agents can search, research, and enrich without bespoke integrations.
• Added API-key authentication using the x-api-key header for MCP tool calls.
• Enforced REST/MCP coverage so the available tools track the public REST endpoints.
• Covered families include search and maps, Google and Yahoo finance and trends, YouTube, TikTok and TikTok Top Ads, reviews, marketplaces, travel, geocoding, podcasts, datasets, Reddit, and the new Brand Data API.
• Exposed usage endpoints over MCP so agents can check consumption while they work.

Published the Reddit endpoint family across the public catalog, platform discovery, docs, Playground, and a use case, with reliability improvements to the Reddit collection path.

• Added Reddit search, post detail, comments, and subreddit posts endpoints that return normalized public post and comment fields.
• Added a Reddit platform landing page with docs, Playground, and related discovery links.
• Added a Reddit social listening use case for brand mentions, sentiment, and community trend research.
• Improved reliability of the Reddit collection path for more consistent public results.
• Kept Reddit copy scoped to public posts and comments with responsible-use guidance.

Rolled out a sitewide visual refresh and expanded the public content footprint across blog, comparison, and discovery surfaces.

• Introduced a new emerald design system with a refined palette and self-hosted typography across marketing, console, Playground, and docs pages.
• Refactored and curated the site footer for clearer navigation and fewer columns with "see all" links.
• Added cornerstone blog posts and topic archive pages for organic discovery.
• Expanded comparison coverage and SERP-focused landing pages for high-intent queries.
• Added social preview (Open Graph and Twitter) images sitewide and tightened sitemap hygiene.
• Added a new contact page and refreshed the brand icon set with an installable web app manifest.

Improved front-end performance and added automated quality and Core Web Vitals monitoring for the public site.

• Reduced page weight with self-hosted fonts, bundle budgets, and catalog caching.
• Split the Playground into lazy, on-demand tabs so initial loads stay light.
• Added Core Web Vitals reporting through site analytics for real-user performance.
• Fixed accessibility issues including focus rings and label-in-name compliance.
• Added an automated Lighthouse quality guardrail to the build pipeline.

Hardened the API request path, expanded observability, and shipped reliability-focused SDK upgrades.

• Added a Redis-backed circuit breaker with fail-open rate limiting so the API stays available during dependency hiccups.
• Added availability cache and hedge metrics plus optional distributed tracing for clearer operational insight.
• Added a dependency security watch and upgraded the Go runtime for the API.
• Added SDK middleware, idempotency, client-side rate limiting, and connection pooling.
• Included pagination metadata in SDK responses and added bearer authorization support for MCP access.

Published the Shopify endpoint family across the frontend catalog, platform discovery, pricing examples, use-case pages, Playground, sitemap, and changelog.

• Added Shopify store, products, product detail, recommendations, collections, collection products, pages, page detail, search suggestions, child sitemaps, and sitemap URL endpoints to public discovery.
• Added a Shopify platform landing page with docs, Playground, pricing, related commerce links, and official-API caveats.
• Added a Shopify storefront intelligence use-case page for catalog monitoring, collection research, page discovery, predictive search analysis, and sitemap audits.
• Synced frontend endpoint credit examples with backend Shopify billing weights.
• Kept Shopify copy scoped to public storefront data and directed private merchant, order, checkout, inventory, and admin workflows to official Shopify APIs.

Published the Shop.app endpoint family across the frontend catalog, platform discovery, pricing examples, use-case pages, Playground, sitemap, and changelog.

• Added Shop.app search, product, product reviews, product variants, related products, product-to-shop, shop profile, shop products, collection products, shop locations, shop reviews, shop typeahead, categories, suggestions, and analysis endpoints to public discovery.
• Added a Shop.app platform landing page with docs, Playground, pricing, related marketplace links, and official-API caveats.
• Added e-commerce product intelligence use-case coverage for Shop.app, Amazon, and eBay workflows.
• Synced frontend endpoint credit examples with backend Shop.app billing weights.
• Included Shop.app docs, Playground, platform, and use-case routes in sitemap-backed discovery.

Published machine-readable discovery surfaces for agent workflows and improved markdown negotiation for clients that prefer structured text responses.

• Added public agent card, skill index, and skill markdown routes for capability discovery.
• Published MCP server card and API catalog linkset metadata for client-side integration flows.
• Added authorization server and protected-resource metadata with canonical public origins.
• Improved markdown response negotiation so agent clients can request readable page content.
• Published Content Signals and DNS-AID records to make site policy and identity metadata easier to discover.

Expanded the YouTube transcript showcase library and improved the generated metadata, page performance, and browse experience for long-form transcript examples.

• Added more real transcript showcase pages across AI, business, documentaries, interviews, and research topics.
• Published accurate video publish dates in showcase page metadata and visible page details.
• Made showcase pages static so generated examples stay fast and consistent for visitors.
• Reduced runtime memory needs for the growing showcase library.
• Added pagination to the YouTube showcase index so the growing library stays easy to scan.
• Included paginated showcase URLs in the public sitemap so crawlers can discover deeper pages.

Refined SDK documentation, public OpenAPI export behavior, and example guidance for teams integrating Crawlora APIs.

• Added a public OpenAPI export path that excludes deprecated surfaces from SDK generation.
• Documented SDK maintenance and release checks so generated clients stay easier to verify.
• Updated public SDK links and examples across developer documentation.
• Kept private account, billing, and deprecated endpoint details out of public SDK discovery.
• Improved developer-facing guidance for typed models, enum handling, and example smoke checks.

Hardened YouTube video metadata extraction and related availability checks used by transcript and video research workflows.

• Improved fallback behavior for YouTube video metadata when upstream responses vary.
• Added retry coverage for automated checks that confirm video availability.
• Aligned metadata reliability work with transcript showcase publish-date verification.
• Preserved clear error behavior for unsupported or unavailable upstream video pages.
• Kept reliability documentation current for future YouTube endpoint maintenance.

Published public discovery paths for stored Google Maps business datasets backed by structured search and validation behavior.

• Added a first-class Datasets platform surface for stored Google Maps business records.
• Published list, search, nearby, facets, and item endpoints in Docs and Playground.
• Aligned dataset read pricing and calculator examples with backend billing weights.
• Validated filtering, pagination, geo radius, and result-window behavior for stored dataset reads.
• Clarified that stored dataset search returns indexed records without triggering live scraping.

Synced the public frontend catalog and marketing paths with newly active CoinGecko endpoints from backend Swagger.

• Added CoinGecko as a first-class Crypto & Digital Assets platform surface.
• Published markets, coin profile, chart analysis, trending, search, category, chain, exchange, NFT, unlock schedule, treasury, news, and Learn article workflows.
• Aligned pricing examples and calculator scenarios with backend billing weights for the new endpoint family.
• Updated homepage, product, feature, platform, footer, getting-started, and Playground surfaces to reflect the active public catalog.
• Kept crypto copy focused on research workflows, not investment advice or real-time trading.

Synced the frontend catalog and marketing paths with newly active Geocoding, JustWatch, and TikTok Top Ads endpoints from backend Swagger.

• Added platform discovery for Geocoding and JustWatch with direct links into Docs and Playground.
• Expanded TikTok public positioning to include Top Ads list, detail, analysis, filter, location, recommendation, safety, spotlight, and suggestion endpoints.
• Aligned pricing examples, calculator scenarios, and credit labels with backend billing weights for the new endpoint families.
• Updated homepage, product, feature, platform, footer, getting-started, and recipe surfaces to reflect the active public catalog.
• Kept internal and deprecated endpoint groups out of public marketing promotion.

Synced the public FE catalog with newly active Yahoo Finance endpoints and added finance-focused discovery paths across platform pages, Docs, Playground, pricing, and marketing surfaces.

• Added the Yahoo Finance platform page with direct paths into endpoint docs and Playground examples.
• Published quote, search, historical price, market summary, calendar, screener, financial, options, and ticker module endpoints from the generated catalog.
• Aligned pricing examples and endpoint credit labels with backend billing weights for the new Yahoo Finance family.
• Updated homepage, product, feature, platform, footer, and getting-started surfaces to include Yahoo Finance alongside Google Finance.
• Kept private and absent endpoint families out of public discovery surfaces.

Expanded the public catalog and marketing paths for newly active travel, property, search discovery, podcast, finance, business, marketplace, app-store, and social data APIs.

• Added public platform pages for Airbnb, TripAdvisor, and Zillow with direct paths into endpoint docs and Playground examples.
• Expanded Bing and Brave coverage for web, image, news, video, and suggestion workflows.
• Added Google Jobs, Google Suggest, and Google Trends discovery coverage alongside Google Search, Maps, and Finance surfaces.
• Aligned pricing examples and calculator data with the active backend billing catalog for the new public endpoints.
• Kept deprecated Etsy routes and non-public account APIs out of public platform, footer, and conversion surfaces.

Expanded and refined endpoint coverage across travel lodging, destination research, trend discovery, and property workflows so more common research jobs can use normalized API responses.

• Improved Airbnb review, calendar, and search coverage for travel research flows.
• Restored TripAdvisor discovery for destination, venue, place, and review workflows.
• Added Google Trends discovery for explore, trending, related query, regional interest, category, location, and enum endpoints.
• Expanded Zillow responses with richer typed property details.
• Kept newly available endpoint groups aligned with public Docs, Playground, and availability checks.

Hardened target-specific extraction paths and tightened endpoint lifecycle handling so unavailable or private surfaces fail more predictably.

• Improved browser-backed fallbacks for harder target pages that need rendered HTML before extraction.
• Preserved typed availability errors so clients can distinguish blocked, unavailable, and unsupported cases.
• Cleaned up deprecated and private endpoint surfaces so public catalogs stay focused on supported APIs.

Polished public discovery paths for Docs and Playground by canonicalizing endpoint routes and keeping internal-only groups out of public catalogs.

• Canonicalized endpoint-specific Playground URLs for cleaner public routing.
• Documented the catalog cleanup workflow used when an endpoint should stop appearing publicly.
• Removed deprecated and internal API groups from public discovery surfaces while preserving supported customer flows.

Improved the runtime paths behind browser-assisted APIs so target probes, rendered fetches, and long-running checks complete more consistently.

• Hardened timeout behavior for longer diagnostics and protected target checks.
• Improved browser capacity placement for workloads that need rendered extraction.
• Added local verification helpers and readiness checks that make future runtime changes easier to validate.

Added the Yelp platform, covering business search with pagination.

• Search by a free-text term and a location — neighborhood, city, state, or zip code.
• Normalized results include rating, review count, price, categories, address, coordinates, and photos.
• Paginate with limit and offset; an empty results array is a valid no-match response.

Added the customer referral flow, reward tracking, and review controls needed to run referral campaigns from signup through approval.

• Referral links now carry through signup so customers can invite other teams.
• Console and admin surfaces support reviewing referral activity and reward state.
• Billing and usage views have the referral context needed for account-level follow-up.

Improved the execution path for public pages that need browser rendering or extra challenge handling before structured data can be extracted.

• Expanded browser-backed execution coverage for harder upstream pages.
• Improved fallback behavior so API responses fail more transparently when an upstream page cannot be used.
• Documented the reliability lessons that matter for future endpoint work.

Expanded audio and podcast coverage with more Spotify surfaces and synchronized the public catalog so Docs, Playground, and pricing stay aligned.

• Added more Spotify profile, playlist, podcast, chart, and recommendation coverage.
• Synced the endpoint catalog into public Docs, Playground, pricing, and platform pages.
• Updated monitoring coverage for newly documented API surfaces after rollout.

Polished public API discovery pages so endpoint families, platform groups, and filters are easier to scan on desktop and mobile.

• Improved Docs filtering layout and platform group grids.
• Tightened homepage and product page sections around real endpoint examples.
• Added regression coverage for public page layout and catalog synchronization.

Added Apple Podcasts and Spotify Podcasts, covering charts, show and episode detail, recommendations, and search.

• Apple Podcasts: chart rankings, related-show recommendations, curated New shelves, show metadata, episodes, and search.
• Spotify Podcasts: home and category browsing, per-genre charts, show and episode detail, related-show recommendations, and search.
• Both platforms return normalized JSON for podcast discovery, chart tracking, and catalog enrichment.

Expanded infrastructure health coverage, recovery guidance, and cleanup automation for service-critical platform components.

• Added alerts for resource pressure that can affect public API availability.
• Improved recovery guidance for service health issues after infrastructure churn.
• Added cleanup automation for long-running observability data so health checks stay responsive.

Consolidated marketing navigation and feature pages around the API capabilities customers evaluate before integration.

• Grouped infrastructure features into a clearer product feature hub.
• Made the homepage endpoint example visible by default.
• Connected feature pages to Docs, pricing, Playground, and supported platforms.

Improved public site quality checks and moved mutable site settings into a safer runtime configuration path.

• Added stronger preview quality checks for public pages.
• Separated runtime settings from the built site where possible.
• Normalized public host references across site pages and examples.

Improved how customers can check Crawlora service status from public pages and subscribe to operational updates.

• Added a public status signal to the site footer.
• Improved the public status page feed experience.
• Kept status checks lightweight for faster public page loads.

Added public build metadata, preview restart shortcuts, and hosted preview support for faster site verification.

• Added a public freshness signal for the running site build.
• Improved preview restart ergonomics for public-page review.
• Allowed hosted preview checks to work cleanly during site development.

Refined public Playground routing and indexing behavior so clean endpoint pages are discoverable while private app routes stay out of search results.

• Improved clean public Playground routes for endpoint-specific testing.
• Kept authenticated app routes separate from public discovery.
• Added coverage for public indexing rules and endpoint route generation.

Expanded the public website into a more complete evaluation path for buyers and developers.

• Added feature, comparison, use-case, pricing, and developer guide pages.
• Aligned the web scraping API overview with real docs and samples.
• Improved public metadata and route coverage for organic discovery.

Added and exercised browser backend options used by harder endpoint families that need a rendered page before extraction.

• Expanded browser backend choices for difficult public pages.
• Improved service routing for rendered extraction paths.
• Kept API availability checks aligned with the new browser-backed paths.

Improved sign-in options, account profile presentation, and layout resilience across public platform pages.

• Added social sign-in support and account profile avatars.
• Hardened platform landing pages against layout overflow.
• Improved form behavior around API key fields and browser autofill.

Improved billing actions, active key visibility, usage period defaults, and pricing-to-console handoff.

• Connected pricing plan actions to billing workflows.
• Improved active API key selection and display in Playground and console pages.
• Defaulted usage views to the current billing period for more relevant reporting.

Aligned public domains, frontend routing, API host references, and MCP access points around the Crawlora brand.

• Moved public examples and site links onto Crawlora hostnames.
• Exposed MCP access from the public product surface.
• Linked pricing and billing flows more directly from the public site.

Strengthened session restore, auth-state loading, Playground parameter feedback, and API key workflows.

• Preserved sign-in status across browser refreshes and restarts.
• Clarified request parameter errors and sample response handling in Playground.
• Added focused loading states for overview, API key, and usage sections.

Reworked the homepage, platform pages, footer links, and docs metadata around the live public catalog.

• Made homepage sections catalog-driven with product endpoint examples.
• Added platform landing pages and footer navigation for supported public surfaces.
• Improved API contract metadata for docs, Playground, and agent-facing surfaces.

Added the Trustpilot platform, covering business-unit search, company profiles, and paginated customer reviews.

• Search Trustpilot business units by name or domain, or browse a category to build a list of companies to track.
• Company profiles return TrustScore, star rating, review count, and category rankings.
• Paginated customer reviews include rating, title, text, and date for reputation tracking and competitor analysis.

Added reliability checks, clearer failure classifications, and endpoint deprecation practices for older surfaces.

• Added scraper reliability auditing and guardrails.
• Clarified behavior for deprecated endpoint families.
• Documented lessons for browser-backed and stealth-sensitive endpoint work.

Improved documentation consistency, deprecated endpoint UX, and admin repair paths used during catalog maintenance.

• Improved docs cross-links and catalog consistency.
• Made retired endpoint states easier to understand.
• Added clearer repair paths for maintaining endpoint metadata.

Improved schema views, Playground forms, billing previews, and admin dashboards for day-to-day operations.

• Enhanced Playground discovery, forms, and schema inspection.
• Improved billing preview and admin data flows.
• Added clearer operational filters for console workflows.

Launched the first Crawlora web experience with public marketing pages, dashboard foundations, and console workflows.

• Added the first public site and console experience.
• Introduced dashboard pages for account and API usage workflows.
• Hardened sensitive console flows before broader public use.

Added the public status foundation and expanded route monitoring around the public API surface.

• Added a public status foundation for customer-facing service checks.
• Improved public API route monitoring and hostname checks.
• Expanded startup and availability budgets for smoother service updates.

Expanded service monitoring, public routing checks, and operational dashboards around the Crawlora API surface.

• Added application metrics and alert coverage for API health.
• Improved public route checks for customer-facing endpoints.
• Expanded dashboards for capacity, traffic, and service health.

Expanded professional network and web intelligence endpoints for company research and competitive analysis.

• Added broader LinkedIn company, product, and profile coverage.
• Expanded web intelligence search and detail surfaces.
• Updated public docs around the new business research coverage.

Strengthened the authenticated request path and broadened test coverage around important API contracts.

• Added static API key support for production-style requests.
• Broadened core endpoint test coverage.
• Added focused parsing checks for Google Maps search behavior.

Improved source quality reporting, validation throughput, and traffic accounting for the extraction network.

• Added source quality reporting for network inputs.
• Improved validation scheduling and throughput measurement.
• Persisted traffic and source refresh counters for clearer operational review.

Added mobile app store endpoint families with docs, examples, and integration coverage.

• Added App Store scraper APIs and documentation.
• Added Google Play scraper APIs, examples, and handler coverage.
• Expanded service and integration tests for mobile app data workflows.

Refreshed search endpoint behavior and browser impersonation support for search result extraction.

• Adjusted Google Search extraction for upstream changes.
• Improved browser impersonation support for harder pages.
• Kept search result parsing aligned with current page behavior.

Added Instagram profile, post, and reel coverage with documentation for social data use cases.

• Added Instagram profile extraction.
• Added post and reel coverage.
• Documented social endpoint behavior for developers.

Expanded YouTube transcript and community data alongside Product Hunt search and product discovery endpoints.

• Added YouTube caption, comment, and tag coverage.
• Added Product Hunt product and user search coverage.
• Improved docs for product discovery workflows.

Added Crunchbase organization and person coverage plus YouTube subtitle extraction for research workflows.

• Added company and person research endpoints.
• Added YouTube subtitle extraction.
• Expanded worker-backed enrichment for business data.

Expanded local business enrichment with business type, contact, status, and place data improvements.

• Added business type and place enrichment.
• Expanded website status and contact enrichment workflows.
• Improved Google Maps place and search behavior for local business research.

Expanded TikTok search, creator, hashtag, trend, and video data for social intelligence workflows.

• Added TikTok search user and trending documentation.
• Added creator and hashtag trend coverage.
• Expanded popular trend video and hashtag detail APIs.

Added Google Maps search coverage, documentation, and usage monitoring improvements.

• Added Google Maps search and place extraction.
• Updated documentation for local business workflows.
• Improved API usage monitoring around authenticated requests.

Added API usage monitoring and account identity improvements that later supported console billing and usage pages.

• Added API usage monitoring foundations.
• Added account email and username fields.
• Improved notification handling for account events.

Expanded endpoint families across Google Finance, Amazon, TikTok, Similarweb, and general web parsing.

• Added finance search, ticker, quote, income statement, balance sheet, cash flow, and crypto coverage.
• Added Amazon suggestion, search, and product improvements.
• Added TikTok post, comment, category, explore, and trending coverage.

Added LinkedIn endpoint coverage for people, schools, products, and jobs.

• Added LinkedIn people and school data extraction.
• Added LinkedIn product page coverage.
• Added LinkedIn job data coverage for recruiting and market research workflows.

Added Google Search, News, Image, and Shopping extraction with API documentation.

• Added Google Search result extraction.
• Added Google News and Image Search coverage.
• Added Google Shopping search and detail coverage.

Added sign-in, sign-up, rate limiting, metrics, and the first structured API documentation updates.

• Added account sign-in and sign-up foundations.
• Added rate limiting for fair API usage.
• Added metrics and documentation updates for early API consumers.

Started the platform foundation that later powered endpoint coverage, availability checks, and structured API delivery.

• Established early network validation and peer health workflows.
• Added the first operational building blocks for high-volume extraction.
• Created the base that later supported public API endpoints and monitoring.