How to Scrape Metaculus in 2026 (API & Python)

Metaculus already runs its own official API, and most of what you'd want to read — questions, forecast aggregates, comments, categories — is available without an API key. So the honest answer for "how to scrape Metaculus" in 2026 is largely the same as for Kalshi: use the platform's own API for reads, and don't scrape the rendered site on top of it. The difference that matters here is what Metaculus actually is. It's a forecasting community, not a trading market — forecasters submit probability estimates on real-world questions and earn reputation points for accuracy, but no money is wagered or paid out. Think of it as Kalshi's non-monetary cousin: same "crowd-implied probability" concept, no bid/ask, no settlement in dollars. This guide covers Metaculus's own API, its Terms of Use, and when a structured API is worth adding on top.

Why scrape Metaculus data?

• Crowd-forecast research — Metaculus's community prediction is a reputation-weighted aggregate across thousands of resolved questions, useful as a calibration benchmark for other forecasting methods.
• Comparing forecasts to polls and expert predictions — set Metaculus's crowd probability beside political polling, expert surveys, or Kalshi/Polymarket's market-implied odds on the same event.
• Forecasting-accuracy and calibration research — Metaculus tracks resolved questions with known outcomes, which is exactly the kind of dataset calibration and superforecasting research needs.
• AI/LLM forecasting benchmarks — Metaculus runs public AI forecasting tournaments, and question/forecast history data is a natural benchmark set for evaluating an LLM's probability estimates against a human crowd.
• Journalism and research use — cite a live, sourced probability on an open question instead of a static poll number.

Is it legal to scrape Metaculus?

Option 1: Metaculus's own API (and its limits)

Metaculus's own /api2/ API is the first thing to reach for, and it's more open than most platforms in this series:

curl "https://www.metaculus.com/api2/questions/?limit=10"

No API key required for most reads. What it covers, and where it still has limits:

• Reads are largely keyless, writes need a token. Listing and retrieving questions, comments, categories, and projects works unauthenticated. Submitting a prediction, posting a comment, or hitting user-specific endpoints needs an API token — Metaculus issues these on request via api-requests@metaculus.com, or via cookie/session auth if you're building on top of a logged-in account.
• No published self-serve rate-limit figure. Metaculus doesn't publish a fixed requests-per-minute number the way some APIs do; heavy unauthenticated polling can still get throttled or blocked, so back off on 429s and keep pull frequency reasonable.
• The schema is a working research API, not a stable product API. /api2/ grew out of Metaculus's own Django backend rather than a versioned public contract, so field names and pagination shapes can shift between releases — expect to re-check responses periodically rather than treating the schema as frozen.
• Terms restrict downstream use, not the reads themselves. The AI/ML-training restriction above is the one to plan around if you're building a benchmark or dataset for model training rather than one-off research.

Option 2: DIY in Python (and why it's mostly unnecessary)

Because Metaculus's own JSON API already serves clean, structured reads, there's little reason to parse the rendered metaculus.com question pages — and doing so would sidestep the exact API carve-out the Terms of Use are built around. A minimal DIY pull against the official API looks like this:

import requests

r = requests.get(
    "https://www.metaculus.com/api2/questions/",
    params={"limit": 10, "status": "open"},
).json()
questions = r["results"]

This is fine for a single script against one platform. The tradeoffs show up once Metaculus is one of several forecasting or market-data sources you track:

• Metaculus's schema is Metaculus's own — field names, question types (binary, numeric, multiple-choice), and pagination are specific to /api2/, a separate integration to maintain alongside anything else you pull.
• No unified auth or normalization across sources. If you're also pulling Kalshi, Polymarket, or traditional market data, each has its own client, schema, and throttling behavior to reconcile.
• You still own polling, storage, and change-handling — Metaculus doesn't dedupe, timestamp, or retry your pulls, and a schema shift on their side is now your bug to catch.

Option 3: A structured Metaculus API

The value-add of a Metaculus scraping API isn't unlocking data Metaculus hides — most of it is already open — it's one normalized schema and one x-api-key across Metaculus and every other platform in the catalog, so forecasting data sits next to market data without a second client to maintain. List questions:

curl "https://api.crawlora.net/api/v1/metaculus/questions?limit=10" \
  -H "x-api-key: $CRAWLORA_API_KEY"

Questions, forecasts, and history in Python:

import requests
h = {"x-api-key": "YOUR_API_KEY"}
base = "https://api.crawlora.net/api/v1/metaculus"

questions = requests.get(f"{base}/questions", headers=h, params={"limit": 10}).json()["data"]
question = requests.get(f"{base}/question/43612", headers=h).json()["data"]
forecasts = requests.get(f"{base}/question/43612/forecasts", headers=h).json()["data"]

A question-forecasts response is normalized JSON you can store directly (real fields):

{
  "code": 200,
  "msg": "OK",
  "data": {
    "question": {
      "id": 43612,
      "title": "Tampa sulphur price in June 2026?",
      "public_page_derived": true
    },
    "methods_count": 1,
    "methods": [
      {
        "method": "recency_weighted",
        "forecaster_count": 54,
        "center": 0.52,
        "lower": 0.37,
        "upper": 0.66,
        "history_points": 25
      }
    ]
  }
}

Forecast history, options for multiple-choice questions, and question metadata behind the same key:

history = requests.get(f"{base}/question/43612/forecast-history", headers=h,
                        params={"max_points": 500}).json()["data"]
options = requests.get(f"{base}/question/43731/options", headers=h).json()["data"]
metadata = requests.get(f"{base}/question/43731/metadata", headers=h).json()["data"]

Feed-style pulls for browsing without a specific question ID — by category, by tournament, or the site's comment feeds:

category_qs = requests.get(f"{base}/category/artificial-intelligence/questions",
                            headers=h, params={"limit": 10}).json()["data"]
tournament_qs = requests.get(f"{base}/tournament/ai-benchmarking/questions",
                              headers=h, params={"limit": 10}).json()["data"]
top_comments = requests.get(f"{base}/top-comments", headers=h, params={"limit": 10}).json()["data"]

Every response carries source_url and fetched_at, so store the pull timestamp with each row alongside the forecast values — a community forecast is a moving average, and yesterday's number isn't today's.

What you can collect

• Questions — title, status (open/closed/resolved), question type (binary, numeric, multiple-choice), and forecaster/comment counts, listed or by ID.
• Forecast aggregates — the community's recency-weighted probability, with confidence bounds and forecaster count, per question.
• Forecast history — time-series points showing how the aggregate probability moved, with configurable point density.
• Options and metadata — per-option probabilities for multiple-choice and group questions, plus question metadata like grouping variables.
• Category, project, and tournament feeds — questions filtered by topic area, Metaculus project, or forecasting tournament.
• Comments — the site's comments feed and weekly top comments by question.

Limitations

• Terms of Use, not API access, is the real gate. Reads are largely open, but scraping the rendered site outside the API, and training AI/ML models on Metaculus content, both need attention before building anything downstream — read the terms directly.
• Crowd probabilities, not certainties. A Metaculus community forecast is a reputation-weighted estimate, not a guarantee — useful as a calibration signal, not a prediction of fact.
• No money changes hands. Unlike Kalshi or Polymarket, there's no bid/ask, no volume, and no financial incentive backing a Metaculus forecast — accuracy is reputational, which changes how you should weight the signal.
• Schema is a working API, not a versioned product. Field shapes on Metaculus's own /api2/ can shift between site releases; treat it as a research API, not a stable contract.
• Not investment advice. Store fetched_at with every pull and treat values as forecasting-research inputs, never as trading or financial signals — there's no position to take on Metaculus in the first place.

Where this gets used

• Forecasting and calibration dashboards — track how a community's implied probability on a question moves toward resolution.
• Crowd-vs-market comparison — set Metaculus's forecast beside Kalshi or Polymarket's market-implied odds on comparable real-world events.
• AI forecasting benchmarks — evaluate an LLM's probability estimates against Metaculus's resolved-question history.
• Research and journalism — cite a sourced, reputation-weighted probability instead of a single expert's guess.

Sources

Start collecting

Try it first, free: run any public URL through the Free Web Scraper, or check whether a site blocks bots with the Anti-Bot Checker — no signup.

Test the questions endpoint in the Playground, check the schema in the API docs, and review pricing. Metaculus is the non-monetary read on crowd-implied probability; for the trading-market side of the same idea, how to scrape Kalshi covers a regulated exchange with real yes/no prices, and how to scrape CoinGecko covers crypto markets — both useful to set beside a Metaculus forecast on the same event. See the broader prediction market data use case for how these sources fit together. For the legal basics of any of this, see is web scraping legal.

Part of our how-to-scrape guide series — every platform we cover, in one index.