Methodology
Overview
PrognostiBench tracks forward-looking, testable claims about AI's effects on the economy and society. We extract these claims from public articles, blog posts, and commentary, then catalog them with structured metadata to enable tracking over time.
What Counts as a Claim
We focus on claims that meet the following criteria:
- AI-specific
- AI is the causal mechanism, not a tangential mention. If you can remove "AI" from the sentence and the claim still holds, it is not AI-specific.
- Forward-looking
- The claim makes a prediction about the future, not a description of the present or a report of existing research results.
- Time-bounded
- The claim has an explicit or implied temporal anchor -- a date, deadline, relative range, or concrete trigger event. Open-ended predictions with no time horizon are excluded.
- Testable
- The claim is specific enough to be confirmed or refuted with empirical evidence.
We exclude definitions, tautologies, citations of existing research, purely philosophical assertions, and open-ended predictions with no time horizon.
How We Extract Claims
Articles are analyzed using Claude Sonnet 4.6 with a structured extraction prompt. The model identifies the author's own forward-looking predictions (not evidence they cite), tags them along multiple dimensions, and extracts verbatim quotes, timeframes, confidence levels, and the reasoning behind each classification.
Author biographies are generated separately using Claude Sonnet 4.6 with web search to provide context about who is making each prediction.
Claim Properties
Each claim is tagged along these dimensions:
- Economy
- Whether the claim concerns AI's effects on the economy (employment, GDP, productivity, wages, etc.).
- Society
- Whether the claim concerns AI's effects on society (education, governance, culture, inequality, etc.), independent of economic effects.
- Prediction
- Whether the claim is a prediction about future events or outcomes.
- Testable
- Whether the claim is specific and potentially confirmable or refutable with empirical evidence.
- Causal
- Whether the claim asserts a causal relationship (e.g., "AI will cause X").
- Comparative
- Whether the claim involves a comparison (e.g., "AI will increase X" or "more than Y").
Valence
Each claim is assigned a valence from the perspective of human welfare and broad societal or economic outcomes:
- Optimistic
- The claim predicts improvement for human welfare (e.g., job creation, productivity gains, better healthcare).
- Pessimistic
- The claim predicts deterioration (e.g., job displacement, inequality, loss of autonomy).
- Neutral
- The claim predicts change without a clear directional impact on welfare (e.g., a shift in how work is done without net gain or loss).
Each valence assignment includes an explanation of the reasoning, so readers can understand and evaluate the classification.
Timeframes
Where available, we extract the predicted timeframe for each claim and note whether it was explicitly stated or implied from context. Timeframes are standardized into one of these categories:
- Imminent
- Less than 1 year out.
- Within 1-5 years
- Medium-term horizon.
- Beyond 5 years
- Long-term predictions.
- Specific year(s)
- When a concrete date or range is mentioned (e.g., "by 2030" or "2028-2035").
This standardization enables consistent filtering, comparison across claims, and timeline visualization.
Confidence Levels
We capture stated confidence levels, whether expressed qualitatively ("highly likely", "almost certainly") or as numeric probabilities ("90% chance"). These reflect the author's own expressed certainty, not our assessment of accuracy.
Editorial Review
All extracted claims are manually reviewed before publication. Automated extraction can produce errors, and we check each claim for accuracy, relevance, and proper attribution before adding it to the public tracker. Author biographies are also reviewed and approved before being displayed publicly.
Outcome Tracking
As time passes and evidence becomes available, claims are updated with outcome assessments: confirmed, refuted, or inconclusive. Each outcome includes supporting evidence and a summary of the findings.