How the AI Risk Index is computed

Task-based, not occupation-based. Occupations are bundles of tasks with very different AI exposure. Scoring whole occupations with a single judgment — the approach of the famous 2013 "47% of jobs" estimate — has a poor empirical track record. We score each of the ~18,800 O*NET task statements individually and derive occupation scores from them, weighted by how important each task is to the occupation.

Uncertainty is part of the score. Large language models disagree with each other substantially when rating AI exposure — replications show the share of "highly exposed" occupations can vary by an order of magnitude depending on which frontier model does the rating. A single-model index is therefore an artifact. Every score we publish carries a confidence interval derived from disagreement across an independent multi-model panel.

Fully reproducible. Weights live in one versioned source file, prompts are versioned, every rating stores the model, prompt version, raw response and parsed score, and every published number traces to a stored, immutable computation run.

Adaptation, not doom. The evidence to date shows concentrated, cohort-specific effects — especially on entry-level hiring in automation-exposed occupations — rather than economy-wide job loss, while measured AI usage remains majority-augmentative. The index is designed to guide adaptation decisions, not to produce headlines.

Every occupation gets three sub-scores on a 0–100 scale, never collapsed into one:

• Exposure — AI is technically capable of performing the occupation's tasks. Computed from the capability dimensions only (automatability and feasibility).
• Substitution — the headline composite: AI actually replaces the human, once cost, adoption barriers and sector adoption velocity are accounted for. Exposure without adoption predicts nothing; this score is exposure discounted by the real world.
• Augmentation — AI assists the human on the task, raising productivity without replacing them. Rated separately per task; it is not part of the substitution composite. High augmentation alongside moderate substitution reads as a changing job, not a disappearing one.

Raw source dumps are immutable and never edited in place; all transformations run through a pipeline that records manifests (hashes and row counts) for every derived artifact.

Each task statement is rated on six dimensions (the five composite dimensions plus augmentation) on an integer 1–5 scale by at least two independent frontier language models using an identical, versioned rubric prompt. Ratings are requested with schema-constrained JSON output; each rating must include a written rationale grounded in the task statement.

For every task × dimension, the panel produces a rating band: low = the panel minimum, mid = the panel mean, high = the panel maximum. Bands — not single numbers — flow into all downstream computation, which is how rater disagreement becomes a visible confidence interval instead of hidden noise.

Audit trail. Every rating stores the model identifier, prompt version, raw API response, parsed score and rationale. On any occupation page, expanding a task shows every model's rating and rationale — nothing is asserted that cannot be inspected.

Human validation. A sample of ratings is reviewed by humans; disagreements above one point on the 5-point scale route the task to an expert review queue whose overrides replace the model band and are flagged in the data. Prompt changes bump the prompt version and invalidate cached ratings.

Weights are defined once, in a versioned source file, and served live at /api/v1/methodology — any change is a new index version with a changelog entry and regenerated sensitivity analyses.

A rating r ∈ [1, 5] normalizes to substitution pressure p ∈ [0, 1]:

p = (r − 1) / 4            direct dimensions
p = 1 − (r − 1) / 4        inverted dimensions (barriers)

substitution_task = 100 · Σ_d  w_d · p_d          over all five dimensions
exposure_task     = 100 · (w_auto·p_auto + w_feas·p_feas) / (w_auto + w_feas)
augmentation_task = 100 · p_augmentation

The exposure sub-score uses only automatability and feasibility, renormalized. The scoring engine is a pure, deterministic library with no network, clock or randomness — identical inputs always produce identical scores, which is what makes runs auditable. Its behavior is pinned by property-based tests and by a published worked example that the test suite reproduces to three decimals.

Occupation scores are the importance-weighted mean of task scores, using O*NET task importance ratings normalized within each occupation. Tasks without an importance rating receive the occupation's mean importance. The same aggregation applies to the low, mid and high bounds.

We additionally report the highly exposed task share: the fraction of an occupation's tasks with a substitution score of 70 or more. "X% of tasks in this occupation are highly exposed" is the preferred headline reading — more honest than a single composite.

The low and high bounds are worst/best-case envelopes over the rater panel: the lower bound takes each dimension's pressure-minimizing rating bound (for the inverted barriers dimension, that is the high rating), the upper bound symmetrically. By construction low ≤ score ≤ high always holds. A wide interval means the models disagreed — treat that score as an open question, not a verdict. Occupations with the widest intervals are exactly where human expert review is prioritized.

The index version (1.0.0-draft.1) follows semantic versioning: formula or weight changes are major; data-source refreshes are minor; recomputations with refreshed adoption inputs are patches. Every computation writes a run record with the index version, prompt version and rater models. Runs are immutable — recomputations create new runs and old runs remain queryable, so any historically published score can be reproduced and checked forever. Every API response carries the index version it was computed under.

Published alongside each major or minor release:

• Convergent validity — rank correlation against independent published measures (Felten AIOE, "GPTs are GPTs" task scores, ILO exposure gradients).
• Rater stability — the distribution of cross-model agreement; the widest-band occupations are flagged in the product.
• Outcome tracking — correlation against realized labor-market indicators (entry-level employment in exposed occupations, measured usage data). Exposure indices individually explain little of realized unemployment risk, so we report outcome tracking honestly rather than claiming prediction.
• Weight sensitivity — rank stability of the composite under ±25% perturbation of each weight.

• LLM raters co-evolve with the technology they measure; multi-model panels bound this "ruler problem" but do not eliminate it.
• Sector-level adoption evidence enters through rubric anchors rather than per-sector statistical joins in the current version.
• Scores describe tasks as currently constituted; occupations reorganize, and task mixes shift precisely when exposure is high.
• Wages shown are U.S. national medians; cross-country generalization awaits the ESCO/ROME release.
• This index measures pressure, not destiny. It cannot tell an individual whether their job is safe; it can tell them which of their tasks are changing and how confident the evidence is.

Cite as: AI Risk Index, version 1.0.0-draft.1, airiskindex.io, Simon-Pierre Boucher (2026). Scores and derived data are published under CC BY 4.0 — reuse freely with attribution and the index version. Occupation and task data incorporate the O*NET database (U.S. Department of Labor, CC BY 4.0); wage data from BLS OEWS.

Methodology questions, corrections, expert-panel participation, data licensing or anything else: contact@spboucher.ai — Simon-Pierre Boucher.