# Methodology

This bundle keeps two evaluation layers separate.

## System readiness

The five-area diagnostic evaluates Foundations, Bindings, Components, Structure, and
Governance against the exact rubric in
`framework/design-system-diagnostic-1.0.0.json`.
Each criterion is scored against explicit anchors from 0 to 4 using immutable collected
evidence. Repeated assessor passes are reconciled through human review. Area and
overall values are normalized to 100. Unknown or inapplicable evidence remains
explicit in the results.

The canonical reviewed records are `reports/<system>/scorecard.json`. The standalone
HTML scorecards expose the rationale, selected and next anchors, scope limitations,
and source citations for every criterion.

## Controlled agent performance

When included, the second layer executes one locked representative task for each of
the five areas. Each task is repeated and checked against a reviewed oracle. Where
applicable, it is materialized in a pinned runtime for type, interaction, accessibility, and
responsive checks. The report records completion, successful outcomes, invalid API or
token use, provenance accuracy, human interventions, duration, model usage and cost,
and reproducibility.

Agent performance and system readiness retain separate scores and evidence trails.
Task results cover the recorded actor profile, task definitions, evidence snapshot,
runtime, and run count.

## Implementation roadmap

When included, `roadmaps/<system>.json` is locked to the reviewed scorecard. Its
score-ceiling ledger includes every criterion below 4. Projected score changes are
conditional: a future diagnostic must verify that each initiative's acceptance
criteria are met with current retained evidence. For public-evidence studies, the
roadmap begins with evidence reconciliation so private capabilities are not mistaken
for missing implementation.

## Included systems

- `carbon-core-v11`: Carbon Core

## Verification trail

- `bundle.json` records checksums for every distributed file.
- `config/evaluation.shared.json` records the sanitized study scope.
- `evidence-manifests/` records source provenance and snapshot checksums in a
  manifest-only bundle; `evidence/` contains cited source files only when the bundle
  was deliberately generated with full evidence.
- `agent-reports/<system>/agent-performance.json` is the canonical agent result.
- `agent-reports/<system>/failure-evidence.json`, when present, is a sanitized runtime
  trace supporting reviewed public failure interpretations.
- `roadmaps/<system>.json`, when present, is the reviewed implementation roadmap and
  complete score-ceiling ledger.