Jared HawkinsDesign-system diagnostic

design-system-diagnostic/1.0.0

Carbon Core

Version v11.114.0 / @carbon/react 1.114.0. Evidence cutoff 2026-08-22. Public-evidence overall score 72.5.

Overall readiness72.5 / 100
Publication statusApproved
Human review100%
Assessor agreement75%
Warnings unresolved0

Evidence sample: 11 deep components across 10 archetypes; 1 flow(s); 2 pattern(s); intentional design dispositions data-table=compound, ui-shell=compound; governance scope public.

Public-evidence scope: Missing public evidence remains unknown.

How to read the score: Scores show how closely the captured evidence meets each rubric anchor. A criterion score of 3 maps to 75 and means the configured anchor is comprehensive, operational, or tested as defined by that criterion.

Diagnostic area

Foundations findings

Token taxonomy and semanticsPasses: 3, 3, 3; approve3 / 4

Passes: 3, 3, 3

The DTCG theme files (g10.json, g100.json) implement hundreds of role-based semantic tokens (background, layer, field, border, text, link, icon, support, focus, skeleton, ai, chat, syntax, toggle, overlay, shadow) each carrying an alias reference to a raw color primitive (e.g. background -> {gray.10}) and an inline $description of its role, repeated consistently across four theme files representing four modes. This is corroborated by the Figma file's confirmed full-file variable coverage: 1205 variables, 1907 alias references, and a dedicated Theme collection of 548 variables across the same four modes (White/Gray10/Gray90/Gray100), resolving the packet's stated known_limitation that full-file coverage needed confirmation via the Variables endpoint or a checked-in export. The independent color-guidance documentation states an operational Theme/Token/Role/Value model and gives systematic, non-arbitrary derivation rules for interaction-state tokens (hover = half-step, active = two full steps, selected = one full step), showing the alias/semantic layer is not just named but rule-governed. This satisfies anchor 3 (comprehensive, operational semantic layers and aliases). It does not reach anchor 4 because no evidence shows an automated process that validates or reconciles the token graph across consumers (for example, a CI check diffing Figma-exported variables against the DTCG/npm package output) or measures token adoption/usage over time.

Completeness: completeVerification: implementedTest definition observed: noAutomation wiring observed: noCurrent execution result observed: noGeneralizability: system-wide

Selected anchor: Semantic layers and aliases are comprehensive and operational

Next anchor: The token graph is validated, measured, and maintained across consumers
Evidence needed: No evidence of an automated mechanism that validates or reconciles the token graph across consumers (for example, CI parity checks between the Figma export and the published npm packages) or that measures token adoption/maintenance across consuming teams, which anchor 4 requires.

Scope: mixed; system-wide. Full-file variable coverage is confirmed for the current Figma library snapshot and the shipped DTCG theme sources reviewed (g10/g100; g90/white were referenced but not fully re-verified line-by-line due to excerpt truncation), and no automated cross-consumer validation pipeline is evidenced.

What broke
No confirmed break in the semantic/alias layer itself; the unmet piece is validation, not structure
Impact
Consumers can trust the documented semantic taxonomy and alias structure for current design and code artifacts, but cannot assume the token graph is actively guarded against future design/code drift since no automated reconciliation is evidenced.
Why
The DTCG theme files and Figma variable export both show a comprehensive, alias-based token graph with documented operational rules, but the packet contains no CI/build artifact or governance record showing the graph is continuously checked across the code and design channels.
What to change
Publish or surface evidence of an automated check (for example, a CI job or release gate) that reconciles the Figma variable export against the published DTCG/npm token output, and track/report on token consumption across dependent packages.
Evidence citations
  • 5143be78329c1b1e4237/files/packages/themes/src/dtcg/g10.json, original source: lines 217-235
  • 5143be78329c1b1e4237/files/packages/colors/src/colors.ts, original source: lines 246-267
  • 31e36527e62be3c4f265/figma-variable-summary.json, original source: /collections/0
  • c9535068f6088e09061e/content.txt, original source: lines 219-258
  • c9535068f6088e09061e/content.txt, original source: lines 342-357
Primitive completenessPasses: 3, 3, 3; approve3 / 4

Passes: 3, 3, 3

Reusable low-level primitives are documented and packaged separately from semantic tokens: @carbon/colors exposes a full raw color scale (10-100 grades plus hover variants) as JS/Sass constants; @carbon/layout exposes a spacing scale (spacing01-13), fluid spacing, container sizes, and icon sizes; @carbon/motion exposes DTCG duration and cubic-bezier easing primitives with usage descriptions. These primitives are not merely declared but are shown being consumed: @carbon/motion's surfaces.json composes named motion 'recipes' (disclosure, contextual, stretch, expand, invoke) entirely from alias references to the duration/easing primitives (e.g. {duration.moderate.01}, {easing.entrance.productive}), and the theme DTCG files alias raw color primitives (e.g. {gray.10}, {blue.60}) throughout. The Figma variable summary independently confirms parallel primitive collections (Spacing: 28 variables, Radius: 2 variables, Breakpoint: 215 variables across 5 modes), meaning the same primitive scales are available to both code and design consumers. This meets anchor 3 (comprehensive and consistently consumed). It falls short of anchor 4 because there is no evidence of measured adoption or governance over primitive usage (for example, linting that enforces primitive-only values, or reported consumption metrics), only the IBM Telemetry package-install notice, which does not measure primitive-level usage.

Completeness: completeVerification: implementedTest definition observed: noAutomation wiring observed: noCurrent execution result observed: noGeneralizability: system-wide

Selected anchor: Primitives are comprehensive and consistently consumed

Next anchor: Primitive coverage and adoption are measured and governed
Evidence needed: No evidence of measured or governed primitive adoption (for example, enforcement tooling, consumption telemetry, or a reported adoption rate across consuming teams/products), which anchor 4 requires.

Scope: mixed; system-wide. Confirms the primitive packages themselves and their alias-based consumption within the themes/motion packages; does not verify that every cataloged component (beyond the sampled deep components) consistently draws on these primitives rather than hard-coded values.

What broke
No confirmed break; the gap is adoption measurement, not primitive completeness
Impact
Component and product authors have a usable, comprehensive primitive set in both code and design tooling, but there is no visibility into how consistently the broader component catalog actually draws on these primitives versus hard-coded values.
Why
Evidence shows the primitives exist, are documented, and are consumed by at least the motion and color semantic layers via aliasing, but no telemetry, lint enforcement, or audit data on primitive usage across the full component catalog is present in the packet.
What to change
Add or surface enforcement (lint rules disallowing raw values in component styles) or usage telemetry so primitive adoption can be measured and governed across the catalog.
Evidence citations
  • 5143be78329c1b1e4237/files/packages/layout/src/tokens.ts, original source: lines 8-56
  • 5143be78329c1b1e4237/files/packages/motion/src/dtcg/motion.json, original source: lines 4-41
  • 5143be78329c1b1e4237/files/packages/motion/src/dtcg/surfaces.json, original source: lines 4-23
  • 31e36527e62be3c4f265/figma-variable-summary.json, original source: /collections/1
  • 864cd765d8e0c3ebb4ae/content.txt, original source: lines 63-77
Modes and adaptationPasses: 3, 3, 3; approve3 / 4

Passes: 3, 3, 3

Carbon documents and implements four theme modes (White, Gray 10, Gray 90, Gray 100) as complete DTCG token sets, each tagged with a color-scheme extension and producing distinct values for the same semantic token names; this is mirrored exactly in the Figma Theme variable collection (548 variables x 4 modes). Responsive/layout modes are likewise systematic: a Breakpoint collection (215 variables across 5 modes: SM/MD/LG/XL/Max) and a Grid mode collection (25 variables across Wide/Narrow/Condensed/Nested) exist alongside documented 2x Grid breakpoint and margin specifications. Accessibility-oriented modes are embedded directly in the token model: the color-guidance documentation describes systematic light/dark layering, explicit high-contrast 'inverse' tokens for deliberate high-contrast moments, a dedicated $focus token family, and WCAG contrast-ratio tables driving token value selection. Sampled component nodes (e.g. Button) show bound_variable_ids drawing directly from the Theme, Breakpoint, and other mode-bearing collections, confirming these modes are wired into at least the sampled components rather than existing only as unused design tokens. This satisfies anchor 3 (systematic across tokens and components, for the modes Carbon claims: theming, responsive, and accessibility). It does not reach anchor 4 because there is no evidence of continuous/automated validation of mode compatibility across supported consumers (for example, automated visual regression or variable-parity checks per mode).

Completeness: completeVerification: implementedTest definition observed: noAutomation wiring observed: noCurrent execution result observed: noGeneralizability: system-wide

Selected anchor: Modes are systematic across tokens and components

Next anchor: Mode compatibility is continuously validated across supported consumers
Evidence needed: No evidence of continuous/automated validation of mode compatibility across consumers (for example, automated cross-mode visual regression, or parity checks confirming every mode renders correctly in shipped code across releases), which anchor 4 requires.

Scope: mixed; system-wide. Mode infrastructure (variable collections, DTCG theme files) is confirmed full-file/system-wide; component-level wiring is directly confirmed only for the sampled deep components (e.g. Button), not verified across the entire catalog.

What broke
No confirmed break; the gap is continuous validation across consumers, not mode coverage
Impact
Designers and developers can rely on documented, token-driven theme/responsive/accessibility modes that are demonstrably wired into at least the sampled components, but there is no assurance that every mode continues to render correctly across all consumers after each release.
Why
Theme, breakpoint, and accessibility-related modes are represented consistently in both the DTCG source and the Figma variable export, and sampled components bind to these mode-bearing variables, but no automated per-mode regression or validation evidence is present in the packet.
What to change
Introduce and publish evidence of automated per-mode validation (for example, visual regression across all four themes and breakpoints in CI) to move from systematic support to continuously validated mode compatibility.
Evidence citations
  • 5143be78329c1b1e4237/files/packages/themes/src/dtcg/g10.json, original source: lines 1-8
  • 5143be78329c1b1e4237/files/packages/themes/src/dtcg/g100.json, original source: lines 1-8
  • 31e36527e62be3c4f265/figma-variable-summary.json, original source: /collections/0/modes
  • 31e36527e62be3c4f265/figma-variable-summary.json, original source: /collections/4
  • c9535068f6088e09061e/content.txt, original source: lines 181-217
  • 31e36527e62be3c4f265/figma-sample-details.json, original source: lines 5-44
Consumable delivery and versioningPasses: 3, 3, 3; revise3 / 4

Passes: 3, 3, 3

Carbon delivers installable foundation packagesNot availableincluding colors, layout, motion, and themesNot availablewith documented Sass and JavaScript consumption paths. Carbon’s Figma guidance separately documents its named v11 libraries and how designers enable them through Figma’s Libraries mechanism, while the collected full-file export confirms that the evaluated design library contains the corresponding variable foundation. Together, this establishes distinct delivery channels for code and design consumers. Score 4 is not supported because public evidence does not demonstrate an automated release-over-release check reconciling the Figma variables with published package output or measured migration impact when foundations change.

Completeness: completeVerification: implementedTest definition observed: noAutomation wiring observed: noCurrent execution result observed: noGeneralizability: system-wide

Selected anchor: Multiple consumers receive consistent versioned artifacts

Next anchor: Delivery contracts are validated and migration impact is measured
Evidence needed: Automated release-over-release validation between Figma variables and published package output, plus measured consumer migration impact for foundation changes, is needed to establish validated and measured delivery contracts.

Scope: mixed; system-wide. Confirms the existence and consistency of two versioned delivery channels at this snapshot; does not confirm automated reconciliation between them over time or measurement of migration impact on downstream consumers.

What broke
No confirmed break; the gap is contract validation and migration measurement, not the existence of versioned delivery
Impact
Both design and engineering consumers can reliably install and reference a versioned foundations artifact today, but there is no evidence that drift between the Figma library and the npm packages would be automatically caught, or that the impact of a foundations change on downstream consumers is tracked.
Why
READMEs confirm independent npm package versioning and Sass/JS consumption paths, and the Figma guidance/summary confirm a parallel versioned design-library channel with full-file coverage, but no CI/contract-test artifact or migration-impact report appears in the packet.
What to change
Add an automated check that reconciles the published Figma variable export against the released npm package token output on each version bump, and publish migration-impact data (for example, which consumers were affected) for foundations token changes.
Evidence citations

Diagnostic area

Bindings findings

Naming parityPasses: 2, 3, 2; approve2 / 4

Passes: 2, 3, 2

Code Connect files (packages/react/code-connect/**) provide a documented, per-component naming translation table between Figma property/variant names and React props for all 11 deep-component samples (e.g. Button 'Style'→kind, 'Size'→size, 'State: Disabled'→disabled; Grid's Figma asset is explicitly named 'Screen' while the code component is 'Grid'). This is systematic documentation, not tribal knowledge, ruling out anchor 1. However, material known gaps are explicit in the evidence itself: Tile's Code Connect mapping has most Figma properties (titleText, descriptionText, disabled, selected, slot, title, description) commented out with an inline comment citing an open, unresolved GitHub issue (#20089) for a 'Code Connect CLI validation bug'; Toggle's Figma 'State text' single property is mapped to two distinct code props (labelA/labelB); and 'Show label' is a negated boolean vs. code's hideLabel. These are documented but real naming-alignment gaps for at least one full sampled component, which prevents claiming systematic alignment across the whole sample.

Completeness: completeVerification: implementedTest definition observed: noAutomation wiring observed: noCurrent execution result observed: noGeneralizability: sample-only

Selected anchor: Common names are documented with known gaps

Next anchor: Names are systematically aligned across surfaces
Evidence needed: To reach 'systematically aligned across surfaces', the Tile sample's naming mapping would need to be fully restored (not commented out) and Data table's TableRow would need its remaining commented-out Figma properties (selectable, select type, selection, state, zebra style) mapped, so that no deep-component sample has an unresolved naming gap.

Scope: deep-component; sample-only. Evidence covers only the 11 declared deep-component samples via Code Connect; catalog-wide naming parity for the remaining ~330 composition candidates in the Figma library is not evidenced and cannot be inferred from this sample.

What broke
Tile's published Code Connect naming map has most Figma properties (title/description/disabled/selected/slot) disabled due to a known, unresolved tooling bug, and Data table's TableRow has multiple Figma properties (selectable, select type, selection, state, zebra style) left commented out.
Impact
Consumers using Figma Dev Mode for these two sampled components will not see accurate prop-name guidance for a meaningful subset of their design properties, increasing the chance of manual, ad hoc translation for exactly the components where it is most needed.
Why
packages/react/code-connect/Tile/Tile.figma.tsx and TableRow.figma.tsx show large blocks of commented-out figma.connect props, with Tile's comment explicitly citing carbon-design-system/carbon#20089 as the blocking cause.
What to change
Resolve the referenced Code Connect CLI validation bug (or work around it) to restore full Tile prop mapping, and complete the TableRow property mapping so naming parity is uniformly documented, not partially disabled, across the full deep-component sample.
Evidence citations
  • ef0fc94e689c12c30608/files/packages/react/code-connect/Button/Button.figma.tsx, original source: lines 17-38
  • ef0fc94e689c12c30608/files/packages/react/code-connect/Grid/Grid.figma.tsx, original source: lines 12-30
  • ef0fc94e689c12c30608/files/packages/react/code-connect/Tile/Tile.figma.tsx, original source: lines 20-44
  • ef0fc94e689c12c30608/files/packages/react/code-connect/Toggle/Toggle.figma.tsx, original source: lines 16-38
  • ef0fc94e689c12c30608/files/packages/react/code-connect/DataTable/TableRow.figma.tsx, original source: lines 18-45
Design-to-code contract correspondencePasses: 2, 2, 2; revise3 / 4

Passes: 2, 2, 2

Across the declared sample, consumer-controlled choices correspond through React APIs, CSS/runtime behavior, or composition. Tile types map to specialized Tile components and content maps through children; DataTable row selection and expansion map through specialized row subcomponents; Figma interaction states map to CSS or runtime state; Grid breakpoint is an authoring preview control; and modal callbacks are runtime-only. The unresolved Tile Code Connect validation defect reduces active traceability but does not remove the supported cross-surface contract. No evidence shows continuous machine validation and drift measurement across the complete contract.

Completeness: completeVerification: implementedTest definition observed: noAutomation wiring observed: noCurrent execution result observed: noGeneralizability: sample-only

Selected anchor: Supported choices, derived states, slots, and composition map coherently, and surface-specific controls are explicitly classified

Next anchor: Machine-readable mappings continuously validate shared contracts and explicitly classify surface-specific controls
Evidence needed: Additional evidence must satisfy the next anchor: Machine-readable mappings continuously validate shared contracts and explicitly classify surface-specific controls

Scope: deep-component; sample-only. Only the 11 declared deep-component samples were evidenced; contract correspondence for catalog-only components is unknown, and the Tile/TableRow gaps should not be generalized to the rest of the sample, which shows coherent mappings.

Binding mappings
  • Tile types and content: derived_mapping; design: Type variants plus Title text, Description text, Selected, State, and Swap slot; code: Tile, ClickableTile, SelectableTile, ExpandableTile, children, disabled, selected, and composition slots. The same choices are expressed through specialized React components and children composition; the source documents intended mappings even though the Code Connect CLI currently rejects several active declarations.
  • DataTable row selection and expansion: derived_mapping; design: Expandable, Selectable, and Select type variants; code: TableExpandRow, TableExpandedRow, and TableSelectRow composition. Separate Code Connect variants demonstrate the equivalent React composition for expandable, checkbox, and radio rows.
  • Grid breakpoint preview: design_only; design: Screen Breakpoint variant; code: CSS media queries and responsive Grid behavior without a consumer breakpoint prop. The Figma control simulates viewport width for authoring; runtime responsiveness is automatic and the Code Connect example does not pass it to Grid.
  • Modal open and close callback: runtime_only; design: No corresponding authoring property required; code: ComposedModal open and onClose runtime props. Visibility and callbacks are runtime state and event concerns, not persistent design-library choices.
  • Button style and size: shared_contract; design: Style and Size variants; code: Button kind, size, and isExpressive props. Code Connect explicitly maps each supported design variant to the corresponding React API.
What broke
Tile's Code Connect mapping has its core content props (titleText, descriptionText, slot, title, description) commented out and non-functional, citing an unresolved upstream Code Connect CLI validation bug (carbon-design-system/carbon#20089); only the swapSlot instance-swap prop remains live.
Impact
Consumers reading Tile's Figma-to-code mapping in Dev Mode will not see how Figma's title/description/content properties correspond to React Tile usage, forcing manual translation for this component.
Why
The Tile.figma.tsx source itself documents the gap: 'TODO: Re-enable these props when Code Connect validation bug is fixed... Properties exist in Figma but fail validation despite correct names.'
What to change
Track and resolve the linked Code Connect CLI validation bug, or ship an interim manual example demonstrating the titleText/descriptionText/slot mapping until the tooling bug is fixed; also clarify whether TableRow's commented base props are dead code or an active gap.
Evidence citations
  • ef0fc94e689c12c30608/files/packages/react/code-connect/Tile/Tile.figma.tsx, original source: lines 70-170
  • 8ecdf97a7dd31b0f1380/files/packages/react/src/components/Tile/Tile.tsx, original source: lines 155-175
  • ef0fc94e689c12c30608/files/packages/react/code-connect/DataTable/TableRow.figma.tsx, original source: lines 47-119
  • 8ecdf97a7dd31b0f1380/files/packages/react/src/components/DataTable/Table.tsx, original source: lines 53-55
Token parityPasses: 3, 3, 2; revise3 / 4

Passes: 3, 3, 2

The checked-in full-file Figma export makes variable names observable. Reviewer comparison found direct semantic matches for 101 of 104 standard core code-token paths (97.1%) and the design and code artifacts share four theme modes. Product/design-only Figma extensions and syntax/implementation-only code tokens are treated as scoped differences, not parity failures. Three standard paths remain unmatched (layer/background/03, shadow, support/caution/undefined), and no automated parity or drift check was evidenced, so score 4 is not supported.

Completeness: completeVerification: implementedTest definition observed: noAutomation wiring observed: noCurrent execution result observed: noGeneralizability: system-wide

Selected anchor: Token identity and modes align across design and code

Next anchor: Cross-surface token parity is automatically validated
Evidence needed: Additional evidence must satisfy the next anchor: Cross-surface token parity is automatically validated

Scope: foundation; system-wide. This assessment covers the color/theme token layer, where full-file Figma variable coverage was explicitly confirmed; motion and layout/spacing token parity (e.g. deprecated layout01-07 tokens in @carbon/layout) were not independently cross-checked against Figma variable bindings in this packet and are not included in this generalization.

What broke
No confirmed break in identity/mode alignment for the evidenced foundation token layer; the remaining gap is the absence of automated cross-surface validation.
Impact
Without automated parity checks, token drift between a future Figma Variables update and the DTCG token export would not be caught until manually noticed, risking silent divergence between design and code color/theme tokens over time.
Why
The evidence shows full-file variable resolution and matching DTCG token group structure/modes, but no CI job comparing the two surfaces was found among the reviewed GitHub workflows.
What to change
Add an automated CI check (e.g. via the Figma Variables REST API) that diffs published Figma Variable names/values against the generated DTCG token files on each release and fails on unexplained divergence.
Evidence citations
  • 31e36527e62be3c4f265/figma-export.json, original source: lines 18-4752
  • 5143be78329c1b1e4237/files/packages/themes/src/dtcg/g10.json, original source: lines 1-842
Traceability and deprecationPasses: 2, 3, 3; revise3 / 4

Passes: 2, 3, 3

Carbon maintains component-to-Figma references through per-component Code Connect files and republishes them through CI when changes reach main. Deprecated APIs are identified in implementation code, while the versioned v11 migration guide documents removed, renamed, and deprecated APIs; the versioning policy and public API snapshot testing further support code API traceability. This meets the versioned maintained-reference anchor. Score 4 is not supported because Code Connect is explicitly exploratory, some sampled mappings contain disabled or incomplete properties, and no automated design-to-code drift check was evidenced.

Completeness: completeVerification: automation-wiredTest definition observed: noAutomation wiring observed: yesCurrent execution result observed: noGeneralizability: system-wide

Selected anchor: Versioned mappings expose supported and deprecated APIs

Next anchor: Traceability and deprecation drift are automatically enforced
Evidence needed: Additional evidence must satisfy the next anchor: Traceability and deprecation drift are automatically enforced

Scope: mixed; system-wide. The versioning/migration documentation explicitly covers the entire @carbon/react public API, supporting a system-wide claim for deprecation tracking; however, the Code Connect design-linking portion of traceability was only verified for the 11-item deep-component sample and should not be assumed complete for catalog items outside that sample.

What broke
No confirmed break in the versioned-mapping mechanism itself, but Code Connect is self-described as being in an 'exploratory phase' and at least one sampled file (TableRow.figma.tsx) contains commented-out properties without an inline explanation comparable to the documented Tile issue.
Impact
Consumers of Code Connect for DataTable's TableRow may not realize several Figma properties (AI spacer, zebra style, state, selectable, select type, selection) are intentionally left unmapped, since no comment or issue link is provided there, reducing traceability confidence for that specific mapping.
Why
TableRow.figma.tsx comments out multiple properties without citing a reason, whereas Tile's equivalent gap is explicitly documented with a linked GitHub issue, showing inconsistent rigor in how gaps are traced.
What to change
Add inline rationale or issue links for all commented-out Code Connect properties (starting with TableRow) so every excluded or deferred property is traceable, consistent with the practice already used for Tile.
Evidence citations
  • ef0fc94e689c12c30608/files/.github/workflows/code-connect.yml, original source: lines 1-47
  • ef0fc94e689c12c30608/files/packages/react/code-connect/README.md, original source: lines 1-30
  • 3eb54126648a690950d8/files/docs/guides/versioning.md, original source: lines 27-72
  • 3eb54126648a690950d8/files/docs/migration/v11.md, original source: lines 683-775
  • 8ecdf97a7dd31b0f1380/files/packages/react/src/components/ComposedModal/ComposedModal.tsx, original source: lines 780-785

Diagnostic area

Components findings

Core component coveragePasses: 3, 3, 3; revise3 / 4

Passes: 3, 3, 3

Carbon's documented catalog spans roughly 40 components across action, input, selection, navigation, overlay, feedback, content, status, layout, and data-display archetypes, and all 11 declared representative deep components resolved in the Figma library. This supports comprehensive coverage for Carbon Core's stated foundational scope. Score 4 is not supported because no public systematic measurement of catalog gaps, adoption, or component health was evidenced; catalog presence is not treated as proof of state, responsive, accessibility, or binding quality.

Completeness: completeVerification: implementedTest definition observed: noAutomation wiring observed: noCurrent execution result observed: noGeneralizability: system-wide

Selected anchor: Coverage is comprehensive for the stated scope

Next anchor: Gaps, adoption, and component health are measured
Evidence needed: Additional evidence must satisfy the next anchor: Gaps, adoption, and component health are measured

Scope: catalog; system-wide. Catalog evidence establishes breadth and inventory only; it does not prove per-component state, accessibility, binding, or responsive quality. The preview-site catalog is release-authority supplemental material, not normative production documentation.

What broke
No confirmed break
Impact
Teams building common enterprise UI needs (forms, tables, navigation, feedback, overlays) have a reusable component available in both design and code for nearly every documented use case.
Why
Catalog and Figma inventory evidence corroborate each other across two independent public sources (preview catalog and Figma library) plus the code package.
What to change
Publish a public gap/adoption metric (for example, component maturity or usage dashboard) to support a higher anchor; current governance scope explicitly withholds this as private.
Evidence citations
Normal, recovery, and permission statesPasses: 2, 2, 3; revise3 / 4

Passes: 2, 2, 3

Across the declared sample, Carbon documents and implements form validation, server-side correction, disabled and duplicate-submission prevention, loading and progress, success, warning, read-only, empty and no-results recovery, and error-management states for permission, system, and configuration failures. The empty-state pattern explicitly requires explaining permission restrictions and providing steps to request access. This supports comprehensive state ownership and guidance across the declared components and compositions. Score 4 is not supported because public evidence does not show continuous, comprehensive measurement of the state matrix, and partial-success behavior is not established as a system-wide contract.

Completeness: completeVerification: documentedTest definition observed: noAutomation wiring observed: noCurrent execution result observed: noGeneralizability: sample-only

Selected anchor: Normal and recovery states have comprehensive ownership and guidance

Next anchor: State contracts are executable, tested, and measured
Evidence needed: Current system-wide state-coverage resultsNot availableincluding permission, partial-success, and recovery scenarios across the catalogNot availableplus tracked gaps or regressions are needed to establish measured state coverage.

Scope: deep-component; sample-only. State-matrix conclusions apply to the sampled deep components and cited compositions; not confirmed for the full ~40-component public catalog.

What broke
Permission-state guidance and evidence (for example, access-denied / insufficient-permission handling) are absent from all sampled deep components and compositions.
Impact
Teams building permission-gated flows have no documented or tested Carbon pattern to rely on, risking inconsistent ad hoc handling of denied-access states.
Why
None of the sampled deep-component Figma state dimensions, docs, or e2e specs reference a permission/authorization state; only loading, error, disabled, warning, and validation states are covered.
What to change
Publish explicit permission/access-denied state guidance with corresponding automated test coverage, or clarify that permission handling is intentionally delegated outside Carbon Core (for example, to Carbon for Products).
Evidence citations
Responsive behaviorPasses: 2, 2, 2; revise2 / 4

Passes: 2, 2, 2

Carbon defines a detailed breakpoint system, matching Figma variants, responsive Grid implementation that generates breakpoint-specific classes, and component guidance such as UI-shell collapse at smaller sizes. This meets documented behavior and constraints. Score 3 is not supported because runtime adaptation is evidenced strongly for Grid but not consistently across the broader declared component and composition sample, and collected tests do not exercise the small, medium, and large viewport matrix. The finding is evidence-bounded and does not claim Carbon is unresponsive.

Completeness: completeVerification: test-definedTest definition observed: yesAutomation wiring observed: noCurrent execution result observed: noGeneralizability: sample-only

Selected anchor: Basic behavior and constraints are documented

Next anchor: Components adapt consistently across supported conditions
Evidence needed: Additional evidence must satisfy the next anchor: Components adapt consistently across supported conditions

Scope: mixed; sample-only. Evidence demonstrates breakpoint/variant definitions and prose guidance for a subset of components; it does not cover container-query behavior, density modes, or content-driven responsive conditions system-wide, and does not confirm runtime adaptation for components outside this citation set.

What broke
No confirmed break, but no direct operational or test evidence of runtime responsive adaptation was found in the packet beyond breakpoint/variant definitions and prose guidance.
Impact
Consumers can rely on documented breakpoint constraints and layout guidance for planning, but cannot confirm from this evidence alone that Grid, Modal, UI shell, or Data table actually re-flow correctly at each breakpoint without separate manual or visual verification.
Why
The evidence packet supplies breakpoint/variant enumerations (Figma Screen, Grid code-connect) and normative prose describing intended responsive behavior, but no CSS/media-query source, no viewport-resize e2e test, and no visual-regression citation tied specifically to breakpoint behavior.
What to change
Surface (or cite) the responsive/media-query implementation in the Grid, Modal, and UI shell Header source, or add breakpoint-driven e2e/visual-regression checks, to move from documented constraints to demonstrated adaptive behavior.
Evidence citations
  • 864cd765d8e0c3ebb4ae/content.txt, original source: lines 152-176
  • 31e36527e62be3c4f265/figma-sample-details.json, original source: lines 445-473
  • 8ecdf97a7dd31b0f1380/files/packages/react/src/components/Grid/Column.tsx, original source: lines 348-399
  • ffb2854c79a762b42e9e/content.txt, original source: lines 239-246
  • c7035884c47c4a634eff/files/e2e/components/Grid/Grid-test.avt.e2e.js, original source: lines 12-22
Accessibility behaviorPasses: 3, 3, 3; revise3 / 4

Passes: 3, 3, 3

Carbon implements ARIA semantics, accessible names, keyboard behavior, focus routing and trapping, and Escape handling, and runs Playwright accessibility tests covering all 11 sampled component families. CI executes automated violation checks plus keyboard, focus-order, disabled-state, activation, and modal focus-trap assertions across four shards on pushes and pull requests, corroborated by published component testing statuses. This meets comprehensive and tested accessible behavior. Score 4 is not supported because screen-reader testing remains primarily manual, Form, Loading, and Modal disclose advanced-state or keyboard-testing gaps, and no continuously measured assistive-technology coverage or regression metric was evidenced.

Completeness: completeVerification: automation-wiredTest definition observed: yesAutomation wiring observed: yesCurrent execution result observed: noGeneralizability: sample-only

Selected anchor: Accessible behavior is comprehensive and tested

Next anchor: Assistive-technology coverage and regressions are continuously measured
Evidence needed: Additional evidence must satisfy the next anchor: Assistive-technology coverage and regressions are continuously measured

Scope: deep-component; sample-only. AVT CI evidence is confirmed only for components with e2e test files present in this packet (the 11 declared deep components); it does not confirm equivalent automated AT/keyboard testing exists or passes for catalog components outside the declared deep sample.

What broke
No confirmed critical accessibility failure; the residual gap is that AT/screen-reader regression checking is manual and a subset of the sample (Loading, Form, Modal) has incomplete advanced-state/keyboard test coverage per Carbon's own disclosure.
Impact
Teams can trust that automated axe-style violation checks and keyboard-focus behavior are continuously verified in CI for the sampled components, but cannot assume the same continuous assurance for screen-reader interaction or for Loading/Form/Modal advanced states, which rely on manual, point-in-time verification.
Why
Direct CI configuration and e2e test-code evidence show executed, gating automated checks; Carbon's own normative testing-status tables directly disclose which states and interaction modes remain manually verified or partially tested.
What to change
Automate AT/screen-reader regression checks where feasible (or document a recurring manual AVT3 cadence with public results), and close the disclosed advanced-state/keyboard-testing gaps for Loading and Form.
Evidence citations
  • c7035884c47c4a634eff/files/.github/workflows/ci.yml, original source: lines 240-352
  • c7035884c47c4a634eff/files/e2e/components/Button/Button-test.avt.e2e.js, original source: lines 124-151
  • c7035884c47c4a634eff/files/e2e/components/ComposedModal/ComposedModal-test.avt.e2e.js, original source: lines 58-83
  • 8ecdf97a7dd31b0f1380/files/packages/react/src/components/ComposedModal/ComposedModal.tsx, original source: lines 517-535
  • c7035884c47c4a634eff/files/docs/guides/accessibility.md, original source: lines 31-38
  • a82ca9adbd35c0913589/content.txt, original source: lines 70-76
  • 050f13ebce3920eb3d9a/content.txt, original source: lines 80-87

Diagnostic area

Structure findings

Layout primitivesPasses: 3, 3, 2; revise3 / 4

Passes: 3, 3, 2

Carbon provides an operational layout system covering mini units, spacing, columns, rows, margins, padding, gutters, fluid/fixed/hybrid grids, breakpoint conditions, panel types, and screen regions. Grid, Row, and Column implementation is present with breakpoint-specific span and offset contracts, a matching Figma Screen asset is captured, and component documentation applies the layout rules to Forms, Data tables, Tiles, and other sampled surfaces. This supports coverage of Carbon Core’s stated product scope. Score 4 is not supported because public evidence does not measure layout adoption or exceptions, and no automated layout-usage validation was found.

Completeness: completeVerification: implementedTest definition observed: noAutomation wiring observed: noCurrent execution result observed: noGeneralizability: system-wide

Selected anchor: Layout primitives cover the stated product scope

Next anchor: Layout use and exceptions are validated and measured
Evidence needed: Automated layout-conformance results, governed exception tracking, and adoption measurement across supported consumer surfaces are needed to establish validated and measured layout use.

Scope: mixed; system-wide. No Grid/Row/Column code artifact was included to verify implementation fidelity to the documented rules; no measurement of layout compliance or exceptions across the product is evidenced.

What broke
No confirmed break; however, no Grid/Row/Column implementation source was included in the evidence packet to verify the documented grid rules are enforced by the runtime components.
Impact
Teams can trust the documented grid/spacing/shell contract as comprehensive design guidance, but this packet cannot confirm the Grid/Row/Column code enforces those rules or that layout exceptions are caught before shipping.
Why
The 2x Grid foundation page and its cross-references from form, data-table, tile, and button docs establish a comprehensive, applied layout contract, but no Grid.tsx/Row.tsx/Column.tsx source or automated conformance check appears in the evidence.
What to change
Include Grid/Row/Column implementation source and any lint or test coverage of grid usage to substantiate a move toward anchor 4 (validated, measured layout use).
Evidence citations
Composition guidancePasses: 3, 3, 3; approve3 / 4

Passes: 3, 3, 3

Composition rules are documented extensively and consistently across the declared create-configure-save flow and loading-recovery pattern: Modal anatomy explicitly allows Data table/Form/Progress-indicator content to occupy the body; Button docs define allowed/disallowed button-group combinations; the Forms pattern defines button alignment, bleed, and emphasis rules inside dialogs/side panels; Notification docs define placement composition relative to forms (above submit/cancel) and callouts. Crucially, this is not prose-only: ModalFooter.tsx contains real operational composition code: SecondaryButtonSet composes Button and ButtonSet, and the file imports InlineLoading for use inside the modal footer, directly evidencing a coded composition contract for Button+Loading+Modal matching two of the three declared compositions. This combination of consistent documented rules plus one directly observed code composition supports 'composition contracts cover supported product assemblies.' No evidence of automated/executable validation of these composition rules (for example, lint rules or Code Connect enforcement) was present, so anchor 4 is not met.

Completeness: completeVerification: implementedTest definition observed: noAutomation wiring observed: noCurrent execution result observed: noGeneralizability: sample-only

Selected anchor: Composition contracts cover supported product assemblies

Next anchor: Composition rules are executable or automatically validated
Evidence needed: No automated or executable validation (lint rule, Code Connect enforcement, or test) of these composition contracts is present, which anchor 4 requires.

Scope: composition; sample-only. Directly observed code composition is limited to the Modal footer (Button, ButtonSet, InlineLoading); Grid, Form, TextInput, and Toggle composition is evidenced only through documentation, not source.

What broke
No confirmed break: documented composition rules are corroborated by real ModalFooter code composing Button, ButtonSet, and InlineLoading.
Impact
Teams assembling the declared flow and loading-recovery pattern have both a documented contract and a working code reference for button/loading composition inside a modal footer, reducing ambiguity for that sub-assembly.
Why
ModalFooter.tsx imports and composes Button, ButtonSet, and InlineLoading with explicit secondary/primary button role logic, matching the documented button-group and modal-footer guidance.
What to change
Extend observed code composition evidence to Form+TextInput+Toggle and Notification+Grid assemblies to remove reliance on documentation alone for the remaining flow components.
Evidence citations
  • 8ecdf97a7dd31b0f1380/files/packages/react/src/components/ComposedModal/ModalFooter.tsx, original source: lines 8-16
  • 8ecdf97a7dd31b0f1380/files/packages/react/src/components/ComposedModal/ModalFooter.tsx, original source: lines 31-71
  • 050f13ebce3920eb3d9a/content.txt, original source: lines 145-163
  • 72227fea8b16b41d54c9/content.txt, original source: lines 428-492
  • 3eb605429f1ccd49de1b/content.txt, original source: lines 431-502
  • 5ddd06356d3de1711f12/content.txt, original source: lines 268-279
Patterns and templatesPasses: 3, 3, 3; approve3 / 4

Passes: 3, 3, 3

The four declared patterns (Forms, Loading, Empty states, Notifications) are each documented as detailed, reusable task patterns with anatomy, variants, best practices, and explicit 'Related' links back to their constituent components (for example, Forms pattern links Button, Checkbox, Text input, Toggle; Loading pattern links Data table, Loading, Pagination and cross-links Empty states/Forms/Notifications patterns; Notification pattern links Modal, Notification, UI shell). This directly matches 'patterns cover important tasks and remain linked to components' for the declared sample. The composition packet's own known_limitations disclose that Carbon does not publish a composed Figma flow/pattern artifact for these: an honest scope boundary rather than a design failure, and design is not a required surface for these patterns. No evidence of pattern usage, outcome, or lifecycle measurement was found, so anchor 4 is not met.

Completeness: completeVerification: documentedTest definition observed: noAutomation wiring observed: noCurrent execution result observed: noGeneralizability: sample-only

Selected anchor: Patterns cover important tasks and remain linked to components

Next anchor: Pattern use, outcomes, and lifecycle are measured
Evidence needed: No evidence of measured pattern use, outcomes, or lifecycle tracking (adoption metrics, deprecation cadence) is present, which anchor 4 requires.

Scope: composition; sample-only. Evidence covers only the four declared patterns; it does not establish coverage of Carbon's full pattern catalog (for example, Dialogs, Filtering, Search) beyond the preview-site index, which is supplemental only.

What broke
No confirmed break: the declared patterns are documented and component-linked, but Carbon's own known_limitations disclose no composed Figma flow artifact exists for them.
Impact
Designers and engineers have clear task-pattern guidance and component linkage to implement the declared flow and patterns, but cannot cross-check a canonical composed design artifact: only code/documentation surfaces are required and available.
Why
carbon-forms-pattern, carbon-loading-pattern, carbon-empty-state-pattern, and carbon-notification-pattern each provide anatomy, when-to-use guidance, and explicit 'Related' component/pattern links matching the declared composition scope.
What to change
No change required for the required (code, documentation) surfaces; if design-surface parity becomes required in future, Carbon would need to publish composed pattern artifacts.
Evidence citations
Navigation, focus, and state ownershipPasses: 3, 3, 3; approve3 / 4

Passes: 3, 3, 3

ComposedModal.tsx provides direct, operational (not merely prose) evidence of focus and dismissal ownership: a Tab-wrap focus trap (wrapFocus/wrapFocusWithoutSentinels), Escape-key handling gated by isTopmostVisibleModal to correctly own only the topmost of nested/stacked modals, outside-click ownership that differs by passive vs. transactional modal state (with a runtime warning() guarding against misconfiguration), initial-focus assignment logic (danger button vs. selectorPrimaryFocus vs. close button), and launcher-focus return on close. This is corroborated by normative Modal usage docs describing trigger/focus/loading/validation ownership, Notification docs describing dismissal and keyboard-focus ownership (Tab between action and close-x, Esc), and Loading docs describing interaction-blocking ownership (overlay blocks all interaction; small loading disables related buttons). Together these comprehensively cover dialog, async-state (loading/validation), and nested-composition ownership for the declared flow and patterns. Drawer/side-panel and route-level ownership are not part of the declared flow's component set (grid, form, text-input, toggle, button, notification, modal, loading) and remain unevidenced/unknown rather than a confirmed gap. No evidence of continuous automated verification (for example, focus-trap tests running in CI) was found, so anchor 4 is not met.

Completeness: completeVerification: implementedTest definition observed: noAutomation wiring observed: noCurrent execution result observed: noGeneralizability: sample-only

Selected anchor: Navigation, focus, and state contracts are comprehensive

Next anchor: Cross-composition contracts are executable and continuously verified
Evidence needed: No CI test, automated focus-trap assertion, or other continuous-verification evidence was found for these ownership contracts, which anchor 4 requires; drawer/route-level ownership also remains unevidenced since those constructs are outside the declared flow's component set.

Scope: mixed; sample-only. Ownership evidence is strongest for Modal (direct code) and documentation-level for Notification/Loading; drawer/side-panel and route-transition ownership fall outside the declared flow and pattern component set and are therefore unknown, not confirmed absent.

What broke
No confirmed break within the declared dialog/async-state sample: ownership is comprehensively evidenced. Drawer/route-level ownership is unevidenced but is also outside the declared flow's component set.
Impact
For the create-configure-save flow and loading-recovery pattern, engineers have a coherent, code-backed contract for who owns focus, escape handling, outside-click dismissal, and loading/validation state during a dialog task, reducing risk of conflicting implementations across teams.
Why
ComposedModal.tsx implements isTopmostVisibleModal-gated escape handling, isPassive-based outside-click rules with a runtime warning, and initial-focus/launcher-focus-return logic; Modal, Notification, and Loading usage docs independently corroborate the same ownership behaviors in prose.
What to change
To reach anchor 4, add automated tests or CI-verified assertions for the modal focus trap, nested-modal escape targeting, and loading/validation state transitions so the contract is continuously verified rather than only implemented and documented.
Evidence citations
  • 8ecdf97a7dd31b0f1380/files/packages/react/src/components/ComposedModal/ComposedModal.tsx, original source: lines 373-419
  • 8ecdf97a7dd31b0f1380/files/packages/react/src/components/ComposedModal/ComposedModal.tsx, original source: lines 503-515
  • 8ecdf97a7dd31b0f1380/files/packages/react/src/components/ComposedModal/ComposedModal.tsx, original source: lines 517-535
  • 8ecdf97a7dd31b0f1380/files/packages/react/src/components/ComposedModal/ComposedModal.tsx, original source: lines 556-592
  • 050f13ebce3920eb3d9a/content.txt, original source: lines 282-326
  • 5ddd06356d3de1711f12/content.txt, original source: lines 465-489
  • a82ca9adbd35c0913589/content.txt, original source: lines 209-234

Diagnostic area

Governance findings

Named ownershipPasses: 3, 3, 3; revise3 / 4

Passes: 3, 3, 3

Carbon's ownership model is operational: CODEOWNERS assigns default and path-specific reviewers, pull-request guidance defines developer and design approval rights, issue-triage automation assigns severity, labels, and project metadata, support documentation establishes rotations and named escalation paths, and security reporting includes a published response expectation. This meets operational decision rights and support boundaries. Score 4 is not supported because the available merge-rate metric does not comprehensively measure ownership health, response-time compliance, reviewer load, escalation resolution, or support-service performance.

Completeness: completeVerification: automation-wiredTest definition observed: noAutomation wiring observed: yesCurrent execution result observed: noGeneralizability: system-wide

Selected anchor: Decision rights and support boundaries are operational

Next anchor: Ownership health and service expectations are measured
Evidence needed: Additional evidence must satisfy the next anchor: Ownership health and service expectations are measured

Scope: governance; system-wide. Evidence is public-repository governance material only; private IBM decision records, internal Slack activity, and adoption/ownership-health dashboards referenced in the governance known_limitations are unknown, not scored as absent.

What broke
No confirmed break in documented ownership, but no public measurement of ownership health or service-expectation adherence exists beyond a single weekly merge-rate metric.
Impact
Consumers can see who owns what and how escalation is supposed to work, but cannot verify from public evidence whether commitments like the 7-business-day security response target are actually being met over time.
Why
metrics-merge-rate.yml is the only recurring, automated measurement present in the evidence packet; it tracks PR throughput, not response-time or escalation-SLA compliance.
What to change
Publish a periodic, automated measurement of core ownership-health signals (issue/PR response time versus committed targets, escalation resolution time) using the same weekly-report pattern already established for merge rate.
Evidence citations
  • 850be183e89b93a74a5d/files/.github/CODEOWNERS, original source: lines 1-24
  • 850be183e89b93a74a5d/files/.github/workflows/issue-triage.yml, original source: lines 1-57
  • 850be183e89b93a74a5d/files/docs/guides/reviewing-pull-requests.md, original source: lines 100-113
  • 850be183e89b93a74a5d/files/docs/guides/support.md, original source: lines 54-93
  • 850be183e89b93a74a5d/files/docs/guides/support.md, original source: lines 468-481
  • 850be183e89b93a74a5d/files/SECURITY.md, original source: lines 3-29
  • 850be183e89b93a74a5d/files/.github/workflows/metrics-merge-rate.yml, original source: lines 16-58
Contribution and reviewPasses: 3, 3, 3; revise3 / 4

Passes: 3, 3, 3

Carbon operates a cross-discipline contribution and review process with a defined four-stage pull-request review, criteria spanning implementation, testing, Storybook, styling, CI, and other change types, two developer approvals, an additional design approval for qualifying visual or interaction changes, author self-review checklists, CODEOWNERS assignment, and automated quality gates. This meets operational cross-discipline review criteria and decisions. Score 4 is not supported because public measurement covers some merge throughput but not review quality and outcomes such as defect escape, post-merge reverts, or review effectiveness.

Completeness: completeVerification: automation-wiredTest definition observed: noAutomation wiring observed: yesCurrent execution result observed: noGeneralizability: system-wide

Selected anchor: Cross-discipline review criteria and decisions are operational

Next anchor: Review quality, throughput, and outcomes are measured
Evidence needed: Additional evidence must satisfy the next anchor: Review quality, throughput, and outcomes are measured

Scope: governance; system-wide. Evidence covers the public GitHub contribution/review process and one supplemental preview-site overview page; internal design-crit and CAG meeting outcomes, and non-public review discussions, are unknown.

What broke
No confirmed break in the documented cross-discipline review process, but no public evidence measures review quality or outcomes beyond weekly merge throughput.
Impact
Stakeholders can confirm an operational, criteria-driven review path exists, but cannot independently verify from public evidence that it is producing measurably better outcomes (fewer defects, fewer reverts) over time.
Why
metrics-merge-rate.yml measures only merge throughput; no review-quality or downstream-outcome metric is present in the evidence packet.
What to change
Extend the existing automated metrics reporting beyond merge rate to include review-quality/outcome signals (for example, post-merge revert rate, accessibility-regression rate tied to reviewed PRs) and publish them on the same recurring cadence.
Evidence citations
  • 850be183e89b93a74a5d/files/docs/guides/reviewing-pull-requests.md, original source: lines 37-113
  • 850be183e89b93a74a5d/files/.github/PULL_REQUEST_TEMPLATE.md, original source: lines 23-44
  • 850be183e89b93a74a5d/files/.github/CODEOWNERS, original source: lines 1-24
  • c7035884c47c4a634eff/files/.github/workflows/ci.yml, original source: lines 29-124
  • 850be183e89b93a74a5d/files/.github/workflows/metrics-merge-rate.yml, original source: lines 16-58
Releases, migration, and deprecationPasses: 3, 3, 3; revise3 / 4

Passes: 3, 3, 3

Carbon operates explicit release-safety contracts through semantic-versioning rules, preview/prerelease/active/maintenance/LTS phases, committed future-major feature flags, versioned migration guides, tested @carbon/upgrade codemods, Public API snapshot checks, automated release workflows with CI and accessibility gates, and supported-version and security-response policies. This meets operational deprecation and migration contracts. Score 4 is not supported because consumer migration outcomes are not publicly measured: no migration completion rates, post-release breakage rates, compatibility incident metrics, or adoption-health evidence was found.

Completeness: completeVerification: automation-wiredTest definition observed: yesAutomation wiring observed: yesCurrent execution result observed: noGeneralizability: system-wide

Selected anchor: Deprecation and migration have explicit operational contracts

Next anchor: Compatibility and migration success are validated and measured
Evidence needed: Additional evidence must satisfy the next anchor: Compatibility and migration success are validated and measured

Scope: governance; system-wide. Evidence covers public versioning/migration documentation and release automation for the current v11 line and the v12 preview; private migration outcome data and consumer adoption telemetry are outside public evidence per the governance known_limitations.

What broke
No confirmed break in the documented deprecation/migration contract, but migration success itself is not measured in public evidence: only API compatibility is checked via the PublicAPI snapshot test.
Impact
Consumers can rely on a concrete, documented semver/migration contract and an automated compatibility check, but cannot independently verify that migrations succeed in practice (for example, how many consumers adopt committed v12 flags cleanly before the major bump).
Why
versioning.md documents the PublicAPI snapshot mechanism enforced in CI, and release-schedule.md documents the committed-flag preview contract, but no adoption or migration-success telemetry is present in public evidence; the governance known_limitations explicitly exclude adoption dashboards.
What to change
Publish aggregate, privacy-safe migration/adoption telemetry (for example, pre-release adoption rate of committed enable-v12-* flags, or post-migration defect reports) to close the gap between the documented migration contract and measured migration success.
Evidence citations
  • 3eb54126648a690950d8/files/docs/guides/versioning.md, original source: lines 27-72
  • 3eb54126648a690950d8/files/docs/guides/versioning.md, original source: lines 292-299
  • 3eb54126648a690950d8/files/docs/release-schedule.md, original source: lines 18-62
  • 3eb54126648a690950d8/files/docs/preview-code.md, original source: lines 80-104
  • 3eb54126648a690950d8/files/docs/migration/v11.md, original source: lines 1-26
  • 3eb54126648a690950d8/files/packages/upgrade/README.md, original source: lines 84-120
  • 3eb54126648a690950d8/files/packages/upgrade/transforms/__tests__/enable-v12-release-test.js, original source: lines 1-35
  • 3eb54126648a690950d8/files/.github/workflows/release.yml, original source: lines 32-57
Quality enforcement and feedbackPasses: 3, 3, 3; revise3 / 4

Passes: 3, 3, 3

Carbon operates quality gates across its delivery lifecycle through formatting and lint checks, unit tests with coverage, build and generated-style checks, end-to-end tests, sharded Playwright accessibility tests, Chromatic visual regression, change-triggered and weekly CodeQL scanning, release-time accessibility checks, and published severity and exception-handling rules. These are active workflows with direct assertions and meet automated lifecycle-and-exception enforcement. Score 4 is not supported because no public quality-trend reporting, waiver and exception frequency, regression rates, recurring accessibility trends, or follow-through outcome measurement was evidenced; merge-rate reporting measures throughput rather than quality.

Completeness: completeVerification: automation-wiredTest definition observed: yesAutomation wiring observed: yesCurrent execution result observed: noGeneralizability: system-wide

Selected anchor: Quality gates cover the supported lifecycle and exceptions

Next anchor: Quality, adoption, exceptions, and regressions are measured and acted on
Evidence needed: Additional evidence must satisfy the next anchor: Quality, adoption, exceptions, and regressions are measured and acted on

Scope: governance; system-wide. Evidence covers CI/CD workflow definitions and a sample of AVT e2e test files across 11 components; it does not establish that every component or pattern in the full catalog has equivalent AVT coverage, and it does not include private DAP audit records referenced by accessibility.md.

What broke
No confirmed break in gate coverage, but no public evidence measures quality trends, exception/waiver frequency, or regressions over time beyond a weekly merge-rate figure.
Impact
The public evidence shows real, currently-enforced automated gates (a blocking AVT accessibility job, CodeQL scanning, unit tests) plus documented exception handling, but leaves it unclear how often exceptions are invoked or whether gate effectiveness is monitored and acted on over time.
Why
accessibility.md's false-positive dismissal and the PR template's strikethrough mechanism show exception handling exists, but no dashboard or periodic report of exception frequency or gate-failure/regression trend is present in the evidence packet, unlike the weekly merge-rate metric.
What to change
Track and periodically report exception/waiver frequency (accessibility false-positive dismissals, PR-checklist strikethroughs) and gate-failure/regression trends, extending the reporting pattern already established for the weekly merge-rate metric.
Evidence citations
  • c7035884c47c4a634eff/files/.github/workflows/ci.yml, original source: lines 83-124
  • c7035884c47c4a634eff/files/.github/workflows/ci.yml, original source: lines 240-351
  • c7035884c47c4a634eff/files/.github/workflows/ci.yml, original source: lines 403-456
  • c7035884c47c4a634eff/files/.github/workflows/codeql-analysis.yml, original source: lines 1-35
  • c7035884c47c4a634eff/files/e2e/components/Button/Button-test.avt.e2e.js, original source: lines 13-23
  • c7035884c47c4a634eff/files/docs/guides/accessibility.md, original source: lines 31-42
  • 850be183e89b93a74a5d/files/docs/guides/support.md, original source: lines 356-425