docs: present AI strategy as a diagram-first story
Build and publish Eldorado documentation / build (pull_request) Successful in 40s

This commit is contained in:
jarvis-at-skic committed 2026-10-04 12:23:41 +00:00
1 parent 7f40de708c
commit 794f2aa2d6
30 files changed
+2496 -4

No files matched your search

+33
View File
@@ -2,11 +2,30 @@
id: measurement
title: Measurement
description: Full KPI contracts, minimal events, illustrative calculations, privacy and limits.
hide_table_of_contents: true
---
import ArchifyDiagram from '@site/src/components/ArchifyDiagram';
<div className="strategy-content">
**TO-BE · Target operating model — not installed controls or company results.**
## Is AI-assisted work worth continuing?
Judge accepted outcomes alongside quality, delivery flow, risk, cost and human experience. Validate source coverage before calculating; missing evidence is unknown, not success.
<ArchifyDiagram name="strategy-measurement" title="Collect evidence, validate it, calculate balanced measures and decide" story />
**Illustrative only:** 1 accepted task ÷ 4 matured starts × 100 = **25%** confirmed success. One unresolved task stays in the denominator; known-outcome coverage is **75%**. These are synthetic data, not Eldorado results.
- **16 contracts:** exact definitions below; 9 prototype calculations, 7 explicitly unimplemented metrics.
- **Balanced evidence:** no productivity rankings, composite score or causal ROI claim.
- **Privacy first:** no raw prompts or code by default; protect small cohorts and agree retention before collection.
<details>
<summary>Collection, baseline, event and privacy contracts</summary>
**TO-BE · Target operating model.** Introduction depends on the [roadmap gates](./roadmap.mdx); these pages do not describe installed controls or measured company results.
### Decide what counts before counting
@@ -33,6 +52,11 @@ Many-to-many task/attempt/skill/client associations preserve exact immutable ver
The privacy starting proposal is 30 days for raw structured analytics events and 12 months for aggregated trends, subject to the responsible owners' review. Keep the approved catalog/approval record for the supported revision's lifetime and its required audit period, which must be decided separately. Incident evidence uses a restricted, explicitly authorized hold and deletion schedule, not an indefinite analytics exception. Suppress reports below five people and prohibit drill-down or overlapping cohort slices that reconstruct identities. Five is a proposed minimum, not a guarantee against re-identification. No raw prompts, code, screens, customer cases or secrets are collected by default. Publish the purpose, lawful collection basis, access/deletion route and retention before collecting. Consent is not presumed to be an appropriate employment-law basis.
</details>
<details>
<summary>All 16 KPI contracts — formulas, populations, owners and limits</summary>
## Metric contracts
Each contract is a prospective collection design. No live source or observed baseline is established. N and D refer to the numerator and denominator shown for that metric.
@@ -341,6 +365,11 @@ Independently accepted domain outcomes against a predeclared business rubric, se
| Gaming and limits | Weaken rubric or count drafts as customer benefit; lock rubric and independently audit acceptance. Descriptive signal, not causal AI productivity attribution. Source completeness, task mix and sampling limit inference; report uncertainty. Domain owner signs acceptance: Operations service accuracy; Marketing claims/rights; Finance reconciliation accuracy; HR policy correctness; Product usability/value. Not revenue uplift. |
</details>
<details>
<summary>Run the nine-metric synthetic prototype and inspect omissions</summary>
## Run the synthetic measurement example
**Executed synthetic example — not Eldorado results.** The bundled fixture produces 1 independently accepted task from 4 matured starts: 25% confirmed success. One unresolved task remains in that denominator; known-outcome coverage is 75%. Selection precision and recall are each 1 / 2 = 50%. Allocated synthetic cost is EUR 60 / 1 accepted outcome = EUR 60 per accepted outcome. These deliberately small samples demonstrate arithmetic and missingness, not a baseline, target or ROI claim.
@@ -349,4 +378,8 @@ From the private documentation checkout, run `node scripts/ai-kpi.mjs` and `node
The executable covers validated success, selection precision/recall, change fail and rework rates, escaped quality, policy effects, cost and human experience. Other catalog metrics remain contracts rather than implemented collectors; output lists the omissions. Snapshot inputs are simplified, pre-reconciled records, not a production ingestion system. Conflicting duplicates reject the calculation rather than silently choosing a value.
</details>
**Next:** [Use findings to maintain, quarantine or replace skills →](./skills-maintenance.mdx)
</div>