docs: present AI strategy as a diagram-first story
Build and publish Eldorado documentation / build (pull_request) Successful in 40s
Build and publish Eldorado documentation / build (pull_request) Successful in 40s
This commit is contained in:
1 parent
7f40de708c
commit
794f2aa2d6
30 files changed
+2496
-4
No files matched your search
@@ -2,11 +2,30 @@
|
||||
id: measurement
|
||||
title: Measurement
|
||||
description: Full KPI contracts, minimal events, illustrative calculations, privacy and limits.
|
||||
hide_table_of_contents: true
|
||||
---
|
||||
|
||||
import ArchifyDiagram from '@site/src/components/ArchifyDiagram';
|
||||
|
||||
<div className="strategy-content">
|
||||
|
||||
**TO-BE · Target operating model — not installed controls or company results.**
|
||||
|
||||
## Is AI-assisted work worth continuing?
|
||||
|
||||
Judge accepted outcomes alongside quality, delivery flow, risk, cost and human experience. Validate source coverage before calculating; missing evidence is unknown, not success.
|
||||
|
||||
<ArchifyDiagram name="strategy-measurement" title="Collect evidence, validate it, calculate balanced measures and decide" story />
|
||||
|
||||
**Illustrative only:** 1 accepted task ÷ 4 matured starts × 100 = **25%** confirmed success. One unresolved task stays in the denominator; known-outcome coverage is **75%**. These are synthetic data, not Eldorado results.
|
||||
|
||||
- **16 contracts:** exact definitions below; 9 prototype calculations, 7 explicitly unimplemented metrics.
|
||||
- **Balanced evidence:** no productivity rankings, composite score or causal ROI claim.
|
||||
- **Privacy first:** no raw prompts or code by default; protect small cohorts and agree retention before collection.
|
||||
|
||||
<details>
|
||||
<summary>Collection, baseline, event and privacy contracts</summary>
|
||||
|
||||
**TO-BE · Target operating model.** Introduction depends on the [roadmap gates](./roadmap.mdx); these pages do not describe installed controls or measured company results.
|
||||
|
||||
### Decide what counts before counting
|
||||
@@ -33,6 +52,11 @@ Many-to-many task/attempt/skill/client associations preserve exact immutable ver
|
||||
|
||||
The privacy starting proposal is 30 days for raw structured analytics events and 12 months for aggregated trends, subject to the responsible owners' review. Keep the approved catalog/approval record for the supported revision's lifetime and its required audit period, which must be decided separately. Incident evidence uses a restricted, explicitly authorized hold and deletion schedule, not an indefinite analytics exception. Suppress reports below five people and prohibit drill-down or overlapping cohort slices that reconstruct identities. Five is a proposed minimum, not a guarantee against re-identification. No raw prompts, code, screens, customer cases or secrets are collected by default. Publish the purpose, lawful collection basis, access/deletion route and retention before collecting. Consent is not presumed to be an appropriate employment-law basis.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>All 16 KPI contracts — formulas, populations, owners and limits</summary>
|
||||
|
||||
## Metric contracts
|
||||
|
||||
Each contract is a prospective collection design. No live source or observed baseline is established. N and D refer to the numerator and denominator shown for that metric.
|
||||
@@ -341,6 +365,11 @@ Independently accepted domain outcomes against a predeclared business rubric, se
|
||||
| Gaming and limits | Weaken rubric or count drafts as customer benefit; lock rubric and independently audit acceptance. Descriptive signal, not causal AI productivity attribution. Source completeness, task mix and sampling limit inference; report uncertainty. Domain owner signs acceptance: Operations service accuracy; Marketing claims/rights; Finance reconciliation accuracy; HR policy correctness; Product usability/value. Not revenue uplift. |
|
||||
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>Run the nine-metric synthetic prototype and inspect omissions</summary>
|
||||
|
||||
## Run the synthetic measurement example
|
||||
|
||||
**Executed synthetic example — not Eldorado results.** The bundled fixture produces 1 independently accepted task from 4 matured starts: 25% confirmed success. One unresolved task remains in that denominator; known-outcome coverage is 75%. Selection precision and recall are each 1 / 2 = 50%. Allocated synthetic cost is EUR 60 / 1 accepted outcome = EUR 60 per accepted outcome. These deliberately small samples demonstrate arithmetic and missingness, not a baseline, target or ROI claim.
|
||||
@@ -349,4 +378,8 @@ From the private documentation checkout, run `node scripts/ai-kpi.mjs` and `node
|
||||
|
||||
The executable covers validated success, selection precision/recall, change fail and rework rates, escaped quality, policy effects, cost and human experience. Other catalog metrics remain contracts rather than implemented collectors; output lists the omissions. Snapshot inputs are simplified, pre-reconciled records, not a production ingestion system. Conflicting duplicates reject the calculation rather than silently choosing a value.
|
||||
|
||||
</details>
|
||||
|
||||
**Next:** [Use findings to maintain, quarantine or replace skills →](./skills-maintenance.mdx)
|
||||
|
||||
</div>
|
||||
Reference in new issue
Block a user