jarvis-at-skic
85852f0744
Build and publish Eldorado documentation / build (pull_request) Successful in 41s
37 lines
2.4 KiB
Plaintext
37 lines
2.4 KiB
Plaintext
---
|
|
id: validated-task-success
|
|
slug: /measurement/validated-task-success
|
|
title: Validated task success
|
|
description: Definition, collection evidence and limitations for validated task success.
|
|
---
|
|
|
|
<div className="strategy-content">
|
|
|
|
**Target operating model — recommended contracts, not installed collection.** All owner and cadence entries below are proposals to confirm before the pilot, not assigned responsibilities or approved schedules.
|
|
|
|
**Prototype calculation — no live telemetry.** The synthetic example calculates this metric; it has no network collectors.
|
|
|
|
{/* contract:start */}
|
|
Confirmed independently accepted tasks as a share of all eligible initiated tasks in a matured start cohort; a conservative confirmed-success share, not observed failure rate.
|
|
|
|
| Contract | Definition |
|
|
| --- | --- |
|
|
| Numerator | Distinct matured task_ids with valid independent task.accepted evidence by as_of |
|
|
| Denominator | All distinct eligible task_ids started in window with started_at + preregistered task_observation_hours <= as_of |
|
|
| Expression | N / D * 100 |
|
|
| Aggregation | percent |
|
|
| Unit | % |
|
|
| Eligible population | Tasks in a preregistered workflow cohort, started in window; each counted once across retries, skills and clients. |
|
|
| Exclusions | Cancelled before start, sandbox/evaluation tasks and out-of-scope workflows; never post-start failures/abandonments. |
|
|
| Events and source | task.started; task.accepted; task.failed; task.abandoned; task.cancelled; work-management acceptance record |
|
|
| Owner / cadence | Quality Engineering / Weekly pilot; monthly departmental outcome review |
|
|
| Missing evidence | Unresolved/missing terminal outcomes remain in denominator with unknown count and known-outcome coverage; do not label them failed. Invalid/missing start excluded from calculation but flagged as orphan evidence. |
|
|
| Gaming and limits | Split easy tasks or omit starts; reconcile starts to approved backlog and audit acceptance rubrics. Descriptive signal, not causal AI productivity attribution. Source completeness, task mix and sampling limit inference; report uncertainty. Maturation horizon is workflow-specific and must be approved; no automatic abandonment on timeout. |
|
|
{/* contract:end */}
|
|
|
|
Read the shared [collection and safeguards](./collection-and-safeguards.mdx) before collecting data; see the [synthetic example and limits](./synthetic-example.mdx).
|
|
|
|
Return to [Skills Measurement](./index.mdx).
|
|
|
|
</div>
|