jarvis-at-skic
f7a68d7e37
Build and publish Eldorado documentation / build (pull_request) Successful in 42s
39 lines
2.1 KiB
Plaintext
39 lines
2.1 KiB
Plaintext
---
|
|
id: skill-selection-recall
|
|
slug: /measurement/skill-selection-recall
|
|
title: Skill selection recall
|
|
description: Definition, collection evidence and limitations for skill selection recall.
|
|
---
|
|
|
|
<div className="strategy-content">
|
|
|
|
**Target operating model — recommended contracts, not installed collection.** All owner and cadence entries below are proposals to confirm before the pilot, not assigned responsibilities or approved schedules.
|
|
|
|
**Prototype calculation — no live telemetry.** The synthetic example calculates this metric; it has no network collectors.
|
|
|
|
{/* contract:start */}
|
|
Relevant task-trials in which the intended skill activates.
|
|
|
|
| Contract | Definition |
|
|
| --- | --- |
|
|
| Numerator | True-positive activations |
|
|
| Denominator | True-positive plus false-negative task-trials |
|
|
| Expression | N / D * 100 |
|
|
| Aggregation | percent |
|
|
| Unit | % |
|
|
| Eligible population | Independently labeled relevant evaluation task-trials for a skill/client tuple. |
|
|
| Exclusions | Irrelevant tasks; missing relevance or activation reported as unscorable, not negative. |
|
|
| Events and source | eval.selection; versioned labeled corpus and runner |
|
|
| Owner / cadence | Data & AI / Each skill/client qualification |
|
|
| Missing evidence | Report evaluation coverage; null if no relevant labeled tasks. |
|
|
| Gaming and limits | Overtrigger to maximize recall; always pair with precision and negative cases. Descriptive signal, not causal AI productivity attribution. Source completeness, task mix and sampling limit inference; report uncertainty. Loading the right skill is not adherence. |
|
|
{/* contract:end */}
|
|
|
|
Read the shared [collection and safeguards](./collection-and-safeguards.mdx) before collecting data; see the [synthetic example and limits](./synthetic-example.mdx).
|
|
|
|
Return to [Skills Measurement](./index.mdx).
|
|
|
|
Use this labeled-case result alongside precision in [Quality's candidate evaluation](../skills-quality.mdx). Finding expected skills does not establish useful artifacts or safe execution. Preserve missing-case coverage and revision-regression evidence before a bounded pilot; do not pool evaluation cases with live Delivery tasks.
|
|
|
|
</div>
|