docs: [verified] answer the six-month AI Architect assignment
Build and publish Eldorado documentation / build (pull_request) Successful in 1m30s

This commit is contained in:
jarvis-at-skic committed 2026-10-08 16:59:36 +00:00
1 parent 88a4375776
commit 1a2d16bc91
53 files changed
+974 -390

No files matched your search

+8 -4
View File
@@ -11,6 +11,8 @@ import ArchifyDiagram from '@site/src/components/ArchifyDiagram';
**TO-BE · Target operating model — not installed controls or company results.**
**Optional skills reference for the [six-month plan](./roadmap.mdx).** Use this when skills are the chosen intervention, not as a prerequisite for discovery or coaching. Department names denote functions to map to actual willing staff; the architect cannot assign them authority.
## What evidence is enough to promote a skill?
Risk determines the evaluation depth, not whether safety checks apply. Independent evidence must show useful outcomes and no failed mandatory gate before promotion.
@@ -23,7 +25,9 @@ Risk determines the evaluation depth, not whether safety checks apply. Independe
## Prove the candidate fits the workflow
Bring the workflow/acceptance map from [Baseline](./baseline.mdx) and the pinned candidate, client tuple and authority envelope from [Governance](./skills-governance.mdx). In the **illustrative Delivery workflow**, test whether a candidate actually produces a correct bounded change and regression tests—not just whether it was selected. Compare candidate, current-approved and no-skill paths on the same starting artifacts. Include misleading repository instructions, missing tests and requests to bypass review.
Bring the workflow/acceptance map from [Baseline](./baseline.mdx) and the pinned candidate, client tuple and authority envelope from [Governance](./skills-governance.mdx). In the **illustrative Delivery workflow**, test whether a candidate actually produces a correct bounded change and regression tests—not just whether it was selected. Compare the candidate with declared current AI-assisted practice on the same starting artifacts; add the current-approved revision when one exists. Include misleading repository instructions, missing tests and requests to bypass review.
**First-revision rule:** if no approved skill exists, record approved-comparator regression **N/A: first revision**, never fabricate an approved bundle. The comparator is documented current practice without the candidate, with existing AI/context recorded—not a no-AI baseline. Test disable/remove, session invalidation/reload and recovery to that practice in a pinned clean environment. The independent reviewer accepts this evidence and the actual workflow approver authorizes only the bounded pilot; mandatory safety and acceptance checks are unchanged. Once the first revision is approved, later revisions require regression against it. See [EXP-01](./stage-01-skills-adoption.mdx) for the assembled packet.
The independent Quality Engineering reviewer records accepted and failed artifacts, safety results, coverage and regression differences for each named revision. The owning team's designated approver then decides whether that evidence supports a bounded pilot; Architecture & Security must accept any higher-risk boundary. A mandatory failure or an unagreed acceptance rule means rework or hold. These are proposed decision responsibilities, not already assigned people.
@@ -42,7 +46,7 @@ The proposed qualification contract uses three tiers. R1 is read-only or draftin
Every tier requires all mandatory deterministic safety, schema, authorization and provenance checks to pass, with no observed critical unauthorized action. An observed critical effect blocks promotion. The proposed pilot evaluation floors below set success thresholds; these are design defaults for owner ratification, not externally validated guarantees or approved service levels. If sample coverage, acceptance criteria or responsible reviewers are missing, qualification remains incomplete. No aggregate score can trade off a failed mandatory safety check.
Use sanitized, predeclared task sets containing positive triggers, near misses, negative cases and the normal workload mix. Compare no-skill, current-approved and candidate paths with the same starting artifacts; repeat stochastic trials and retain failed and abandoned attempts. Independently inspect output state. Deterministic tests grade checkable constraints; calibrated humans grade usefulness, maintainability and experience. A model judge is supplementary, versioned and checked against human disagreements, never the sole safety or promotion authority.
Use sanitized, predeclared task sets containing positive triggers, near misses, negative cases and the normal workload mix. Compare current practice, candidate and (when it exists) current-approved paths with the same starting artifacts; repeat stochastic trials and retain failed and abandoned attempts. A no-standardized-skill path may still use AI. Independently inspect output state. Deterministic tests grade checkable constraints; calibrated humans grade usefulness, maintainability and experience. A model judge is supplementary, versioned and checked against human disagreements, never the sole safety or promotion authority.
Include adversarial repository text, malicious referenced code, wrong permissions, client toggles, name shadowing, unsupported versions, hook timeout/crash/malformed response, stale sessions, offline recovery and rollback. Run central checks against forged/missing client declarations. Hook presence is not fail-closed enforcement: failure semantics must be tested on each client, and central authorization remains independent.
@@ -55,7 +59,7 @@ Report denominators, task mix, trial counts, per-client results and uncertainty.
## Proposed qualification floors
These are local design defaults for responsible-owner ratification, not approved service levels or guarantees. All mandatory checks must pass regardless of aggregate score.
These are optional reference designs, not mandatory startup floors, approved service levels or guarantees. Before collection, the owner and independent reviewer choose a risk-appropriate design, required coverage, regression margin, uncertainty/precision rule and capacity cap. Start with one workflow and one client configuration. Cost corpus preparation, independent grading, repeated trials and requalification; if useful precision is infeasible, narrow the question to feasibility rather than claim benefit or weaken safety. All mandatory checks must pass regardless of aggregate score. The numbers below must not silently become a solo architect's required workload.
### R1
@@ -87,7 +91,7 @@ These are local design defaults for responsible-owner ratification, not approved
**Selection precision / recall** — 99% / 95%
Artifact success denominator is every planned executed trial, retaining failures and abandonment. Trials of same task are correlated: show per-task and per-client strata; do not inflate independent sample size. Selection floors use independent labels. Report 95% uncertainty intervals clustered by task for outcome comparisons; no regression versus current approved path beyond a preregistered owner-approved margin. Missing margin/precision rule prevents promotion.
Artifact success denominator is every planned executed trial, retaining failures and abandonment. Trials of same task are correlated: show per-task and per-client strata; do not inflate independent sample size. Selection floors use independent labels. Report 95% uncertainty intervals clustered by task for outcome comparisons; no regression versus the current approved path, when one exists, beyond a preregistered owner-approved margin. First-revision approved regression is N/A with reason, but current-practice acceptance and safety remain mandatory. Missing applicable margin/precision rule prevents promotion. Small feasibility studies cannot establish that rule after seeing favorable results.
</details>