diff --git a/SKILL.md b/SKILL.md new file mode 100644 index 0000000..d678813 --- /dev/null +++ b/SKILL.md @@ -0,0 +1,101 @@ +--- +name: corp-v1-channel-uat +description: Use when validating a deployed Corp v1 environment through UAT journeys. +version: 1.0.0 +author: Hermes Agent +license: MIT +metadata: + hermes: + tags: [corp-v1, discord, channel, uat, e2e, playwright, acceptance] +--- + +# Corp v1 UAT Channel + +## Purpose + +The UAT channel owns repeatable end-to-end acceptance testing against an already deployed environment. It sits after Delivery and before Releases. + +UAT does not build application images, change manifests, synchronize Argo CD, patch workloads, or approve a release. It verifies user-visible behavior and returns evidence tied to an exact deployed revision. + +## Required input gate + +Do not start a run until Delivery provides: + +- application repository and exact commit; +- immutable OCI image digest for every tested workload; +- deployable GitOps repository and exact commit; +- Argo CD Application name, synchronized revision, sync state, and health state; +- target environment and base URL; +- deployment timestamp and relevant task/feature/release identity; +- known limitations and required test-data preconditions. + +If any identity is mutable, missing, or contradictory, record `BLOCKED` rather than testing an unknown deployment. + +## Responsibilities + +1. Maintain a dedicated pnpm E2E repository with versioned journeys. +2. Execute journeys against the exact deployed environment supplied by Delivery. +3. Collect machine-readable and human-reviewable evidence. +4. Distinguish product failures, environment failures, and test defects. +5. Return product/environment failures to Kanban and Delivery with reproducible evidence. +6. Rerun the affected journey set after a verified redeployment. +7. Hand a `UAT_PASSED` or `UAT_FAILED` verdict to Releases without implying release approval. + +## UAT verdicts + +These verdicts are acceptance dimensions; they do not replace the project's canonical Task status: + +- `NOT_READY` — deployment evidence is incomplete. +- `READY_FOR_UAT` — the exact deployment gate is satisfied. +- `UAT_IN_PROGRESS` — a run is active against the frozen deployment identity. +- `UAT_FAILED` — one or more required journeys failed. +- `UAT_PASSED` — every required journey passed for the frozen identity. +- `BLOCKED` — execution cannot proceed for a recorded external reason. + +A new application image, GitOps commit, Argo CD revision, or materially changed environment invalidates an earlier verdict and requires a new run identity. + +## Workflow + +1. Verify the Delivery handoff and freeze the deployment identity. +2. Select journeys by explicit tags or release scope; never silently omit a required journey. +3. Run the repository's pnpm/Taskfile validation and Playwright suite. +4. Preserve JUnit/JSON results, traces, screenshots on failure, browser/project matrix, start/end time, and target identity. +5. Classify every failure and publish the smallest reproducible report. +6. Route implementation or environment defects back to Delivery through the project's Kanban process. +7. Rerun after Delivery supplies a new verified deployment identity. +8. Send the final UAT verdict and evidence index to Releases. + +## Evidence minimum + +Every reported run includes: + +```text +run_id +journey_repository + exact commit +application_repository + exact commit +image digests +GitOps repository + exact commit +Argo CD application + synchronized revision + sync/health +base URL and environment +journey IDs/tags and browser matrix +started_at + completed_at +result counts +artifact paths/URLs +failure classification and linked defect/task +verdict +``` + +Do not publish credentials, cookies, tokens, personal test data, videos containing secrets, or raw environment dumps. + +## Authority + +UAT may autonomously run approved journeys, collect evidence, identify reproducible defects, maintain test implementation, and issue a verdict grounded in complete evidence. + +UAT must not modify product requirements, application behavior, deployment desired state, cluster resources, release allocation, release approval, or canonical Task lifecycle to make a test pass. + +## References + +- [Journey repository contract](references/journey-repository-contract.md) +- [Deployment evidence contract](references/deployment-evidence-contract.md) +- [Defect and retest workflow](references/defect-and-retest-workflow.md) +- [Status and reporting](references/status-and-reporting.md) diff --git a/references/defect-and-retest-workflow.md b/references/defect-and-retest-workflow.md new file mode 100644 index 0000000..2e876aa --- /dev/null +++ b/references/defect-and-retest-workflow.md @@ -0,0 +1,18 @@ +# Defect and retest workflow + +## Classification + +- **Product defect:** deployed behavior contradicts an approved acceptance outcome or requirement. +- **Environment defect:** routing, dependency, data, identity, or infrastructure prevents valid execution. +- **Test defect:** the journey implementation or fixture is incorrect. +- **Unresolved:** evidence is insufficient; investigate without assigning blame. + +## Return packet + +A returned failure names the journey ID, deployment tuple, first failing step/assertion, expected and actual behavior, trace/screenshot reference, reproducibility, classification, and owning Kanban/Delivery task. + +UAT does not patch the application or desired state. Delivery supplies a new application/image/GitOps identity after remediation. + +## Retest + +Run the failed journey, materially coupled journeys, and the required smoke set. A narrow retest may verify the correction, but the final verdict must satisfy the release's declared journey set against one frozen deployment identity. diff --git a/references/deployment-evidence-contract.md b/references/deployment-evidence-contract.md new file mode 100644 index 0000000..969e931 --- /dev/null +++ b/references/deployment-evidence-contract.md @@ -0,0 +1,25 @@ +# Deployment evidence contract + +A UAT run binds to one immutable deployment tuple: + +```text +application_commit +image_digests +GitOps_commit +ArgoCD_application +ArgoCD_sync_revision +cluster_environment +base_url +``` + +Before execution, assert that the Argo CD application is synchronized to the declared GitOps commit and healthy. Where the environment exposes a version endpoint, compare its source/image identity with the handoff. + +Reject the handoff when: + +- an image tag is mutable and no digest is supplied; +- Argo CD reports `OutOfSync`, `Unknown`, `Degraded`, or another non-healthy state; +- the deployed source cannot be correlated to the handoff; +- the base URL resolves to a different environment; +- a newer synchronization occurs after the run begins. + +If deployment identity changes during execution, end the run as invalidated and start a new run after the new identity is frozen. diff --git a/references/journey-repository-contract.md b/references/journey-repository-contract.md new file mode 100644 index 0000000..14542a1 --- /dev/null +++ b/references/journey-repository-contract.md @@ -0,0 +1,29 @@ +# Journey repository contract + +The UAT repository is a standalone pnpm project. It contains no Kubernetes/Helm/Argo CD manifests and no application implementation. + +Required baseline: + +```text +journeys/ + smoke/ + regression/ +fixtures/ +helpers/ +playwright.config.ts +package.json +pnpm-lock.yaml +Taskfile.yml +.gitea/workflows/validate.yaml +``` + +Rules: + +- Stable journey IDs appear in test titles and evidence. +- `UAT_BASE_URL` is required; no production-looking default is embedded. +- Authentication is loaded from runtime secrets and never persisted in storage-state files committed to Git. +- Retries are explicit and low; a retry cannot turn a flaky first failure into an undisclosed pass. +- Failure evidence includes trace and screenshot; video is opt-in when data handling permits it. +- CI validates types, lint, journey discovery, and a non-network contract test. Environment execution is a separately authorized job. +- Taskfile is the human/automation entry point and delegates to package scripts. +- The repository README documents local execution, evidence locations, environment variables by name only, and failure classification. diff --git a/references/status-and-reporting.md b/references/status-and-reporting.md new file mode 100644 index 0000000..0e925a0 --- /dev/null +++ b/references/status-and-reporting.md @@ -0,0 +1,18 @@ +# Status and reporting + +Lead with the verdict and exact environment. Keep canonical Task flow, deployment state, UAT verdict, and release decision separate. + +Recommended update: + +```markdown +## UAT result — +- Run: +- Environment: +- Deployment: app ; images ; GitOps ; Argo CD +- Journeys: //; +- Evidence: +- Defects/blockers: +- Next action: +``` + +Never say “released,” “production-ready,” or “DONE” from a UAT pass alone. Releases owns promotion and final release status.