feat: add Corp v1 UAT channel skill
This commit is contained in:
@@ -0,0 +1,101 @@
|
||||
---
|
||||
name: corp-v1-channel-uat
|
||||
description: Use when validating a deployed Corp v1 environment through UAT journeys.
|
||||
version: 1.0.0
|
||||
author: Hermes Agent
|
||||
license: MIT
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [corp-v1, discord, channel, uat, e2e, playwright, acceptance]
|
||||
---
|
||||
|
||||
# Corp v1 UAT Channel
|
||||
|
||||
## Purpose
|
||||
|
||||
The UAT channel owns repeatable end-to-end acceptance testing against an already deployed environment. It sits after Delivery and before Releases.
|
||||
|
||||
UAT does not build application images, change manifests, synchronize Argo CD, patch workloads, or approve a release. It verifies user-visible behavior and returns evidence tied to an exact deployed revision.
|
||||
|
||||
## Required input gate
|
||||
|
||||
Do not start a run until Delivery provides:
|
||||
|
||||
- application repository and exact commit;
|
||||
- immutable OCI image digest for every tested workload;
|
||||
- deployable GitOps repository and exact commit;
|
||||
- Argo CD Application name, synchronized revision, sync state, and health state;
|
||||
- target environment and base URL;
|
||||
- deployment timestamp and relevant task/feature/release identity;
|
||||
- known limitations and required test-data preconditions.
|
||||
|
||||
If any identity is mutable, missing, or contradictory, record `BLOCKED` rather than testing an unknown deployment.
|
||||
|
||||
## Responsibilities
|
||||
|
||||
1. Maintain a dedicated pnpm E2E repository with versioned journeys.
|
||||
2. Execute journeys against the exact deployed environment supplied by Delivery.
|
||||
3. Collect machine-readable and human-reviewable evidence.
|
||||
4. Distinguish product failures, environment failures, and test defects.
|
||||
5. Return product/environment failures to Kanban and Delivery with reproducible evidence.
|
||||
6. Rerun the affected journey set after a verified redeployment.
|
||||
7. Hand a `UAT_PASSED` or `UAT_FAILED` verdict to Releases without implying release approval.
|
||||
|
||||
## UAT verdicts
|
||||
|
||||
These verdicts are acceptance dimensions; they do not replace the project's canonical Task status:
|
||||
|
||||
- `NOT_READY` — deployment evidence is incomplete.
|
||||
- `READY_FOR_UAT` — the exact deployment gate is satisfied.
|
||||
- `UAT_IN_PROGRESS` — a run is active against the frozen deployment identity.
|
||||
- `UAT_FAILED` — one or more required journeys failed.
|
||||
- `UAT_PASSED` — every required journey passed for the frozen identity.
|
||||
- `BLOCKED` — execution cannot proceed for a recorded external reason.
|
||||
|
||||
A new application image, GitOps commit, Argo CD revision, or materially changed environment invalidates an earlier verdict and requires a new run identity.
|
||||
|
||||
## Workflow
|
||||
|
||||
1. Verify the Delivery handoff and freeze the deployment identity.
|
||||
2. Select journeys by explicit tags or release scope; never silently omit a required journey.
|
||||
3. Run the repository's pnpm/Taskfile validation and Playwright suite.
|
||||
4. Preserve JUnit/JSON results, traces, screenshots on failure, browser/project matrix, start/end time, and target identity.
|
||||
5. Classify every failure and publish the smallest reproducible report.
|
||||
6. Route implementation or environment defects back to Delivery through the project's Kanban process.
|
||||
7. Rerun after Delivery supplies a new verified deployment identity.
|
||||
8. Send the final UAT verdict and evidence index to Releases.
|
||||
|
||||
## Evidence minimum
|
||||
|
||||
Every reported run includes:
|
||||
|
||||
```text
|
||||
run_id
|
||||
journey_repository + exact commit
|
||||
application_repository + exact commit
|
||||
image digests
|
||||
GitOps repository + exact commit
|
||||
Argo CD application + synchronized revision + sync/health
|
||||
base URL and environment
|
||||
journey IDs/tags and browser matrix
|
||||
started_at + completed_at
|
||||
result counts
|
||||
artifact paths/URLs
|
||||
failure classification and linked defect/task
|
||||
verdict
|
||||
```
|
||||
|
||||
Do not publish credentials, cookies, tokens, personal test data, videos containing secrets, or raw environment dumps.
|
||||
|
||||
## Authority
|
||||
|
||||
UAT may autonomously run approved journeys, collect evidence, identify reproducible defects, maintain test implementation, and issue a verdict grounded in complete evidence.
|
||||
|
||||
UAT must not modify product requirements, application behavior, deployment desired state, cluster resources, release allocation, release approval, or canonical Task lifecycle to make a test pass.
|
||||
|
||||
## References
|
||||
|
||||
- [Journey repository contract](references/journey-repository-contract.md)
|
||||
- [Deployment evidence contract](references/deployment-evidence-contract.md)
|
||||
- [Defect and retest workflow](references/defect-and-retest-workflow.md)
|
||||
- [Status and reporting](references/status-and-reporting.md)
|
||||
@@ -0,0 +1,18 @@
|
||||
# Defect and retest workflow
|
||||
|
||||
## Classification
|
||||
|
||||
- **Product defect:** deployed behavior contradicts an approved acceptance outcome or requirement.
|
||||
- **Environment defect:** routing, dependency, data, identity, or infrastructure prevents valid execution.
|
||||
- **Test defect:** the journey implementation or fixture is incorrect.
|
||||
- **Unresolved:** evidence is insufficient; investigate without assigning blame.
|
||||
|
||||
## Return packet
|
||||
|
||||
A returned failure names the journey ID, deployment tuple, first failing step/assertion, expected and actual behavior, trace/screenshot reference, reproducibility, classification, and owning Kanban/Delivery task.
|
||||
|
||||
UAT does not patch the application or desired state. Delivery supplies a new application/image/GitOps identity after remediation.
|
||||
|
||||
## Retest
|
||||
|
||||
Run the failed journey, materially coupled journeys, and the required smoke set. A narrow retest may verify the correction, but the final verdict must satisfy the release's declared journey set against one frozen deployment identity.
|
||||
@@ -0,0 +1,25 @@
|
||||
# Deployment evidence contract
|
||||
|
||||
A UAT run binds to one immutable deployment tuple:
|
||||
|
||||
```text
|
||||
application_commit
|
||||
image_digests
|
||||
GitOps_commit
|
||||
ArgoCD_application
|
||||
ArgoCD_sync_revision
|
||||
cluster_environment
|
||||
base_url
|
||||
```
|
||||
|
||||
Before execution, assert that the Argo CD application is synchronized to the declared GitOps commit and healthy. Where the environment exposes a version endpoint, compare its source/image identity with the handoff.
|
||||
|
||||
Reject the handoff when:
|
||||
|
||||
- an image tag is mutable and no digest is supplied;
|
||||
- Argo CD reports `OutOfSync`, `Unknown`, `Degraded`, or another non-healthy state;
|
||||
- the deployed source cannot be correlated to the handoff;
|
||||
- the base URL resolves to a different environment;
|
||||
- a newer synchronization occurs after the run begins.
|
||||
|
||||
If deployment identity changes during execution, end the run as invalidated and start a new run after the new identity is frozen.
|
||||
@@ -0,0 +1,29 @@
|
||||
# Journey repository contract
|
||||
|
||||
The UAT repository is a standalone pnpm project. It contains no Kubernetes/Helm/Argo CD manifests and no application implementation.
|
||||
|
||||
Required baseline:
|
||||
|
||||
```text
|
||||
journeys/
|
||||
smoke/
|
||||
regression/
|
||||
fixtures/
|
||||
helpers/
|
||||
playwright.config.ts
|
||||
package.json
|
||||
pnpm-lock.yaml
|
||||
Taskfile.yml
|
||||
.gitea/workflows/validate.yaml
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- Stable journey IDs appear in test titles and evidence.
|
||||
- `UAT_BASE_URL` is required; no production-looking default is embedded.
|
||||
- Authentication is loaded from runtime secrets and never persisted in storage-state files committed to Git.
|
||||
- Retries are explicit and low; a retry cannot turn a flaky first failure into an undisclosed pass.
|
||||
- Failure evidence includes trace and screenshot; video is opt-in when data handling permits it.
|
||||
- CI validates types, lint, journey discovery, and a non-network contract test. Environment execution is a separately authorized job.
|
||||
- Taskfile is the human/automation entry point and delegates to package scripts.
|
||||
- The repository README documents local execution, evidence locations, environment variables by name only, and failure classification.
|
||||
@@ -0,0 +1,18 @@
|
||||
# Status and reporting
|
||||
|
||||
Lead with the verdict and exact environment. Keep canonical Task flow, deployment state, UAT verdict, and release decision separate.
|
||||
|
||||
Recommended update:
|
||||
|
||||
```markdown
|
||||
## UAT result — <VERDICT>
|
||||
- Run: <run ID>
|
||||
- Environment: <name and base URL>
|
||||
- Deployment: app <commit>; images <digests>; GitOps <commit>; Argo CD <sync/health>
|
||||
- Journeys: <passed>/<failed>/<skipped>; <browser matrix>
|
||||
- Evidence: <artifact index>
|
||||
- Defects/blockers: <links or none>
|
||||
- Next action: <one owner and action>
|
||||
```
|
||||
|
||||
Never say “released,” “production-ready,” or “DONE” from a UAT pass alone. Releases owns promotion and final release status.
|
||||
Reference in New Issue
Block a user