Channel Wiki: #cogarch-v1-argo-workflows

Channel ID: 1548051220533481536 Created: 2026-09-11 UTC Last sync: 2026-09-17 03:13 UTC

Purpose

Observe, analyze, and preserve evidence about Cognitive Architect Solution Factory behavior and workflows. Observation is read-only by default.

Canonical Resources

Channel workspace: /opt/data/cogarch-v1-argo-workflows/
Channel skill repository: https://gitea.lego-cloud.eu/cognitive-architect-v1-skills-code-agent/cogarch-v1-channel-argo-workflows
IBM documentation repository, when relevant: CTOTools/Documentation on github.ibm.com

Key Decisions

Channel-owned files belong in the dedicated local workspace.
Reusable operating rules belong in the dedicated channel skill; mutable continuity belongs in this wiki.
Observation does not authorize upstream edits, external publication, or IBM merges. IBM merges require human approval.
Hard authority boundary: Hermes must perform no write action against Cognitive Architect Argo Workflows in any environment unless Discord user 1518725627845283888 explicitly overrides this restriction.
Production Argo Workflows access is read-only through the dedicated channel-skill helper. Its API token must never enter model context, command arguments, logs, or artifacts.
BYOT MVP decision (2026-09-16): use one persistent OpenShift Secret per user and do not introduce IBM Secrets Manager or another internal secrets service for the initial version. The existing settings backend creates, patches, and deletes the Secret; a trusted Solution Factory WorkflowTemplate selects it for the user's workflow pod.
BYOT Secret names are server-derived from opaque internal user identifiers. The browser and workflow parameters must not choose Secret names.
The settings backend uses a dedicated OpenShift ServiceAccount whose Role allows only create, patch, and delete on Secrets, with no get, list, or watch. Because native RBAC cannot constrain creates by name prefix or label, a ValidatingAdmissionPolicy must restrict that identity to correctly labelled byot-* Opaque Secrets containing only the approved token key.
Users must not be able to supply arbitrary Workflow YAML, Secret references, volumes, service accounts, or pod patches. Worker pods should not mount Kubernetes API tokens unless required, and BYOT values must never enter Argo parameters, logs, errors, annotations, command arguments, traces, or artifacts.
BYOT fails closed when selected but unavailable; it must not silently fall back to the shared credential pool. Already-running workflows may retain an old token after rotation/deletion, and deleting Solution Factory's copy does not revoke the credential at the provider. Initial disaster recovery may use encrypted OpenShift backup or require user re-entry.

Active Topics

Investigate production Solution Factory workflow-template behavior and failures through GET-only Argo access.
Implement and validate the approved per-user OpenShift Secret BYOT MVP, including write-only backend RBAC, admission restriction, trusted WorkflowTemplates, no secret leakage, fail-closed behavior, and an explicit cluster-loss recovery choice.

Key Context

Use live source inspection for current-state claims.
Load cognitive-architect for architecture/model work and cogarch-github for IBM CTOTools Documentation operations.
The channel skill retrieves the Argo API token and Swagger URL by exact Bitwarden IDs inside a subprocess, permits only Swagger-declared GET operations, and sanitizes generic output.
Live verification on 2026-09-11 identified Argo Workflows v4.0.5 and service-account namespace solution-factory.
The production namespace exposes six namespaced WorkflowTemplates: one four-phase coordinator, four generation/refinement phases, and a separate propagation phase. No CronWorkflows were listed; cluster-scoped template inventory is inaccessible to the read-only service account, so its existence is unknown.
For architecture arch_up7BoYhj- and iteration iter_QfJ3shQvZ, no Workflow object exists in the live or archived lists. An Argo GET label selector using that architecture ID returns HTTP 400 because Kubernetes label values cannot end with -; normal Solution Factory submissions copy architecture-id into Workflow metadata labels. This is strong evidence that submission is rejected before the Processing workflow is created, not that a Processing pod starts and fails.
Live inspection on 2026-09-13 found no active or archived Workflow labeled architecture-id=arch_kiSj9jBKI; the daily inventory also contained no record for it from 2026-09-06 through 2026-09-13. The current Processing template does include the intended image path: conversion-to-markdown → markdown-chunking → describe-images → merge-descriptions. Processing Workflows have ttlStrategy.secondsAfterCompletion: 86400, while the accessible archived list is empty, so Argo logs disappear with the Workflow after roughly 24 hours unless retained elsewhere. For a recent iteration, absence strongly suggests that the Processing Workflow was never created; for an older iteration, Argo alone cannot distinguish non-submission from TTL cleanup.
Live inspection on 2026-09-15 for architecture arch_xOQ5jks0b, iteration iter_fMfOM35fb, found Processing succeeded (phase-0100-processing-sn74b, 6/6) but High-Level Insights failed (phase-0200-high-level-insights-ffmgf, 1/2). The first high-level step, 0200-phase-0600-step-generate-knowledge-wiki-plan, launched the Cline/code agent but it exited with code 201 on the initial attempt and all three error retries. The original centralized agent log reportedly showed an operation timeout and retry-limit failure, but a rerun at 2026-09-15 12:02:57 UTC overwrote that file. A Redis DNS warning (redis:6379 unresolved) appeared only in the failed run's Argo main-container stdout, was handled fail-open, and was not causal. Fresh read-only inspection of rerun pod phase-0200-high-level-insights-9p7vz-run-step-2817085442 found 17 log entries, zero Redis references/errors, and normal one-key credential-pool initialization.

People & Roles

RootAtSkic (1518725627845283888) — channel owner and human authority.
whyt6249 (204689314796273674) — supplied the overwritten agent-log finding and identified the rerun pod for comparison.
Hermes (1526860732917088266) — assisting bot.

Action Items

Select the first Solution Factory behavior or workflow to observe.
✅ The requested standalone HTML investigation page was delivered in Discord messages 1548064910263586922–1548064913602379868.
Obtain the exact iter_... ID and approximate start time for arch_kiSj9jBKI to determine whether it should remain inside Argo's 24-hour retention window and to correlate any external S3 or Solution Factory history.
Validate the BYOT MVP end to end: create/rotate/delete only the authenticated user's Secret; prove the backend cannot read Secrets or mutate unrelated ones; prove workflows cannot select another user's Secret; verify no token leakage; and choose encrypted backup versus user re-entry after catastrophic cluster loss.

Evidence

1548051382714507325 — RootAtSkic requested a local folder, dedicated channel skill, and Gitea publication.
Previous processed human message: 1548759488570466527 (2026-09-13 18:17 UTC; RootAtSkic confirmed that the lookup must use the exact Kubernetes selector architecture-id=arch_kiSj9jBKI; live and archived Argo queries using that selector returned no items).
1549364076239397005 — RootAtSkic requested diagnosis of failed High-Level Insights for arch_xOQ5jks0b / iter_fMfOM35fb; read-only Argo inspection isolated the failure to knowledge-wiki-plan agent exit code 201 after retries.
Latest processed human message: 1549399634302992447 (2026-09-15 12:40 UTC; whyt6249 requested comparison against rerun pod phase-0200-high-level-insights-9p7vz-run-step-2817085442; live read-only inspection found no recurrence of the Redis warning).
1549783805877813279 — whyt6249 accepted the direct per-user OpenShift Secret approach as the BYOT decision and requested its summary after exploring IBM Secrets Manager, encrypted-database, broker, Vault/OpenBao, and direct-cluster alternatives.
Latest processed human message: 1549783805877813279 (2026-09-16 14:07 UTC; requested summarizing the agreed BYOT decision).

8.3 KiB Raw Blame History Unescape Escape