knowledge-wiki/meta/infrastructure.md

Infrastructure

Core Platform

• TrueNAS SCALE 26.0.0-BETA.2 at lego-cloud.eu ✅
  • Data path: /mnt/lego-cloud-v1-2025-11-28/

Services

Service	Endpoint	Status
TrueNAS SCALE	lego-cloud.eu	✅
Gitea	https://gitea.lego-cloud.eu (org: home-v1)	✅ 1.27.1; API, SSH Git, and HTTPS Git verified 2026-08-11
Keycloak	https://keycloak.lego-cloud.eu (realm: master)	⚠️ Public ingress degraded 2026-08-02
Zot Registry	port 30264, OIDC via Keycloak (client: zot)	✅
STT (Whisper)	http://192.168.1.218:39000/v1 (model: Systran/faster-whisper-base)	🔲
Piper TTS	http://192.168.1.218:39001/v1	🔲
Hermes Dashboard	https://jarvis-jr-v1.lego-cloud.eu	✅

Notes

• Zot registry uses OIDC auth through Keycloak
• STT and TTS endpoints not yet verified by this agent
• Gitea org: skic-v1-playground — knowledge-wiki repo: https://gitea.lego-cloud.eu/skic-v1-playground/knowledge-wiki
• Gitea SSH access: ssh://git@gitea.lego-cloud.eu:30009/ (key: jarvis-at-skic-hermes-access)
• Reusable Gitea operations skill: ✅ gitea-repository-operations at /opt/data/skills/software-development/gitea-repository-operations/SKILL.md; covers safe discovery, create/clone/change/push, collaborators, issues/PRs, Actions, automation, and authenticated remote verification. Wiki runbook: gitea-repository-operations.md.
• Global network repository: ✅ world-v1/network-v1 — source-backed Internet address book. IP mapping separates registry holder, route origin, operator, and service; details in network-v1.md.
• Gitea write access restored (verified 2026-07-20 15:08 UTC): after Lego requested another access check at 11:08 UTC, authenticated SSH git ls-remote succeeded and local/remote main both resolved to 35eb834. The disabled-account incident recorded at 11:06 UTC is resolved, and all queued commits reached the remote.
• Gondor v1 node access (verified 2026-08-02): reusable gondor-v1-nodes skill at /opt/data/skills/cloud-architecture/gondor-v1-nodes/SKILL.md. Hermes SSH access as lego is verified to Osgiliath 000 (192.168.148.249), Minas Tirith 010 (192.168.153.55), and Minas Tirith 020 (192.168.6.208).
• Ingress incident (open, 2026-08-02): Argo CD deployed ingress-nginx 4.11.3 as a DaemonSet with host ports 80/443 on Osgiliath, conflicting with its existing system Nginx. Kubernetes ingress intercepts legacy public hosts, returning the fake ingress.local certificate/default 404; argo-workflows.apps.lego-cloud.eu works and Gitea's internal upstream is healthy. Recommended durable repair is in GitOps (home-v1/gondor-v1.git, cluster/0500-resources/0000-ingress-nginx/ingress-nginx/values.yml): keep system Nginx as Osgiliath's public ingress, schedule Kubernetes ingress on workers, and proxy required app routes. Exact owner, due date, and final approved implementation remain UNKNOWN.
• Hermes multi-day outage (2026-08-04–11): read-only analysis separated two failures. Codex quota exhaustion is the confirmed cause of failed model responses while Discord/cron still ran. After the final preserved event at 2026-08-04 14:00:42 UTC, gateway, cron, logs, and health activity all ceased until the physical host restart at 2026-08-11 00:11:40 UTC; restart restored service without a Hermes repair. A host/container-runtime availability failure is most likely, but the exact TrueNAS Apps, kernel, ZFS/storage, hardware, watchdog, thermal, or power trigger is UNKNOWN because previous-boot host telemetry is unavailable inside the container. Current SQLite integrity and capacity checks are healthy.
• Gitea Git-transport incident resolved (2026-08-11): Gitea 1.27.0-rootless served web/API but crashed its Git-serving process during Go runtime module verification over both SSH and Smart HTTP. Upgrade to 1.27.1 restored API and both Git transports. The queued VSSA commit 6fc42ab was then pushed and remotely verified.
• Gitea Pages publication contract (verified 2026-08-11): Gitea Actions job containers receive /mnt/lego-cloud-v1-2025-11-28/gondor-v1-gitea-pages at /gondor-v1-gitea-pages; workflows publish to /gondor-v1-gitea-pages/<organization>/<repository>/, served at https://pages.apps.lego-cloud.eu/<organization>/<repository>/. The runner source path must be listed under container.valid_volumes. skic-v1-playground/documentation run 2326 and its public site were verified successful/HTTP 200.