diff --git a/README.md b/README.md index 346f6d8..75fd1d7 100644 --- a/README.md +++ b/README.md @@ -14,7 +14,8 @@ development-scripts/ │ ├── shared-modules.md # loggers and base, and the logging bootstrap rule │ ├── taskfile.md # Module and root Taskfile integration │ ├── workflows.md # Create, change, validate -│ └── module-patterns.md # Module type and dependency decision guide +│ ├── module-patterns.md # Module type and dependency decision guide +│ └── environment-model.md # Layered .env model, samples, shipping into a target repo └── assets/ ├── module-example-v1/ # Scaffold to copy for a new module ├── loggers/ # Working reference implementation diff --git a/SKILL.md b/SKILL.md index 6b4d397..3ee0f4f 100644 --- a/SKILL.md +++ b/SKILL.md @@ -36,6 +36,7 @@ Pick the reference that matches the change; do not load all of them. |---|---| | A reader, validator, implementation function, or `api/` wrapper | `references/module-anatomy.md` | | The `loggers` or `base` module, or logging that produces no output | `references/shared-modules.md` | +| `.env` files, per-environment layers, or shipping `.scripts/` into a rendered repo | `references/environment-model.md` | | A module `Taskfile.yml` or the root `Taskfile.yml` | `references/taskfile.md` | | Creating a module, changing one, or checking work before delivery | `references/workflows.md` | | Deciding what kind of module something should be | `references/module-patterns.md` | @@ -145,7 +146,8 @@ command (`-name.sh`) or wires the module together (`--name.sh`). - **Every new `lib/-*.sh` gets added to `lib/--index-api.sh`.** Forgetting this is the most common cause of "command not found" for a function that plainly exists. - **Readers preserve already-exported values** — `export X="${X:-default}"` — so a caller can - override any setting before invoking the module. + override any setting before invoking the module. A module reader never sources `.env` files + itself; the `base` module owns that. See `references/environment-model.md`. - **Log through `_loggers_*` helpers, never raw `echo`.** Loggers write to `stderr`, keeping `stdout` clean for real output. - **Never log secrets.** Redact tokens, API keys, passwords, and kube config. diff --git a/references/environment-model.md b/references/environment-model.md new file mode 100644 index 0000000..5195c55 --- /dev/null +++ b/references/environment-model.md @@ -0,0 +1,142 @@ +# Layered Environment Model + +How a repository supplies configuration to its script modules when one repository is operated +against several environments — a selector file plus per-environment layers, loaded once by the +`base` module. + +Read this when adding an environment variable, wiring a module reader to the base loader, +introducing a second environment, deciding which file a value belongs in, or shipping +`.scripts/` into a rendered target repository. + +This is an **optional layer on top of** `references/shared-modules.md` §2. A repository +operated against exactly one environment needs a single `.env` and none of this. + +--- + +## 1. The three layers + +Loaded in order by the base module's `--env-vars-reader.sh`, each later file overriding the +earlier: + +| Order | File | Holds | Tracked in git | +|---|---|---|---| +| 1 | `.env` | **selector only** — the environment name | no | +| 2 | `.env-` | environment-specific, non-secret configuration | no | +| 3 | `.env--credentials` | secrets | no | + +```bash +_ENVIRONMENT="test" # .env — the only variable this file holds +``` + +The selector picks the other two: `.env-test` and `.env-test-credentials`. Switching +environment is a one-line edit to `.env`, never an edit to a module. + +The validator fails fast when the selector is unset — an unset selector would otherwise load +neither layer and produce a cascade of "missing variable" errors pointing at the wrong cause. + +**Which file does a value belong in?** Secret → credentials layer. Differs per environment → +`.env-`. Selects which environment → `.env`. Nothing else belongs in `.env`; a value +parked there silently applies to every environment. + +Repositories in the gondor family use `HL_V1_` as the prefix (`HL_V1_ENVIRONMENT`). The +prefix is per-repository; the layering is not. + +## 2. Load with `set -a`, not `export $(grep …)` + +```bash +set -a; . ""; set +a +``` + +The common alternative is broken: + +```bash +export $(grep -v '^#' "" | xargs) # do not use +``` + +It does not re-expand `$`, so `${HOME}/.kube/config` stays a literal string, and values +containing quotes or spaces break apart. Dot-sourcing runs the file as shell assignments, so +expansion, quoting, and spaces all behave. + +The cost of this choice: env files must contain **only** `KEY="value"` assignments. They are +executed, so a stray command in one runs. + +## 3. Tracked samples + +Every layer has a committed sample carrying placeholders, never real values: + +| Real file (git-ignored) | Sample (tracked) | +|---|---| +| `.env` | `.sample.env` | +| `.env-` | `.sample.env-` | +| `.env--credentials` | `.sample.env--credentials` | + +Operator flow: copy each sample to its real name, then fill in. Adding a variable to a layer +without adding it to that layer's sample is how the next operator gets a missing-variable +failure on a fresh clone. + +## 4. Module readers must not re-source + +The base module has already loaded all three layers. A module's own `--env-vars-reader.sh`: + +- **Never sources the env files again.** A second read re-applies `.env` on top of + `.env-`, silently clobbering the environment-specific overrides it just loaded. +- Sets defaults only when unset, and parses CLI flags that override them: + +```bash +export __TARGET="${__TARGET:-}" # keep any base-provided value +``` + +This is the same rule as SKILL.md §5 "readers preserve already-exported values", applied to +the layered case. Chain order in `lib/--index.sh` is base loader → module reader → API index. + +Sourcing `base/lib/--index.sh` from a library function re-triggers the whole env load as a +side effect. When you only need shared functions, source `base/lib/--index-api.sh` instead — +see SKILL.md §8. + +## 5. Shipping `.scripts/` into a rendered repository + +When `.scripts/` is templated into a target repository, the env files must **land in the +target** so its copy is operable, while real values stay out of git. + +- **Do not exclude** `.env`, `.env-*`, or `.sample.env*` from the render. Excluding them + produces a target whose scripts cannot run. +- The target's `.gitignore` must ignore `.env` and `.env-*`. Real values then live locally and + are never committed or pushed. +- `.sample.env*` stay **tracked** — they are safe placeholders and the operator's starting + point. +- Exclude only genuine template-internal files from the render (for example `.copier`). + +### Repo-presence is not deployment + +Three distinct things get conflated: + +| | Meaning | +|---|---| +| Present in the repo | the file exists on disk in the target | +| Git-tracked | committed and pushed | +| **Deployed** | a reconciler (for example Argo CD) applies it from a watched path | + +**Deployed** means only the git-tracked manifests under the paths a reconciler actually +watches. A gitignored local `.env` is present but never tracked, therefore never deployed. +Shipping env files into a target is safe for exactly this reason — but it depends on the +target's `.gitignore` being correct, so verify it rather than assuming. + +Regardless of any of this, never bake secret values into git-tracked manifests. Use an +external secrets provider. + +### Verify exclusions on the effective layer + +A template tool's config-file `_exclude` list (for example Copier's `.copier/copier.yaml`) is +**inert if the tool never loads that config file**. An exclusion that appears correct in a +config file may be doing nothing. Confirm the behaviour on the layer that actually takes +effect — usually the CLI invocation — by rendering and inspecting the output. + +## 6. Checklist + +- [ ] `.env` holds the selector and nothing else. +- [ ] Non-secret per-environment values are in `.env-`; secrets in `.env--credentials`. +- [ ] Every new variable was added to its layer's `.sample.*` file. +- [ ] The module reader sets `:-` defaults only and does not source env files. +- [ ] `lib/--index.sh` chains the base loader before the module reader. +- [ ] The target repository's `.gitignore` covers `.env` and `.env-*`. +- [ ] No secret value appears in any git-tracked file. diff --git a/references/module-patterns.md b/references/module-patterns.md index 21ee06f..eb7d61d 100644 --- a/references/module-patterns.md +++ b/references/module-patterns.md @@ -13,6 +13,14 @@ others. | Shared foundation with public commands | Yes | Optional / no | `base` | | Feature module | Yes | Usually yes | `kube`, `github`, `data-migration` | | Wrapper around an external CLI | Yes | Usually yes | `helm`, `openshift`, `github` | +| Lifecycle hook | Yes | No | `devbox` (an `on-init-hook`) | + +A **lifecycle hook** module is invoked by external tooling at a fixed moment — a devbox init +hook, a container entrypoint, a git hook — rather than by an operator. It keeps `api/` because +the tool calls that path directly, and skips `Taskfile.yml` because nobody runs it by hand. +Its implementation commonly branches per platform (`-install-dependencies-osx.sh`, +`-install-dependencies-linux-ubuntu.sh`), one file per branch, sourced from `--index-api.sh` +like any other implementation file. Pick the type before creating any file — it decides whether `api/` and `Taskfile.yml` exist at all, and adding them later means reworking every consumer that has already started sourcing diff --git a/references/taskfile.md b/references/taskfile.md index abb77c3..b6cb2e9 100644 --- a/references/taskfile.md +++ b/references/taskfile.md @@ -41,6 +41,42 @@ task module-example-v1:initialize-one -- --name demo --dry-run Without the `--`, task treats the flags as its own and the script never sees them. +### One env-driven task, not a `:default` pair + +Where the repository uses the layered environment model +(`references/environment-model.md`), the env layers already supply every value a task needs. +A module therefore needs **one** task, not a `` + `:default` pair that hardcodes +what the environment already provides: + +```yaml +tasks: + initialize-one: + desc: Initialize using the .env layers (override with --flag=…) + cmds: + - ./.scripts/module-example-v1/api/initialize-one.sh {{.CLI_ARGS}} + silent: true +``` + +A `:default` wrapper duplicates configuration in a second place, and the two drift. +Environment-specific behaviour belongs in `.env-`; one-off overrides belong in flags +after `--`. + +### Composing commands + +When an operator-facing command already exists, prefer calling it through `task` rather than +re-implementing or sourcing its internals, and pass parameters explicitly: + +```bash +task keycloak:realm-registration-enable-one -- --realm-id="${KEYCLOAK_ROOT_REALM_ID}" +``` + +Composing at the task surface keeps the public contract the only contract. Reaching into +another module's `lib/` couples you to its internals and bypasses its reader and validator. + +Do not add a second entrypoint for an operation that already has one unless it carries clear +operator value — duplicate entrypoints drift apart and it stops being obvious which is +authoritative. + --- ## 2. Root `Taskfile.yml` include