Add layered environment model and taskfile composition rules
Fold two overlapping local skills (scripts-shell, shell-scripts-module) into this one, keeping only what it did not already cover. - Add references/environment-model.md: the three-layer .env selector model, why set -a dot-sourcing beats export $(grep|xargs), tracked samples, the no-re-sourcing rule for module readers, and the deployment boundary (repo-presence is not git-tracked is not deployed). - Extend references/taskfile.md with the one-env-driven-task rule (no :default pairs) and composing commands through the task surface. - Add the lifecycle-hook module type to references/module-patterns.md. - Keep this skill's _moduleExampleV1_ naming as canonical; the retired skills' HL_V1_ and double-underscore dialects were not carried over. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -14,7 +14,8 @@ development-scripts/
|
||||
│ ├── shared-modules.md # loggers and base, and the logging bootstrap rule
|
||||
│ ├── taskfile.md # Module and root Taskfile integration
|
||||
│ ├── workflows.md # Create, change, validate
|
||||
│ └── module-patterns.md # Module type and dependency decision guide
|
||||
│ ├── module-patterns.md # Module type and dependency decision guide
|
||||
│ └── environment-model.md # Layered .env model, samples, shipping into a target repo
|
||||
└── assets/
|
||||
├── module-example-v1/ # Scaffold to copy for a new module
|
||||
├── loggers/ # Working reference implementation
|
||||
|
||||
@@ -36,6 +36,7 @@ Pick the reference that matches the change; do not load all of them.
|
||||
|---|---|
|
||||
| A reader, validator, implementation function, or `api/` wrapper | `references/module-anatomy.md` |
|
||||
| The `loggers` or `base` module, or logging that produces no output | `references/shared-modules.md` |
|
||||
| `.env` files, per-environment layers, or shipping `.scripts/` into a rendered repo | `references/environment-model.md` |
|
||||
| A module `Taskfile.yml` or the root `Taskfile.yml` | `references/taskfile.md` |
|
||||
| Creating a module, changing one, or checking work before delivery | `references/workflows.md` |
|
||||
| Deciding what kind of module something should be | `references/module-patterns.md` |
|
||||
@@ -145,7 +146,8 @@ command (`-name.sh`) or wires the module together (`--name.sh`).
|
||||
- **Every new `lib/-*.sh` gets added to `lib/--index-api.sh`.** Forgetting this is the most
|
||||
common cause of "command not found" for a function that plainly exists.
|
||||
- **Readers preserve already-exported values** — `export X="${X:-default}"` — so a caller can
|
||||
override any setting before invoking the module.
|
||||
override any setting before invoking the module. A module reader never sources `.env` files
|
||||
itself; the `base` module owns that. See `references/environment-model.md`.
|
||||
- **Log through `_loggers_*` helpers, never raw `echo`.** Loggers write to `stderr`, keeping
|
||||
`stdout` clean for real output.
|
||||
- **Never log secrets.** Redact tokens, API keys, passwords, and kube config.
|
||||
|
||||
@@ -0,0 +1,142 @@
|
||||
# Layered Environment Model
|
||||
|
||||
How a repository supplies configuration to its script modules when one repository is operated
|
||||
against several environments — a selector file plus per-environment layers, loaded once by the
|
||||
`base` module.
|
||||
|
||||
Read this when adding an environment variable, wiring a module reader to the base loader,
|
||||
introducing a second environment, deciding which file a value belongs in, or shipping
|
||||
`.scripts/` into a rendered target repository.
|
||||
|
||||
This is an **optional layer on top of** `references/shared-modules.md` §2. A repository
|
||||
operated against exactly one environment needs a single `.env` and none of this.
|
||||
|
||||
---
|
||||
|
||||
## 1. The three layers
|
||||
|
||||
Loaded in order by the base module's `--env-vars-reader.sh`, each later file overriding the
|
||||
earlier:
|
||||
|
||||
| Order | File | Holds | Tracked in git |
|
||||
|---|---|---|---|
|
||||
| 1 | `.env` | **selector only** — the environment name | no |
|
||||
| 2 | `.env-<env>` | environment-specific, non-secret configuration | no |
|
||||
| 3 | `.env-<env>-credentials` | secrets | no |
|
||||
|
||||
```bash
|
||||
<PREFIX>_ENVIRONMENT="test" # .env — the only variable this file holds
|
||||
```
|
||||
|
||||
The selector picks the other two: `.env-test` and `.env-test-credentials`. Switching
|
||||
environment is a one-line edit to `.env`, never an edit to a module.
|
||||
|
||||
The validator fails fast when the selector is unset — an unset selector would otherwise load
|
||||
neither layer and produce a cascade of "missing variable" errors pointing at the wrong cause.
|
||||
|
||||
**Which file does a value belong in?** Secret → credentials layer. Differs per environment →
|
||||
`.env-<env>`. Selects which environment → `.env`. Nothing else belongs in `.env`; a value
|
||||
parked there silently applies to every environment.
|
||||
|
||||
Repositories in the gondor family use `HL_V1_` as the prefix (`HL_V1_ENVIRONMENT`). The
|
||||
prefix is per-repository; the layering is not.
|
||||
|
||||
## 2. Load with `set -a`, not `export $(grep …)`
|
||||
|
||||
```bash
|
||||
set -a; . "<file>"; set +a
|
||||
```
|
||||
|
||||
The common alternative is broken:
|
||||
|
||||
```bash
|
||||
export $(grep -v '^#' "<file>" | xargs) # do not use
|
||||
```
|
||||
|
||||
It does not re-expand `$`, so `${HOME}/.kube/config` stays a literal string, and values
|
||||
containing quotes or spaces break apart. Dot-sourcing runs the file as shell assignments, so
|
||||
expansion, quoting, and spaces all behave.
|
||||
|
||||
The cost of this choice: env files must contain **only** `KEY="value"` assignments. They are
|
||||
executed, so a stray command in one runs.
|
||||
|
||||
## 3. Tracked samples
|
||||
|
||||
Every layer has a committed sample carrying placeholders, never real values:
|
||||
|
||||
| Real file (git-ignored) | Sample (tracked) |
|
||||
|---|---|
|
||||
| `.env` | `.sample.env` |
|
||||
| `.env-<env>` | `.sample.env-<env>` |
|
||||
| `.env-<env>-credentials` | `.sample.env-<env>-credentials` |
|
||||
|
||||
Operator flow: copy each sample to its real name, then fill in. Adding a variable to a layer
|
||||
without adding it to that layer's sample is how the next operator gets a missing-variable
|
||||
failure on a fresh clone.
|
||||
|
||||
## 4. Module readers must not re-source
|
||||
|
||||
The base module has already loaded all three layers. A module's own `--env-vars-reader.sh`:
|
||||
|
||||
- **Never sources the env files again.** A second read re-applies `.env` on top of
|
||||
`.env-<env>`, silently clobbering the environment-specific overrides it just loaded.
|
||||
- Sets defaults only when unset, and parses CLI flags that override them:
|
||||
|
||||
```bash
|
||||
export <PREFIX>_<MODULE>_TARGET="${<PREFIX>_<MODULE>_TARGET:-}" # keep any base-provided value
|
||||
```
|
||||
|
||||
This is the same rule as SKILL.md §5 "readers preserve already-exported values", applied to
|
||||
the layered case. Chain order in `lib/--index.sh` is base loader → module reader → API index.
|
||||
|
||||
Sourcing `base/lib/--index.sh` from a library function re-triggers the whole env load as a
|
||||
side effect. When you only need shared functions, source `base/lib/--index-api.sh` instead —
|
||||
see SKILL.md §8.
|
||||
|
||||
## 5. Shipping `.scripts/` into a rendered repository
|
||||
|
||||
When `.scripts/` is templated into a target repository, the env files must **land in the
|
||||
target** so its copy is operable, while real values stay out of git.
|
||||
|
||||
- **Do not exclude** `.env`, `.env-*`, or `.sample.env*` from the render. Excluding them
|
||||
produces a target whose scripts cannot run.
|
||||
- The target's `.gitignore` must ignore `.env` and `.env-*`. Real values then live locally and
|
||||
are never committed or pushed.
|
||||
- `.sample.env*` stay **tracked** — they are safe placeholders and the operator's starting
|
||||
point.
|
||||
- Exclude only genuine template-internal files from the render (for example `.copier`).
|
||||
|
||||
### Repo-presence is not deployment
|
||||
|
||||
Three distinct things get conflated:
|
||||
|
||||
| | Meaning |
|
||||
|---|---|
|
||||
| Present in the repo | the file exists on disk in the target |
|
||||
| Git-tracked | committed and pushed |
|
||||
| **Deployed** | a reconciler (for example Argo CD) applies it from a watched path |
|
||||
|
||||
**Deployed** means only the git-tracked manifests under the paths a reconciler actually
|
||||
watches. A gitignored local `.env` is present but never tracked, therefore never deployed.
|
||||
Shipping env files into a target is safe for exactly this reason — but it depends on the
|
||||
target's `.gitignore` being correct, so verify it rather than assuming.
|
||||
|
||||
Regardless of any of this, never bake secret values into git-tracked manifests. Use an
|
||||
external secrets provider.
|
||||
|
||||
### Verify exclusions on the effective layer
|
||||
|
||||
A template tool's config-file `_exclude` list (for example Copier's `.copier/copier.yaml`) is
|
||||
**inert if the tool never loads that config file**. An exclusion that appears correct in a
|
||||
config file may be doing nothing. Confirm the behaviour on the layer that actually takes
|
||||
effect — usually the CLI invocation — by rendering and inspecting the output.
|
||||
|
||||
## 6. Checklist
|
||||
|
||||
- [ ] `.env` holds the selector and nothing else.
|
||||
- [ ] Non-secret per-environment values are in `.env-<env>`; secrets in `.env-<env>-credentials`.
|
||||
- [ ] Every new variable was added to its layer's `.sample.*` file.
|
||||
- [ ] The module reader sets `:-` defaults only and does not source env files.
|
||||
- [ ] `lib/--index.sh` chains the base loader before the module reader.
|
||||
- [ ] The target repository's `.gitignore` covers `.env` and `.env-*`.
|
||||
- [ ] No secret value appears in any git-tracked file.
|
||||
@@ -13,6 +13,14 @@ others.
|
||||
| Shared foundation with public commands | Yes | Optional / no | `base` |
|
||||
| Feature module | Yes | Usually yes | `kube`, `github`, `data-migration` |
|
||||
| Wrapper around an external CLI | Yes | Usually yes | `helm`, `openshift`, `github` |
|
||||
| Lifecycle hook | Yes | No | `devbox` (an `on-init-hook`) |
|
||||
|
||||
A **lifecycle hook** module is invoked by external tooling at a fixed moment — a devbox init
|
||||
hook, a container entrypoint, a git hook — rather than by an operator. It keeps `api/` because
|
||||
the tool calls that path directly, and skips `Taskfile.yml` because nobody runs it by hand.
|
||||
Its implementation commonly branches per platform (`-install-dependencies-osx.sh`,
|
||||
`-install-dependencies-linux-ubuntu.sh`), one file per branch, sourced from `--index-api.sh`
|
||||
like any other implementation file.
|
||||
|
||||
Pick the type before creating any file — it decides whether `api/` and `Taskfile.yml` exist at
|
||||
all, and adding them later means reworking every consumer that has already started sourcing
|
||||
|
||||
@@ -41,6 +41,42 @@ task module-example-v1:initialize-one -- --name demo --dry-run
|
||||
|
||||
Without the `--`, task treats the flags as its own and the script never sees them.
|
||||
|
||||
### One env-driven task, not a `:default` pair
|
||||
|
||||
Where the repository uses the layered environment model
|
||||
(`references/environment-model.md`), the env layers already supply every value a task needs.
|
||||
A module therefore needs **one** task, not a `<task>` + `<task>:default` pair that hardcodes
|
||||
what the environment already provides:
|
||||
|
||||
```yaml
|
||||
tasks:
|
||||
initialize-one:
|
||||
desc: Initialize using the .env layers (override with --flag=…)
|
||||
cmds:
|
||||
- ./.scripts/module-example-v1/api/initialize-one.sh {{.CLI_ARGS}}
|
||||
silent: true
|
||||
```
|
||||
|
||||
A `:default` wrapper duplicates configuration in a second place, and the two drift.
|
||||
Environment-specific behaviour belongs in `.env-<env>`; one-off overrides belong in flags
|
||||
after `--`.
|
||||
|
||||
### Composing commands
|
||||
|
||||
When an operator-facing command already exists, prefer calling it through `task` rather than
|
||||
re-implementing or sourcing its internals, and pass parameters explicitly:
|
||||
|
||||
```bash
|
||||
task keycloak:realm-registration-enable-one -- --realm-id="${KEYCLOAK_ROOT_REALM_ID}"
|
||||
```
|
||||
|
||||
Composing at the task surface keeps the public contract the only contract. Reaching into
|
||||
another module's `lib/` couples you to its internals and bypasses its reader and validator.
|
||||
|
||||
Do not add a second entrypoint for an operation that already has one unless it carries clear
|
||||
operator value — duplicate entrypoints drift apart and it stops being obvious which is
|
||||
authoritative.
|
||||
|
||||
---
|
||||
|
||||
## 2. Root `Taskfile.yml` include
|
||||
|
||||
Reference in New Issue
Block a user