development-gitops-argo-cd-3darch/references/copier-templates.md

# Copier Templating Layer

The `copier.yaml` variable schema, `.jinja` template authoring, conditional directory names,
and the `render()` macro. Read this when adding a variable, editing any `.jinja` file, or
debugging a rendering failure.

---

## 1. How Copier renders this repo

The repository **is** a Copier template. `copier copy` (or `copier update`) against an answer
file causes Copier to:

1. Read every variable definition from `copier.yaml`.
2. Substitute values from `copier-answers.yaml` into every `*.jinja` file.
3. Evaluate Jinja2 expressions used as **directory names** — a falsy result omits the whole
   subtree from the output.
4. Write rendered files without the `.jinja` extension to the destination.

The rendered output is what ArgoCD reads from Git. The template source is what you edit.
These are different trees; never confuse a rendering bug with a cluster bug.

---

## 2. Conditional directory names

```text
{{ '210-eso-operator-group' if operators.eso_is_enabled }}
```

- Flag `true` → directory renders as `210-eso-operator-group/`.
- Flag `false` → the directory **and everything inside it** is omitted.

This is the feature-flag mechanism for the entire repository. Consequences:

- No `{% if %}` blocks are needed inside manifests, and none should be added.
- The expression must evaluate to exactly the intended name string when truthy, and to a
  falsy value when not. `{{ '210-x' if flag else '' }}` and `{{ '210-x' if flag }}` both work;
  keep the shorter form for consistency.
- The variable named in the expression must be the one declared in `copier.yaml` — a typo
  produces `UndefinedError` or, worse, a silently falsy value that drops the resource.
- `cluster/argocd/` and `cluster/resources/` must use **identical** expressions. Renaming a
  flag means editing both.

---

## 3. The `render()` macro pattern

Every `values.yaml.jinja` wraps its content in a macro and strips blank lines:

```jinja
{% macro render() %}
key: {{ variable }}
other_key: {{ other_variable }}
{% endmacro %}
{%- set result = render() -%}
{%- for line in result.split('\n') -%}
{% if line | trim | length != 0 -%}
{{ line }}
{% endif -%}
{%- endfor %}
```

Mandatory for all `values.yaml.jinja` files. Unexpanded Jinja2 constructs leave blank lines
in the output; in YAML those can break block structure and produce a parse error at sync
time — long after the commit that caused it.

Do not try to solve the same problem with Jinja2 whitespace control (`{%-`/`-%}`) inside the
macro body. The footer loop is the agreed mechanism; mixing approaches makes the output
depend on which one wins.

---

## 4. Variable schema (`copier.yaml`)

Each variable declares:

- `type` — `str`, `bool`, `dict`, `yaml`
- `default` — used when no answer file supplies a value
- `help` — shown during interactive prompting

Naming conventions:

| Pattern | Purpose |
|---|---|
| `<domain>.is_enabled` | Top-level flag for a whole domain (`operators.is_enabled`) |
| `<domain>.<sub>_is_enabled` | Sub-feature flag (`operators.eso_is_enabled`) |
| `<domain>.<sub>.image.address` | Container image repository path |
| `<domain>.<sub>.image.tag` | Container image tag |
| `<domain>.<sub>.env.<var>` | Application environment variable |
| `gitops.repo_url` | Git URL used in ArgoCD sources |
| `gitops.branch` | Git branch used in ArgoCD sources |

Rules:

- Declare every variable before any template references it — an undeclared reference fails
  the render for every environment.
- Give every variable a sensible `default` so `--defaults` rendering works without an answer
  file.
- Feature flags are `type: bool`, never `str`.
- New flags default to `false` — resources are opt-in.
- Group variables under a domain object (`operators:`, `applications:`). Never add isolated
  top-level variables.

---

## 5. Accessing variables in templates

```jinja
{# Direct — raises UndefinedError if the key is missing #}
{{ operators.eso_is_enabled }}

{# Safe with default — preferred for any optional nested key #}
{{ applications.get('integration', {}).get('replicas', 1) }}
```

Use the `.get()` chain for anything that might be absent in some environment. A direct
reference to an optional key is a latent failure: it renders fine in the environment you
tested and breaks the one you did not.

Defaults inside `.get()` should match the field's type — `''` for strings, a number for
numeric fields.

---

## 6. Adding a new variable

1. Open `copier.yaml`.
2. Add it under the correct domain key:

   ```yaml
   <domain>:
     <descriptor>_is_enabled:
       type: bool
       default: false
       help: "Enable <NNN>-<name>"

     <descriptor>:
       image:
         address:
           type: str
           default: ""
           help: "Container image for <name>"
         tag:
           type: str
           default: ""
           help: "Container image tag for <name>"
   ```

3. Reference it from the appropriate `.jinja` template.
4. Add its environment-specific value to each relevant `copier-answers.yaml`.
5. Render and validate.

---

## 7. Authoring rules

- Every new `.jinja` values file uses the `render()` macro pattern.
- Keep Jinja2 logic minimal — directories handle feature flags, templates handle substitution.
- No credentials, tokens, or secrets inside any template.
- Answer files stay YAML; never `.json` or `.env`.
- `_src_path` and `_commit` in answer files are Copier metadata — never hand-edited.

---

## 8. Debugging a render failure

```bash
# Parse every template without rendering — fastest way to find a syntax error
python3 -c "
from jinja2 import Environment
import pathlib
env = Environment()
for f in pathlib.Path('cluster').rglob('*.jinja'):
    env.parse(f.read_text()); print('OK:', f)
"

# Full render with defaults
copier copy . /tmp/gitops-render-defaults --overwrite --defaults

# Full render with a specific environment
copier copy . /tmp/gitops-render-env --overwrite --answers-file <env>/copier-answers.yaml

# No .jinja may survive into the output
find /tmp/gitops-render-env -name "*.jinja" && echo "ERROR: .jinja files in output"
```

| Error | Cause |
|---|---|
| `UndefinedError: '<x>' is undefined` | Variable not declared in `copier.yaml`, or a direct reference to an optional nested key — switch to `.get()` |
| Directory missing from output | Flag false, or a typo in the variable name inside the conditional expression |
| `.jinja` file in the output | File is inside a directory Copier excluded, or the extension is misspelled |
| Rendered YAML fails to parse | Missing `render()` stripping footer leaving blank lines |
| `copier update` refuses to run | `_src_path` or `_commit` was hand-edited |

---

## 9. Mistakes to avoid

- Skipping the `render()` blank-line-stripping wrapper.
- Using string `"true"`/`"false"` for `bool` feature flags.
- Adding isolated top-level variables instead of grouping under a domain.
- Renaming a flag on only one of `cluster/argocd/` and `cluster/resources/`.
- Direct `{{ variable }}` access for optional nested keys instead of `.get()`.
- Committing an answer file containing real secrets.