template-engine-copier-3darch/instructions/01-reference-architecture.md

Copier + Jinja Template Engine Reference Architecture

This reference architecture is synthesized from:

• iac-tmpl-aws-projects/
• iac-tmpl-aws-organizations/

and should be treated as the default blueprint for new template repositories.

1) Copier core contract

Use .copier/copier.yaml as central control plane.

Recommended shape

_template_suffix: ".jinja"

_exclude:
  - ".git"
  - ".copier"
  - ".scripts"
  - ".venv"
  - ".devbox"
  - "devbox.json"
  - "devbox.lock"
  - ".gitlab-ci.yml"
  - "Taskfile.yml"
  - "pyproject.toml"
  - "uv.lock"
  - "copier-answers-new.yml"
  - "copier-answers.example.yaml"

_jinja_extensions:
  - copier_templates_extensions.TemplateExtensionLoader
  # plus custom extension(s)

_answers_file: copier-answers.yml

!include .copier/questionnaire/*.yml

2) Questionnaire strategy

Keep questions modular under .copier/questionnaire/.

Why:

• supports domain separation,
• scales better than one monolithic YAML,
• makes review and versioning easier.

3) Context shaping strategy

For advanced templates (especially conditional directories + repeated resource modules), use custom context hook extension.

Pattern used in iac-tmpl-aws-projects:

• .copier/extensions/context_hook_extension/hook.py
• merged orchestration in hook_function.py
  1. compute/normalize flat variables,
  2. expand [INDEX] instance folders.

Why flat.* variables

They reduce deep nested path usage in Jinja and make conditional file/folder naming concise:

• flat.dev_is_enabled
• flat.dev_vm_is_enabled
• flat.dev_vm_instances_names

They should also carry any normalized/safe identifiers needed by templates:

• flat.project_safe
• flat.environment_safe
• flat.component_safe_names

Rule: do not repeat normalization logic such as lower | replace(...) | trim throughout template paths or file bodies. If a safe identifier is needed more than once, calculate it once in the context hook (or a deterministic resolver) and expose it as a stable context variable like flat.project_safe.

4) Dynamic folder generation ([INDEX])

In iac-tmpl-aws-projects, folders like:

iac/{{ 'dev' if flat.dev_is_enabled }}/.../{{ flat.dev_vm_instances_names[INDEX] }}/

are expanded by context hook functions that:

1. find [INDEX] patterns,
2. map pattern -> context list,
3. create concrete per-instance folders,
4. replace INDEX placeholders in contents,
5. remove the template placeholder folder.

5) Custom Jinja resolvers

Keep custom Jinja helpers in .copier/extensions/resolvers/ and register them in _jinja_extensions.

Examples discovered:

• resolve_safe_tf_module_name(...)
• resolve_regions(...)
• resolve_availability_zones(...)
• tags resolver(s) for project/organization context.

Rule: put transformation logic in extensions, not inline repeated Jinja blocks.

For safe names specifically, prefer context-hook derived variables over inline filter chains. Templates should read:

applications/{{ flat.project_safe }}/...

not:

applications/{{ project.name | lower | replace(' ', '-') | replace('_', '-') }}/...

6) Operational wrapper architecture

Use shell wrapper-first command surface:

Taskfile -> api -> lib

Concrete layout:

• .scripts/copier/Taskfile.yml - operator tasks
• .scripts/copier/api/bootstrap-one.sh - thin wrapper
• .scripts/copier/lib/*.sh - actual behavior

This keeps CLI UX stable while allowing internal evolution.

Bootstrap from a temporary render workspace

Follow the iac-tmpl-aws-projects runtime pattern: the public API wrapper should not render directly from the live template repo folder. It should first create a sibling temporary copy and execute rendering from there.

Canonical flow:

1. Capture the live template folder name from $PWD.
2. Remove any previous sibling temp folder, e.g. ../tmp-${TEMPLATE_FOLDER}.
3. Copy the live template folder to ../tmp-${TEMPLATE_FOLDER}.
4. cd into the temp folder.
5. Load logger and copier indexes from the temp folder.
6. Run _copier_bootstrap_one from the temp folder.
7. Return to the live template folder after rendering.

Inside _copier_bootstrap_one, before invoking copier copy:

1. Copy .copier/copier.yaml to root copier.yaml when the template stores Copier config under .copier/.
2. Remove temp-copy internals that interfere with Copier local rendering:
   • .git
   • .devbox
   • .venv
3. Then run copier copy --trust --data-file ... . "$TARGET_DIR" from the temp folder.

Why this matters:

• Copier sees a plain local template directory, not the live Git repository.
• It avoids Git/tag discovery noise such as No git tags found in template; using HEAD as ref.
• It prevents the render from mutating or deleting files in the live template repo.
• It keeps generated transient files (copier.yaml, copier-answers-new.yml, temp merge files) isolated in the temp copy.

Do not solve local bootstrap Git warnings by creating tags just to satisfy Copier. Tags are for intentional template releases, not for local render hygiene.

7) Answers merge pipeline

Use 3-step pre-render merge flow before copier copy:

1. ensure copier-answers-new.yml exists,
2. pull previous target answers (<target>/copier-answers.yml) if present,
3. merge explicit params file into the working answers file.

Then execute:

copier copy --trust --data-file "copier-answers-new.yml" ...

8) Generated CI file safety pattern

When template repo itself has CI files that must not be copied literally, use a temporary rendered filename:

• template file: {{ '.tmp-gitlab-ci.yml' }}.jinja
• post-render rename in bootstrap lib:
  • .tmp-gitlab-ci.yml -> .gitlab-ci.yml

9) Dependency baseline

Keep Copier stack pinned in pyproject.toml:

• copier
• copier-templates-extensions
• supportive tooling (e.g. check-jsonschema, logging libs)

Use lockfile (uv.lock) for deterministic local behavior.

10) Where this is implemented in this skill

See concrete scaffold:

• example/.copier/copier.yaml
• example/.copier/questionnaire/0100-template.yml
• example/.copier/extensions/context_hook_extension/hook.py
• example/.copier/extensions/resolvers/resolve_safe_identifier.py
• example/.scripts/copier/

11) Environment variable prefix convention

Before implementing shell wrappers, propose and confirm the environment variable prefix with the user (e.g. HL_V1_, PALANTIR_, etc.).

Rules:

• Do not hardcode an inherited prefix from old scaffolds without confirmation.
• Keep one prefix consistently across --env-vars-reader.sh, bootstrap libs, and answer merge libs.
• Document the chosen prefix in README or implementation notes.