corp-v1-3darch-skills-code-agent/template-engine-copier-3darch
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

template-engine-copier skill: initial import

Browse Source 
Copier + Jinja template-repository design skill: reference architecture,
implementation workflow, validation checklist, and a full worked example
template (questionnaire, extensions, resolvers, wrapper-first .scripts).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
root-at-skic
2026-08-12 22:30:29 +03:00
co-authored by Claude Opus 5
commit 97991021ff
29 changed files with 932 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
instructions/01-reference-architecture.md
+218
View File
@@ -0,0 +1,218 @@
# Copier + Jinja Template Engine Reference Architecture
This reference architecture is synthesized from:
- `iac-tmpl-aws-projects/`
- `iac-tmpl-aws-organizations/`
and should be treated as the default blueprint for new template repositories.
## 1) Copier core contract
Use `.copier/copier.yaml` as central control plane.
### Recommended shape
```yaml
_template_suffix: ".jinja"
_exclude:
- ".git"
- ".copier"
- ".scripts"
- ".venv"
- ".devbox"
- "devbox.json"
- "devbox.lock"
- ".gitlab-ci.yml"
- "Taskfile.yml"
- "pyproject.toml"
- "uv.lock"
- "copier-answers-new.yml"
- "copier-answers.example.yaml"
_jinja_extensions:
- copier_templates_extensions.TemplateExtensionLoader
# plus custom extension(s)
_answers_file: copier-answers.yml
!include .copier/questionnaire/*.yml
```
## 2) Questionnaire strategy
Keep questions modular under `.copier/questionnaire/`.
Why:
- supports domain separation,
- scales better than one monolithic YAML,
- makes review and versioning easier.
## 3) Context shaping strategy
For advanced templates (especially conditional directories + repeated resource modules), use custom context hook extension.
Pattern used in `iac-tmpl-aws-projects`:
- `.copier/extensions/context_hook_extension/hook.py`
- merged orchestration in `hook_function.py`
1. compute/normalize flat variables,
2. expand `[INDEX]` instance folders.
### Why `flat.*` variables
They reduce deep nested path usage in Jinja and make conditional file/folder naming concise:
- `flat.dev_is_enabled`
- `flat.dev_vm_is_enabled`
- `flat.dev_vm_instances_names`
They should also carry any normalized/safe identifiers needed by templates:
- `flat.project_safe`
- `flat.environment_safe`
- `flat.component_safe_names`
Rule: do **not** repeat normalization logic such as `lower | replace(...) | trim` throughout template paths or file bodies. If a safe identifier is needed more than once, calculate it once in the context hook (or a deterministic resolver) and expose it as a stable context variable like `flat.project_safe`.
## 4) Dynamic folder generation (`[INDEX]`)
In `iac-tmpl-aws-projects`, folders like:
```text
iac/{{ 'dev' if flat.dev_is_enabled }}/.../{{ flat.dev_vm_instances_names[INDEX] }}/
```
are expanded by context hook functions that:
1. find `[INDEX]` patterns,
2. map pattern -> context list,
3. create concrete per-instance folders,
4. replace `INDEX` placeholders in contents,
5. remove the template placeholder folder.
## 5) Custom Jinja resolvers
Keep custom Jinja helpers in `.copier/extensions/resolvers/` and register them in `_jinja_extensions`.
Examples discovered:
- `resolve_safe_tf_module_name(...)`
- `resolve_regions(...)`
- `resolve_availability_zones(...)`
- tags resolver(s) for project/organization context.
Rule: put transformation logic in extensions, not inline repeated Jinja blocks.
For safe names specifically, prefer context-hook derived variables over inline filter chains. Templates should read:
```text
applications/{{ flat.project_safe }}/...
```
not:
```text
applications/{{ project.name | lower | replace(' ', '-') | replace('_', '-') }}/...
```
## 6) Operational wrapper architecture
Use shell wrapper-first command surface:
`Taskfile -> api -> lib`
Concrete layout:
- `.scripts/copier/Taskfile.yml` - operator tasks
- `.scripts/copier/api/bootstrap-one.sh` - thin wrapper
- `.scripts/copier/lib/*.sh` - actual behavior
This keeps CLI UX stable while allowing internal evolution.
### Bootstrap from a temporary render workspace
Follow the `iac-tmpl-aws-projects` runtime pattern: the public API wrapper should not render directly from the live template repo folder. It should first create a sibling temporary copy and execute rendering from there.
Canonical flow:
1. Capture the live template folder name from `$PWD`.
2. Remove any previous sibling temp folder, e.g. `../tmp-${TEMPLATE_FOLDER}`.
3. Copy the live template folder to `../tmp-${TEMPLATE_FOLDER}`.
4. `cd` into the temp folder.
5. Load logger and copier indexes from the temp folder.
6. Run `_copier_bootstrap_one` from the temp folder.
7. Return to the live template folder after rendering.
Inside `_copier_bootstrap_one`, before invoking `copier copy`:
1. Copy `.copier/copier.yaml` to root `copier.yaml` when the template stores Copier config under `.copier/`.
2. Remove temp-copy internals that interfere with Copier local rendering:
- `.git`
- `.devbox`
- `.venv`
3. Then run `copier copy --trust --data-file ... . "$TARGET_DIR"` from the temp folder.
Why this matters:
- Copier sees a plain local template directory, not the live Git repository.
- It avoids Git/tag discovery noise such as `No git tags found in template; using HEAD as ref`.
- It prevents the render from mutating or deleting files in the live template repo.
- It keeps generated transient files (`copier.yaml`, `copier-answers-new.yml`, temp merge files) isolated in the temp copy.
Do not solve local bootstrap Git warnings by creating tags just to satisfy Copier. Tags are for intentional template releases, not for local render hygiene.
## 7) Answers merge pipeline
Use 3-step pre-render merge flow before `copier copy`:
1. ensure `copier-answers-new.yml` exists,
2. pull previous target answers (`<target>/copier-answers.yml`) if present,
3. merge explicit params file into the working answers file.
Then execute:
```bash
copier copy --trust --data-file "copier-answers-new.yml" ...
```
## 8) Generated CI file safety pattern
When template repo itself has CI files that must not be copied literally, use a temporary rendered filename:
- template file: `{{ '.tmp-gitlab-ci.yml' }}.jinja`
- post-render rename in bootstrap lib:
- `.tmp-gitlab-ci.yml` -> `.gitlab-ci.yml`
## 9) Dependency baseline
Keep Copier stack pinned in `pyproject.toml`:
- `copier`
- `copier-templates-extensions`
- supportive tooling (e.g. `check-jsonschema`, logging libs)
Use lockfile (`uv.lock`) for deterministic local behavior.
## 10) Where this is implemented in this skill
See concrete scaffold:
- `example/.copier/copier.yaml`
- `example/.copier/questionnaire/0100-template.yml`
- `example/.copier/extensions/context_hook_extension/hook.py`
- `example/.copier/extensions/resolvers/resolve_safe_identifier.py`
- `example/.scripts/copier/`
## 11) Environment variable prefix convention
Before implementing shell wrappers, **propose and confirm** the environment variable prefix with the user (e.g. `HL_V1_`, `PALANTIR_`, etc.).
Rules:
- Do not hardcode an inherited prefix from old scaffolds without confirmation.
- Keep one prefix consistently across `--env-vars-reader.sh`, bootstrap libs, and answer merge libs.
- Document the chosen prefix in README or implementation notes.
instructions/02-implementation-workflow.md
+156
View File
@@ -0,0 +1,156 @@
# Implementation Workflow (Copier + Jinja Template Repository)
Follow this sequence when creating a new template-engine repository.
## Phase 1 — Define rendering contract
1. Define template scope (what downstream repository you generate).
2. Identify stable input schema sections (e.g. tenant/project/environments/resources).
3. Decide what is:
- static files,
- conditional files/folders,
- repeatable per-instance blocks.
Deliverable: first draft of `copier-answers.example.yaml`.
## Phase 2 — Bootstrap Copier skeleton
Create:
- `.copier/copier.yaml`
- `.copier/questionnaire/*.yml`
- `{{ _copier_conf.answers_file }}.jinja`
Use baseline:
- `_template_suffix: ".jinja"`
- modular `!include .copier/questionnaire/*.yml`
- explicit `_exclude` list
- `_answers_file: copier-answers.yml`
## Phase 3 — Create operational wrappers
Implement wrapper chain:
1. root `Taskfile.yml` includes `.scripts/copier/Taskfile.yml`
2. `.scripts/copier/Taskfile.yml` exposes `bootstrap-one`
3. `.scripts/copier/api/bootstrap-one.sh` remains thin
4. **confirm ENV var prefix with user** (or propose one) before implementing lib internals
5. `.scripts/copier/lib/` contains logic and answers merge helpers
Minimum behavior in lib:
- parse CLI args with both styles: `--arg value` and `--arg=value` (for `--copier-target-dir`, `--copier-force-copy`, `--copier-params-file`),
- prepare merged answers file,
- run `copier copy --trust --data-file ...` from a sibling temporary copy of the template repo, not from the live template repo folder,
- post-render sanitization/renames if needed.
Mandatory temp-copy bootstrap pattern from `iac-tmpl-aws-projects`:
- In `api/bootstrap-one.sh`:
1. derive the live template folder name from `basename "$PWD"`,
2. delete any old sibling `../tmp-${TEMPLATE_FOLDER}`,
3. copy the whole template repo into `../tmp-${TEMPLATE_FOLDER}`,
4. `cd` into the temp folder,
5. source `.scripts/loggers/lib/--index.sh` and `.scripts/copier/lib/--index.sh`,
6. execute `_copier_bootstrap_one`,
7. return to the original live template folder.
- In `lib/-bootstrap-one.sh`, before `copier copy`:
1. copy `.copier/copier.yaml` to root `copier.yaml` if the repo uses `.copier/copier.yaml`,
2. remove `.git`, `.devbox`, and `.venv` from the temp copy,
3. run Copier from the temp folder into the real target.
Do **not** create Git tags merely to silence Copier's local template version warning. The preferred local bootstrap hygiene is rendering from the sanitized temp copy. Use `--vcs-ref=HEAD` only when there is an explicit reason to render directly from a Git-backed live folder.
## Phase 4 — Add Jinja template layers
1. Add top-level templates (`iac/main.tf.jinja`, providers, backend, etc.).
2. Gate directories via folder-name expressions:
- `{{ 'dev' if flat.dev_is_enabled }}`
3. Gate module calls inside files via `{% if ... %}`.
Pattern:
- folder condition controls existence,
- inner condition controls file content clarity.
**Mandatory wrapped render-path rule:**
- For every optional module/resource, apply conditions at the **path level** (folder/file names), not only inside file contents.
- Preferred pattern (from `iac-tmpl-aws-projects`):
- `iac/{{ 'dev' if flat.dev_is_enabled }}/{{ 'compute' if flat.dev_compute_is_enabled }}/main.tf.jinja`
- Apply this systematically to all optional Jinja outputs so disabled features do not produce placeholder files/directories.
**Mandatory safe-variable rule:**
- Do not embed repeated normalization/sanitization logic in template paths or file bodies, e.g. avoid:
- `{{ project.name | lower | replace(' ', '-') | replace('_', '-') | replace('.', '-') }}`
- If a safe name is needed, calculate it once in the context hook and expose a named variable, e.g.:
- `flat.project_safe`
- `flat.environment_safe`
- `flat.component_safe_names`
- Use the named variable in templates:
- `applications/{{ flat.project_safe }}/main.yaml.jinja`
- If a one-off transformation is genuinely needed, prefer a small deterministic resolver over duplicating filter chains.
## Phase 5 — Add custom extensions (if complexity requires)
When logic gets repetitive, move it into extensions.
### Resolver extension examples
- normalize names: `resolve_safe_tf_module_name(name)`
- derive tags / regions / zones
### Context hook extension examples
- calculate `flat.*` shortcuts,
- calculate safe/normalized names such as `flat.project_safe` and `flat.environment_safe`,
- expand `[INDEX]` directories into per-instance folders.
Register in `.copier/copier.yaml` under `_jinja_extensions`.
## Phase 6 — Implement answers merge strategy
Use this exact order to avoid destructive updates:
1. `copier-answers-new.yml` create if missing,
2. copy `<target>/copier-answers.yml` if present,
3. merge params file into working answers file (`yq eval-all ...`).
Then run copier using merged file.
## Phase 7 — Handle template/self-repo conflicts
If template repo contains files that should not be copied 1:1 (e.g. self CI):
1. render as temp name (`.tmp-gitlab-ci.yml`),
2. rename in target during bootstrap.
## Phase 8 — Add docs and examples
Add/update:
- `README.md` with local setup and bootstrap command,
- `copier-answers.example.yaml` with realistic schema examples,
- optional `bootstrap-one-test` task for local maintainer testing.
## Phase 9 — Validate end-to-end
Run two critical scenarios:
1. fresh render into empty target,
2. re-render into existing target with existing `copier-answers.yml`.
Both must succeed without manual edits.
## Quick-start from this skill example
If you want to accelerate implementation, copy `example/` and then:
1. expand questionnaire files,
2. expand `flat.*` derivation logic in context hook,
3. add domain-specific resolver extensions,
4. extend `iac/**/*.jinja` resource tree,
5. keep bootstrap/answers flow unchanged unless you have a strong reason.
instructions/03-validation-checklist.md
+75
View File
@@ -0,0 +1,75 @@
# Validation Checklist (Template Engine Repository)
Use this checklist before considering Copier/Jinja template work complete.
## A. Copier configuration
- [ ] `.copier/copier.yaml` exists and is valid YAML.
- [ ] `_template_suffix` is set to `.jinja`.
- [ ] `_answers_file` is explicitly configured.
- [ ] `_exclude` blocks repo-internal operational files (`.copier`, `.scripts`, env/lock files as needed).
- [ ] Questionnaire is loaded via `!include .copier/questionnaire/*.yml`.
## B. Input schema and answers
- [ ] `copier-answers.example.yaml` reflects real expected shape.
- [ ] Required keys used in templates are documented in the example answers.
- [ ] `{{ _copier_conf.answers_file }}.jinja` renders `_copier_answers|to_nice_yaml`.
## C. Wrapper runtime surface
- [ ] Root `Taskfile.yml` includes copier module Taskfile.
- [ ] `.scripts/copier/Taskfile.yml` has an operator-friendly `bootstrap-one` task.
- [ ] API wrapper (`api/bootstrap-one.sh`) creates and enters a sibling `tmp-<template-folder>` copy before sourcing libs and rendering.
- [ ] API wrapper returns to the live template repo after the temp-copy render completes.
- [ ] ENV variable prefix is explicitly confirmed/proposed and consistently applied across wrapper libs.
- [ ] Lib layer parses CLI args for:
- [ ] Lib layer parses CLI args for **both** forms: `--arg value` and `--arg=value`.
- [ ] `--copier-target-dir`
- [ ] `--copier-force-copy`
- [ ] `--copier-params-file`
## D. Answers merge behavior
- [ ] `copier-answers-new.yml` is created if missing.
- [ ] Existing target answers are imported when present.
- [ ] Params file is merged into working answers file.
- [ ] Render uses `--data-file` pointing to merged answers file.
- [ ] Local bootstrap render runs from the sanitized temp copy, not directly from the live Git-backed template folder.
- [ ] Temp copy removes `.git`, `.devbox`, and `.venv` before `copier copy` to avoid Copier/Git interference.
- [ ] If Copier config lives under `.copier/copier.yaml`, temp-copy bootstrap copies it to root `copier.yaml` before rendering.
- [ ] No Git tags are created merely to suppress Copier local render warnings.
## E. Templating behavior
- [ ] Conditional folder names render correctly (e.g. `{{ 'dev' if ... }}`).
- [ ] **All optional resources** are path-wrapped with Jinja conditions (folder and/or file names), not only guarded in file bodies.
- [ ] Conditional Jinja blocks in files match folder-level conditions.
- [ ] Safe/normalized names are provided by context-hook or resolver variables (e.g. `flat.project_safe`), not repeated inline filter chains.
- [ ] Template paths and file bodies do not duplicate complex expressions like `lower | replace(...) | trim` for identifiers used in more than one place.
- [ ] Any generated temporary filenames (e.g. `.tmp-gitlab-ci.yml`) are renamed correctly post-render.
## F. Extensions (if used)
- [ ] All custom extensions are registered in `_jinja_extensions`.
- [ ] Resolver helpers are deterministic and side-effect free.
- [ ] Context hook logic is idempotent on repeated runs.
- [ ] Context hook exposes stable safe variables for reusable identifiers before templates consume them.
- [ ] `[INDEX]` expansion creates concrete instance directories and removes placeholder template dir.
## G. End-to-end runs
- [ ] Fresh render into empty target directory succeeds.
- [ ] Re-render into existing target with prior answers succeeds.
- [ ] `--copier-force-copy true` path works.
- [ ] Default/non-force path works.
## H. Documentation quality
- [ ] README explains local setup and one canonical bootstrap command.
- [ ] Skill/instructions mention architecture and anti-patterns.
- [ ] Examples use realistic paths and argument names.
## Exit criteria
You are done when all checklist sections pass and rendering is reproducible in both fresh and update scenarios.