development-gitops-argo-cd-3darch/references/environments.md

# Environment Configuration

Per-environment `copier-answers.yaml` files: feature flags, image tags, global application
settings, drift between environments, and re-rendering with `copier update`.

Read this when enabling or disabling a resource for one environment, onboarding a new
environment, auditing what is switched on, or investigating why two environments differ.

---

## 1. How environments work

Each deployed environment is a rendered copy of the template repository. The rendering is
driven by an answer file supplying environment-specific values for the variables declared in
`copier.yaml`:

```text
Template repo
    + <env>/copier-answers.yaml
    ─────────────────────────────
    → rendered cluster state (what ArgoCD reads)
```

The rendered state normally lives on a separate branch or repository per environment. The
answer files themselves live at `<env>/copier-answers.yaml` in the template repo.

---

## 2. Answer file structure

Copier maintains two metadata fields automatically; everything else is user-defined:

```yaml
_src_path: ../                    # relative path to the template source
_commit: <git-commit-sha>         # template commit used for the last render

gitops:
  repo_url: https://github.com/org/repo
  branch: main

operators:
  is_enabled: true
  eso_is_enabled: true
  cnpg_is_enabled: false
  redis_is_enabled: false

configuration:
  is_enabled: true
  cluster_secret_store_is_enabled: true

applications:
  is_enabled: true
  global:
    is_internal: "false"
    ca_env: production
    host: myapp.example.com
```

Keys must match `copier.yaml` exactly. Copier **silently ignores unknown keys** — a typo does
not error, it just leaves the variable at its default, which is the most common cause of a
"why is this environment misconfigured" investigation.

---

## 3. Variable categories

### Feature flags

Flags follow `<domain>.is_enabled` and `<domain>.<sub>_is_enabled`:

| Variable | Effect when true |
|---|---|
| `operators.is_enabled` | Renders the `200-operators` band |
| `operators.eso_is_enabled` | Renders ESO operator group, subscription, configuration |
| `operators.cnpg_is_enabled` | Renders CNPG operator group and subscription |
| `operators.redis_is_enabled` | Renders Redis operator group and subscription |
| `configuration.is_enabled` | Renders the `300-configuration` band |
| `configuration.cluster_secret_store_is_enabled` | Renders ClusterSecretStore |
| `databases.is_enabled` | Renders the `400-databases` band |
| `advantage_mcp.is_enabled` | Renders the `700-advantage-mcp` band |
| `applications.is_enabled` | Renders the `800-applications` band |

**Parent flags gate whole bands.** With `operators.is_enabled: false`, every sub-flag under
`operators` is irrelevant — the entire `200-operators` directory is omitted regardless of
what the sub-flags say. When a resource unexpectedly fails to render, check the parent flag
before the specific one.

### Image variables

```yaml
<domain>:
  <service>:
    image:
      address: <registry>/<image-name>
      tag: <semver-or-sha>
```

Every enabled workload needs non-empty values for both.

### Global application settings

`applications.global` carries settings shared by all services in the release:

```yaml
applications:
  global:
    is_internal: "true"          # string bool
    enable_instana: "false"      # string bool
    ca_secret_name: ca-secret
    ca_env: <env-name>           # dev | staging | production
    maintenance: "false"         # string bool
    host: <public-hostname>
    ca_host: <internal-hostname>
    suffix_url: <path-suffix>
    node_selector_app: <node-label>
    mongo:
      host: <mongodb-host>
      port: "27017"
      user: <mongodb-user>
    redis:
      host: <redis-host>
      port: "6379"
      max_retries: "3"
    neo4j:
      url: <neo4j-bolt-url>
      user: <neo4j-user>
```

---

## 4. Booleans: two different kinds

This trips people up constantly:

| Kind | Written as | Why |
|---|---|---|
| Copier feature flag (`*_is_enabled`) | YAML boolean `true` / `false` | Evaluated as a Jinja2 conditional deciding whether a directory renders |
| Helm string-bool (`is_internal`, `maintenance`, `enable_instana`) | Quoted string `"true"` / `"false"` | The values templates emit them as strings and the chart reads them as strings |

A feature flag written as `"false"` is a **non-empty string** — truthy in Jinja2 — so the
resource renders when you meant to disable it. This fails silently and deploys something.

---

## 5. Rules

### Answer files

- `_src_path` and `_commit` are Copier-managed; never edit them by hand.
- Keep answer files minimal — only override what differs from the `copier.yaml` default.
- Never commit real secrets. Use placeholders and inject at runtime via ExternalSecret.

### Multi-environment

- One answer file per environment; never share a file across environments.
- When adding a variable to `copier.yaml`, update every answer file needing a non-default
  value — the rest correctly fall back to the default.

### Enabling a resource

1. Set the flag to `true`.
2. Supply **all** required image and configuration variables for that resource.
3. Re-render the environment.
4. Review the rendered diff before committing.

Skipping step 2 renders a values file with empty fields; Helm then fails at sync with an
error that names the chart, not the answer file, which makes it slow to trace back.

### Disabling a resource

1. Set the flag to `false`.
2. Re-render.
3. ArgoCD prunes the Application and its cluster resources on the next sync, because
   `prune: true` is set. This is destructive and immediate — confirm the resource is really
   meant to leave the cluster, not just be paused.

---

## 6. Workflows

### New environment

```bash
mkdir <new-env>

# Option A — interactive prompts
copier copy . /tmp/gitops-<new-env> --answers-file <new-env>/copier-answers.yaml

# Option B — start from the nearest existing environment
cp <base-env>/copier-answers.yaml <new-env>/copier-answers.yaml
# then edit for environment-specific values
```

Option B is faster but carries over every override from the source environment; walk the
whole file rather than only the fields you came to change.

### Re-render after a template change

```bash
copier update /path/to/rendered-<env> \
  --answers-file <env>/copier-answers.yaml \
  --overwrite

git -C /path/to/rendered-<env> diff
```

### Compare two environments

```bash
copier copy . /tmp/env-a --overwrite --answers-file env-a/copier-answers.yaml
copier copy . /tmp/env-b --overwrite --answers-file env-b/copier-answers.yaml
diff -rq /tmp/env-a/cluster /tmp/env-b/cluster
```

### Audit which resources are enabled

```bash
python3 -c "
import yaml, sys
a = yaml.safe_load(open(sys.argv[1]))
def walk(d, prefix=''):
    for k, v in (d.items() if isinstance(d, dict) else []):
        if k.endswith('_is_enabled') or k == 'is_enabled':
            print(f'{prefix}{k}: {v}')
        elif isinstance(v, dict):
            walk(v, prefix + k + '.')
walk(a)
" <env>/copier-answers.yaml
```

---

## 7. Validation

```bash
# Render the environment
copier copy . /tmp/gitops-validate --overwrite --answers-file <env>/copier-answers.yaml

# No .jinja may leak through
find /tmp/gitops-validate -name "*.jinja" | wc -l   # must be 0

# Every rendered YAML parses
find /tmp/gitops-validate -name "*.yaml" -exec \
  python3 -c "import yaml, sys; yaml.safe_load(open(sys.argv[1])); print('OK:', sys.argv[1])" {} \;

# Every Application has its resources directory
for app in $(find /tmp/gitops-validate/cluster/argocd -name "application.yaml" -exec \
    python3 -c "import yaml,sys; print(yaml.safe_load(open(sys.argv[1]))['spec']['source']['path'])" {} \;); do
  [ -d "/tmp/gitops-validate/$app" ] || echo "MISSING resources dir: $app"
done
```

---

## 8. Mistakes to avoid

- Editing `_src_path` or `_commit` — breaks `copier update`.
- Enabling a flag without supplying the resource's image and configuration variables.
- Writing a feature flag as `"true"` instead of `true` — truthy either way, so it renders
  when disabled was intended.
- Editing an answer file without re-rendering and reviewing the diff.
- Setting `applications.global` addresses (host, mongo, redis, neo4j) without confirming the
  environment's real infrastructure — wrong values misconfigure every service silently.
- Putting environment secrets in the answer file rather than an external secret manager.
- Assuming a sub-flag controls a resource when its parent band flag is `false`.