documentation-docusaurus/references/migrating-existing-markdown-corpora.md

Migrating an Existing Markdown Corpus to Docusaurus

Use this pattern when an established evidence, registry, research, or documentation repository must become a Docusaurus site without turning generated presentation files into the source of truth.

Preserve the canonical corpus

Keep canonical files in their existing paths when validators, scheduled jobs, links, or provenance depend on them. Do not bulk-move or inject front matter into hundreds of source documents merely to satisfy Docusaurus.

Recommended split:

clients/ or records/             # canonical Markdown
INDEX.md                         # canonical registry
source/                          # supplied/source artifacts
scripts/validate_registry.py     # canonical content validation
site-docs/                       # authored website guidance
.generated-docs/                 # ignored, generated MDX-safe copies
static/raw/                      # ignored, staged byte-identical artifacts
scripts/generate_site_content.py
scripts/publish_raw_artifacts.py
scripts/validate_site_build.py

Configure a second @docusaurus/plugin-content-docs instance for .generated-docs/. Build scripts should regenerate it from canonical sources every time.

Strict generated-copy transform

The generator should:

1. Parse the canonical index strictly and require the expected IDs, paths, statuses, and order.
2. Verify every indexed source exists.
3. Recreate the generated tree from scratch.
4. Apply only an explicit compatibility allowlist.
5. Fail if a new or unexpected transform is needed.
6. Generate navigation/index content from the same canonical registry.

A common MDX hazard is raw <br> in Markdown tables. Convert it to <br /> only in generated copies and assert the exact files/replacement counts. Literal placeholders such as <ID> in templates should remain outside the compiled docs tree or be escaped in a generated display copy.

Preserve stable numeric routes

Docusaurus treats numeric filename prefixes as sidebar ordering metadata and can remove them from inferred routes. For canonical file 001-client-name.md, prepend generated front matter:

---
slug: /001-client-name
---

This preserves /clients/001-client-name/ while leaving the canonical file untouched.

For hundreds of documents, generate category directories (for example 001-050, 051-100) with _category_.json. Keep the explicit slug so grouping changes navigation without changing public routes.

Raw artifact publication

For provenance-sensitive repositories, stage byte-identical canonical files under static/raw/ before docusaurus build and generate a checksum manifest. Validate hashes between canonical files and build/raw/ after the build.

If trailingSlash: true causes Docusaurus broken-link checking to append / to extensionless static paths, publish/link a manifest with an extension such as SHA256SUMS.txt, or use the final absolute Pages URL. Do not relax onBrokenLinks: 'throw' merely to hide this.

Build pipeline

A robust package script is:

{
  "scripts": {
    "validate": "python3 scripts/generate_site_content.py && python3 scripts/validate_registry.py && tsc --noEmit",
    "build": "python3 scripts/generate_site_content.py && python3 scripts/validate_registry.py && python3 scripts/publish_raw_artifacts.py && docusaurus build && python3 scripts/validate_site_build.py"
  }
}

Commit pnpm-lock.yaml, pin one Docusaurus release across packages, and keep generated directories ignored.

Site-build validation

After the real production build, assert:

• expected number of rendered document routes;
• representative first, middle, compatibility-transformed, longest-path, and last routes;
• expected production baseUrl in emitted HTML/assets;
• generated directory/index record count;
• byte equality or SHA-256 equality for every published raw artifact;
• homepage, directory, representative deep links, and checksum manifest return HTTP 200 under the exact subpath.

Also inspect the site visually at the production subpath. For large tables, enforce horizontal scrolling without removing columns. For a square UI, use both Infima radius variables and a final global border-radius: 0 !important override.

Actions and continuity

The Gitea workflow must run source validation, project validation, the real build, and build-output validation before publishing. Observe the actual Actions conclusion and then verify the public site; workflow YAML presence is not completion.

Before conversion, pause scheduled writers. After publication:

• update their workdir and remote URL;
• tell them to edit canonical files only, never generated site output;
• resume only after local worktrees and remote synchronization are verified.