172 lines
11 KiB
Markdown
172 lines
11 KiB
Markdown
# Archify Sync
|
|
|
|
The server behind live, shared models in [Archify Diagram Viewer](https://marketplace.visualstudio.com/items?itemName=root-at-skic.archify-vscode-ext). People edit one model at the same time and see each other type. Every edit is kept, versions are taken on their own, and anyone can publish one by hand.
|
|
|
|
One Rust process and PostgreSQL. The design, with its decisions, is the Archify model in `ws-storm/2026-10-30-archify-sync-backend/`.
|
|
|
|
## How it works
|
|
|
|
- **A space is one shared model.** Its content is a Yjs document with one `Y.Text` per model file, in the root map `files`, keyed by the path relative to the model root.
|
|
- **Live editing.** Each editor keeps one WebSocket per space and speaks the y-websocket protocol: sync step 1 and 2, updates and awareness. Every update is appended to PostgreSQL before anyone else sees it.
|
|
- **Offline.** Editors keep their own copy. On reconnect, both sides exchange state vectors and send what the other is missing.
|
|
- **Versions.**
|
|
- **Snapshots (0.1, 1.2, …):** taken after a pause in editing and when the last editor leaves.
|
|
- **Publish (1.0, 2.0, …):** marked by hand, with a name.
|
|
- **Restore:** brings an earlier version back as a new edit and a new version; history is never rewritten.
|
|
- **Update log:** taking a version compacts it.
|
|
- **Sign-in.** The editor opens `/authorize` in the browser and gets a one-time code back through its URI handler. It exchanges the code with its PKCE verifier for an access token (JWT, 15 minutes) and a refresh token. The refresh token is single use: reusing one signs out the whole session.
|
|
- **First start.** A server without accounts prints a first-start code in its log. The first person to use it at `/setup` becomes the admin.
|
|
- **Invites.** An invite link (`/join/<token>`) opens the extension. Someone new makes an account on the sign-in page, so no mail server is needed.
|
|
|
|
## Layout
|
|
|
|
| Path | What |
|
|
|---|---|
|
|
| `crates/protocol` | Types shared with the extension; exported to TypeScript with ts-rs |
|
|
| `crates/store` | PostgreSQL through sqlx; migrations in `crates/store/migrations` |
|
|
| `crates/server` | axum: the HTTP API, the pages, the live hub (`hub.rs`), the CRDT helpers (`doc.rs`) |
|
|
| `packages/protocol` | `@archify/sync-protocol`, the generated types for the extension |
|
|
|
|
moon runs the tasks: `moon run :check`, `moon run :test`, `moon run protocol:types`. Plain `cargo` works too.
|
|
|
|
## Run it locally
|
|
|
|
```bash
|
|
docker run -d --name archify-pg -e POSTGRES_PASSWORD=dev -e POSTGRES_DB=archify -p 5432:5432 postgres:17-alpine
|
|
ARCHIFY_SYNC_DATABASE_URL=postgres://postgres:dev@127.0.0.1:5432/archify cargo run -p archify-sync-server
|
|
```
|
|
|
|
The log shows the first-start code. Open `http://localhost:8080/setup` and enter it.
|
|
|
|
Or the server and its database in containers:
|
|
|
|
```bash
|
|
docker compose up -d --build
|
|
docker compose logs server | grep "with code"
|
|
```
|
|
|
|
## Kubernetes
|
|
|
|
The homelab deployment is in [archify-vscode-ext-sync-gitops](https://gitea.lego-cloud.eu/vscode-extensions/archify-vscode-ext-sync-gitops) (rendered from `archify-vscode-ext-sync-gitops-tmpl`); Argo CD takes this chart from here.
|
|
|
|
The chart is in `charts/archify-sync`. It always runs one pod with the `Recreate` strategy: one server holds the live copy of each space, and the extension works offline while the pod is replaced. Probes use `/healthz` and `/readyz`; on `SIGTERM` the server stores a snapshot of every open space.
|
|
|
|
```bash
|
|
helm install sync charts/archify-sync -n archify-sync --create-namespace \
|
|
--set publicUrl=https://archify-sync.example.com \
|
|
--set database.existingSecret=archify-sync-db # a Secret with the URL under "url"
|
|
```
|
|
|
|
`postgresql.enabled=true` adds a single PostgreSQL to the release, for trying out. `postgresql.existingClaim` puts its data on a claim made outside the chart; PostgreSQL runs as uid 70 and needs that directory owned by 70, mode 0700, so an NFS share must not map every user to root. Behind ingress-nginx, raise `proxy-read-timeout` and `proxy-send-timeout` (see `values.yaml`): edits travel over a WebSocket that stays open.
|
|
|
|
On a local cluster (OrbStack, kind, k3d) with the image from `docker build -t archify-sync:local .`:
|
|
|
|
```bash
|
|
helm install sync charts/archify-sync -n archify-sync --create-namespace -f charts/archify-sync/ci/local-values.yaml
|
|
```
|
|
|
|
The server is then at `http://localhost:18081`.
|
|
|
|
## Single sign-on (OIDC)
|
|
|
|
With an OpenID Connect provider (Keycloak, Authentik, Entra ID, …) the sign-in page shows **Sign in with Keycloak** above the password form. Nothing changes in the extension: it opens the same page, and the provider sends the person back to `<public URL>/oidc/callback`. People are matched by the provider's issuer and subject, never by name; a new person gets an account named after `preferred_username` (`ana`, or `ana-2` when `ana` is taken).
|
|
|
|
Who gets in:
|
|
|
|
- **Allowed groups set** (`ARCHIFY_SYNC_OIDC_ALLOWED_GROUPS=archify-users`): members of those groups get an account on their first sign-in and join spaces through invites; everyone else is refused, also later, when they leave the group.
|
|
- **No allowed groups**: anyone the provider signs in, but a new person needs an invite link the first time.
|
|
- **Admin groups** (`ARCHIFY_SYNC_OIDC_ADMIN_GROUPS=archify-admins`): their members are admins of the server.
|
|
|
|
The first admin is still set up with a password on `/setup`; keep that account as the way in when the provider is down.
|
|
|
|
### The client at the provider
|
|
|
|
One client serves every place the server runs: list each place's callback. For Keycloak, in the realm the people sign in to:
|
|
|
|
| Field | Value |
|
|
|---|---|
|
|
| Client ID | `archify-sync` |
|
|
| Client authentication | on (confidential), authenticator *client secret* |
|
|
| Flows | standard flow only (no implicit, no direct access grants, no service account) |
|
|
| PKCE | S256 (the server always sends it) |
|
|
| Valid redirect URIs | `https://<your host>/oidc/callback`, plus `http://localhost:8080/oidc/callback` and `http://localhost:18081/oidc/callback` for local runs |
|
|
| Mapper | *Group Membership*, claim `groups`, full path off, in the ID token |
|
|
|
|
Then give the server the realm's issuer (`https://<keycloak>/realms/<realm>`), the client id and the secret:
|
|
|
|
```bash
|
|
ARCHIFY_SYNC_OIDC_ISSUER=https://keycloak.example.com/realms/home
|
|
ARCHIFY_SYNC_OIDC_CLIENT_ID=archify-sync
|
|
ARCHIFY_SYNC_OIDC_CLIENT_SECRET=… # or ARCHIFY_SYNC_OIDC_CLIENT_SECRET_FILE=/run/secrets/oidc
|
|
ARCHIFY_SYNC_OIDC_LABEL=Keycloak
|
|
ARCHIFY_SYNC_OIDC_ALLOWED_GROUPS=archify-users
|
|
ARCHIFY_SYNC_OIDC_ADMIN_GROUPS=archify-admins
|
|
```
|
|
|
|
In the Helm chart: `oidc.enabled=true`, `oidc.issuer`, `oidc.clientId`, `oidc.existingSecret` (a Secret with the client secret under `client-secret`, for example from an ExternalSecret), `oidc.allowedGroups`, `oidc.adminGroups`. The server logs the callback to register when it starts.
|
|
|
|
The issuer must be `https`, except on this machine. A session lasts `ARCHIFY_SYNC_REFRESH_TTL_DAYS` after the last use: disabling someone at the provider stops new sign-ins at once, and an admin removes them from spaces to end an open session sooner.
|
|
|
|
## Sign-in protection
|
|
|
|
Ten wrong passwords for one name within 15 minutes lock that name for the rest of the window, even with the right password. The lock lives in the server's memory and is cleared on restart.
|
|
|
|
## Metrics
|
|
|
|
With `ARCHIFY_SYNC_METRICS_LISTEN` set, `/metrics` serves `archify_sync_rooms_open`, `connections_open`, `connections_total`, `connections_lagged_total`, `updates_total`, `update_bytes_total`, `update_store_seconds_total`, `snapshots_total`, `publishes_total`, `restores_total`, `sign_in_failures_total`, `sign_in_blocked_total` and `refresh_reuse_total`.
|
|
|
|
## Settings
|
|
|
|
All settings come from the environment.
|
|
|
|
| Variable | Default | |
|
|
|---|---|---|
|
|
| `ARCHIFY_SYNC_DATABASE_URL` | (required) | PostgreSQL 17 |
|
|
| `ARCHIFY_SYNC_LISTEN` | `0.0.0.0:8080` | Plain HTTP and WebSocket; TLS ends in front (ingress or Caddy) |
|
|
| `ARCHIFY_SYNC_METRICS_LISTEN` | off | Prometheus `/metrics` on its own address, e.g. `0.0.0.0:9090` |
|
|
| `ARCHIFY_SYNC_PUBLIC_URL` | `http://localhost:<port>` | Where people reach the server; invite links use it |
|
|
| `ARCHIFY_SYNC_NAME` | `Archify Sync` | Shown on the pages |
|
|
| `ARCHIFY_SYNC_SNAPSHOT_IDLE_SECS` | `30` | Quiet time before a snapshot |
|
|
| `ARCHIFY_SYNC_SNAPSHOT_MAX_SECS` | `600` | A space edited without a pause still gets a snapshot this often |
|
|
| `ARCHIFY_SYNC_SNAPSHOT_MAX_EDITS` | `2000` | …or after this many edits; each snapshot empties the update log |
|
|
| `ARCHIFY_SYNC_INVITE_HOURS` | `168` | How long an invite link works |
|
|
| `ARCHIFY_SYNC_MEMBERS_CAN_INVITE` | `false` | Whether editors may invite too, not only owners and admins |
|
|
| `ARCHIFY_SYNC_ACCESS_TTL_SECS` | `900` | Access token lifetime |
|
|
| `ARCHIFY_SYNC_REFRESH_TTL_DAYS` | `30` | Session lifetime |
|
|
| `ARCHIFY_SYNC_REFRESH_GRACE_SECS` | `30` | A refresh token used again within this time (two windows refreshing at once) still works; later, the whole session is signed out |
|
|
| `ARCHIFY_SYNC_REDIRECT_SCHEMES` | `vscode,vscode-insiders,vscodium,cursor,windsurf` | Editors that may receive a sign-in code |
|
|
| `ARCHIFY_SYNC_EXTENSION_ID` | `root-at-skic.archify-vscode-ext` | Authority of the extension's callback URI |
|
|
| `ARCHIFY_SYNC_MAX_MESSAGE_BYTES` | `8388608` | Largest WebSocket frame |
|
|
| `ARCHIFY_SYNC_LOG_JSON` | `false` | JSON logs |
|
|
| `RUST_LOG` | `info` | Log filter |
|
|
|
|
## API
|
|
|
|
| Method and path | |
|
|
|---|---|
|
|
| `GET /v1/info` | Name, version, protocol, whether it is set up |
|
|
| `POST /v1/setup` | First admin with the first-start code |
|
|
| `POST /v1/auth/token` | `authorization_code` with PKCE, or `refresh_token` |
|
|
| `POST /v1/auth/logout` | Ends the session of a refresh token |
|
|
| `GET /v1/me` | The signed-in account |
|
|
| `GET, POST /v1/spaces` | Your spaces; create one (you own it) |
|
|
| `GET /v1/spaces/{id}` | One space |
|
|
| `GET /v1/spaces/{id}/members`, `DELETE …/members/{account}` | Members; leave, or remove someone (owner, admin) |
|
|
| `POST /v1/spaces/{id}/invites` | An invite link (owner, admin) |
|
|
| `POST /v1/invites/{token}/accept` | Join with a link while signed in |
|
|
| `GET, POST /v1/spaces/{id}/versions` | The history; publish the current state |
|
|
| `GET /v1/spaces/{id}/versions/{v}/files` | A version's files as text |
|
|
| `POST /v1/spaces/{id}/versions/{v}/restore` | Restore as a new version |
|
|
| `GET /v1/spaces/{id}/sync?v=1` | WebSocket, y-websocket protocol, `Authorization: Bearer` |
|
|
| `GET /healthz`, `GET /readyz` | Liveness; readiness checks PostgreSQL |
|
|
| `GET /setup`, `/authorize`, `/join/{token}` | The browser pages |
|
|
|
|
## Not yet
|
|
|
|
- **Model check.** The server does not validate files against the Archify schemas; each editor reports a file that two edits left broken in its Problems panel.
|
|
- **Structural compare.** The server returns a version's files, and the client compares them.
|
|
|
|
## Licence
|
|
|
|
MIT
|