180 lines
12 KiB
Markdown
180 lines
12 KiB
Markdown
# Archify Sync
|
|
|
|
[](https://buymeacoffee.com/root.at.skic)
|
|
|
|
The server behind live, shared models in [Archify Diagram Viewer](https://marketplace.visualstudio.com/items?itemName=root-at-skic.archify-vscode-ext). People edit one model at the same time and see each other type. Every edit is kept, versions are taken on their own, and anyone can publish one by hand.
|
|
|
|
One Rust process and PostgreSQL.
|
|
|
|
## How it works
|
|
|
|
- **A space is one shared model.** Its content is a Yjs document with one `Y.Text` per model file, in the root map `files`, keyed by the path relative to the model root.
|
|
- **Live editing.** Each editor keeps one WebSocket per space and speaks the y-websocket protocol: sync step 1 and 2, updates and awareness. Every update is appended to PostgreSQL before anyone else sees it.
|
|
- **Offline.** Editors keep their own copy. On reconnect, both sides exchange state vectors and send what the other is missing.
|
|
- **Versions.**
|
|
- **Snapshots (0.1, 1.2, …):** taken after a pause in editing and when the last editor leaves.
|
|
- **Publish (1.0, 2.0, …):** marked by hand, with a name.
|
|
- **Restore:** brings an earlier version back as a new edit and a new version; history is never rewritten.
|
|
- **Update log:** taking a version compacts it.
|
|
- **Sign-in.** The editor opens `/authorize` in the browser and gets a one-time code back: on the desktop through a one-off listener on `127.0.0.1`, in remote and web editors through its URI handler (`vscode://…/auth`). After sign-in the server shows a page that opens the editor, with a button for when the browser holds the link back. The editor exchanges the code with its PKCE verifier for an access token (JWT, 15 minutes) and a refresh token. The refresh token is single use: reusing one signs out the whole session.
|
|
- **First start.** With single sign-on there is no setup step: the identity provider says who may sign in and who is admin, and the first sign-in makes the organisation (named after `ARCHIFY_SYNC_NAME`). Without single sign-on, a server without accounts prints a first-start code in its log; the first person to use it at `/setup` becomes the admin.
|
|
- **Invites.** An invite link (`/join/<token>`) opens the extension. Someone new makes an account on the sign-in page, so no mail server is needed. With single sign-on, the identity provider is the only way in: sign-in goes straight to it, with no password form, and an invited person signs in there.
|
|
|
|
## Layout
|
|
|
|
| Path | What |
|
|
|---|---|
|
|
| `crates/protocol` | Types shared with the extension; exported to TypeScript with ts-rs |
|
|
| `crates/store` | PostgreSQL through sqlx; migrations in `crates/store/migrations` |
|
|
| `crates/server` | axum: the HTTP API, the pages, the live hub (`hub.rs`), the CRDT helpers (`doc.rs`) |
|
|
| `packages/protocol` | `@archify/sync-protocol`, the generated types for the extension |
|
|
|
|
moon runs the tasks: `moon run :check`, `moon run :test`, `moon run protocol:types`. Plain `cargo` works too.
|
|
|
|
## Run it locally
|
|
|
|
```bash
|
|
docker run -d --name archify-pg -e POSTGRES_PASSWORD=dev -e POSTGRES_DB=archify -p 5432:5432 postgres:17-alpine
|
|
ARCHIFY_SYNC_DATABASE_URL=postgres://postgres:dev@127.0.0.1:5432/archify cargo run -p archify-sync-server
|
|
```
|
|
|
|
Without single sign-on, the log shows the first-start code. Open `http://localhost:8080/setup` and enter it.
|
|
|
|
Or the server and its database in containers:
|
|
|
|
```bash
|
|
docker compose up -d --build
|
|
docker compose logs server | grep "with code"
|
|
```
|
|
|
|
## Kubernetes
|
|
|
|
For Argo CD, [archify-vscode-ext-sync-gitops-tmpl](https://gitea.lego-cloud.eu/vscode-extensions/archify-vscode-ext-sync-gitops-tmpl) is a Copier template that renders an app-of-apps with this chart, its values and, optionally, its secrets through External Secrets.
|
|
|
|
The chart is in `charts/archify-sync`. It always runs one pod with the `Recreate` strategy: one server holds the live copy of each space, and the extension works offline while the pod is replaced. Probes use `/healthz` and `/readyz`; on `SIGTERM` the server stores a snapshot of every open space.
|
|
|
|
```bash
|
|
helm install sync charts/archify-sync -n archify-sync --create-namespace \
|
|
--set publicUrl=https://archify-sync.example.com \
|
|
--set database.existingSecret=archify-sync-db # a Secret with the URL under "url"
|
|
```
|
|
|
|
`postgresql.enabled=true` adds a single PostgreSQL to the release, for trying out. `postgresql.existingClaim` puts its data on a claim made outside the chart; PostgreSQL runs as uid 70 and needs that directory owned by 70, mode 0700, so an NFS share must not map every user to root. Behind ingress-nginx, raise `proxy-read-timeout` and `proxy-send-timeout` (see `values.yaml`): edits travel over a WebSocket that stays open.
|
|
|
|
On a local cluster (OrbStack, kind, k3d) with the image from `docker build -t archify-sync:local .`:
|
|
|
|
```bash
|
|
helm install sync charts/archify-sync -n archify-sync --create-namespace -f charts/archify-sync/ci/local-values.yaml
|
|
```
|
|
|
|
The server is then at `http://localhost:18081`.
|
|
|
|
## Single sign-on (OIDC)
|
|
|
|
With an OpenID Connect provider (Keycloak, Authentik, Entra ID, …) the provider is the only way in: `/authorize` goes straight to it, there is no password form, and no local accounts are made. Nothing changes in the extension: it opens the same page, and the provider sends the person back to `<public URL>/oidc/callback`. People are matched by the provider's issuer and subject, never by name; a new person gets an account named after `preferred_username` (`ana`, or `ana-2` when `ana` is taken).
|
|
|
|
Who gets in:
|
|
|
|
- **Allowed groups set** (`ARCHIFY_SYNC_OIDC_ALLOWED_GROUPS=archify-users`): members of those groups get an account on their first sign-in and join spaces through invites; everyone else is refused, also later, when they leave the group.
|
|
- **No allowed groups**: anyone the provider signs in, but a new person needs an invite link the first time.
|
|
- **Admin groups** (`ARCHIFY_SYNC_OIDC_ADMIN_GROUPS=archify-admins`): their members are admins of the server.
|
|
|
|
There is no `/setup` step: the first sign-in by an allowed or admin member makes the organisation, named after `ARCHIFY_SYNC_NAME`.
|
|
|
|
### The client at the provider
|
|
|
|
One client serves every place the server runs: list each place's callback. For Keycloak, in the realm the people sign in to:
|
|
|
|
| Field | Value |
|
|
|---|---|
|
|
| Client ID | `archify-sync` |
|
|
| Client authentication | on (confidential), authenticator *client secret* |
|
|
| Flows | standard flow only (no implicit, no direct access grants, no service account) |
|
|
| PKCE | S256 (the server always sends it) |
|
|
| Valid redirect URIs | `https://<your host>/oidc/callback`, plus `http://localhost:8080/oidc/callback` and `http://localhost:18081/oidc/callback` for local runs |
|
|
| Mapper | *Group Membership*, claim `groups`, full path off, in the ID token |
|
|
|
|
Then give the server the realm's issuer (`https://<keycloak>/realms/<realm>`), the client id and the secret:
|
|
|
|
```bash
|
|
ARCHIFY_SYNC_OIDC_ISSUER=https://keycloak.example.com/realms/home
|
|
ARCHIFY_SYNC_OIDC_CLIENT_ID=archify-sync
|
|
ARCHIFY_SYNC_OIDC_CLIENT_SECRET=… # or ARCHIFY_SYNC_OIDC_CLIENT_SECRET_FILE=/run/secrets/oidc
|
|
ARCHIFY_SYNC_OIDC_LABEL=Keycloak
|
|
ARCHIFY_SYNC_OIDC_ALLOWED_GROUPS=archify-users
|
|
ARCHIFY_SYNC_OIDC_ADMIN_GROUPS=archify-admins
|
|
```
|
|
|
|
In the Helm chart: `oidc.enabled=true`, `oidc.issuer`, `oidc.clientId`, `oidc.existingSecret` (a Secret with the client secret under `client-secret`, for example from an ExternalSecret), `oidc.allowedGroups`, `oidc.adminGroups`. The server logs the callback to register when it starts.
|
|
|
|
The issuer must be `https`, except on this machine. A session lasts `ARCHIFY_SYNC_REFRESH_TTL_DAYS` after the last use: disabling someone at the provider stops new sign-ins at once, and an admin removes them from spaces to end an open session sooner.
|
|
|
|
## Sign-in protection
|
|
|
|
Ten wrong passwords for one name within 15 minutes lock that name for the rest of the window, even with the right password. The lock lives in the server's memory and is cleared on restart.
|
|
|
|
## Metrics
|
|
|
|
With `ARCHIFY_SYNC_METRICS_LISTEN` set, `/metrics` serves `archify_sync_rooms_open`, `connections_open`, `connections_total`, `connections_lagged_total`, `updates_total`, `update_bytes_total`, `update_store_seconds_total`, `snapshots_total`, `publishes_total`, `restores_total`, `sign_in_failures_total`, `sign_in_blocked_total` and `refresh_reuse_total`.
|
|
|
|
## Settings
|
|
|
|
All settings come from the environment.
|
|
|
|
| Variable | Default | |
|
|
|---|---|---|
|
|
| `ARCHIFY_SYNC_DATABASE_URL` | (required) | PostgreSQL 17 |
|
|
| `ARCHIFY_SYNC_LISTEN` | `0.0.0.0:8080` | Plain HTTP and WebSocket; TLS ends in front (ingress or Caddy) |
|
|
| `ARCHIFY_SYNC_METRICS_LISTEN` | off | Prometheus `/metrics` on its own address, e.g. `0.0.0.0:9090` |
|
|
| `ARCHIFY_SYNC_PUBLIC_URL` | `http://localhost:<port>` | Where people reach the server; invite links use it |
|
|
| `ARCHIFY_SYNC_NAME` | `Archify Sync` | Shown on the pages |
|
|
| `ARCHIFY_SYNC_SNAPSHOT_IDLE_SECS` | `30` | Quiet time before a snapshot |
|
|
| `ARCHIFY_SYNC_SNAPSHOT_MAX_SECS` | `600` | A space edited without a pause still gets a snapshot this often |
|
|
| `ARCHIFY_SYNC_SNAPSHOT_MAX_EDITS` | `2000` | …or after this many edits; each snapshot empties the update log |
|
|
| `ARCHIFY_SYNC_INVITE_HOURS` | `168` | How long an invite link works |
|
|
| `ARCHIFY_SYNC_MEMBERS_CAN_INVITE` | `false` | Whether editors may invite too, not only owners and admins |
|
|
| `ARCHIFY_SYNC_ACCESS_TTL_SECS` | `900` | Access token lifetime |
|
|
| `ARCHIFY_SYNC_REFRESH_TTL_DAYS` | `30` | Session lifetime |
|
|
| `ARCHIFY_SYNC_REFRESH_GRACE_SECS` | `30` | A refresh token used again within this time (two windows refreshing at once) still works; later, the whole session is signed out |
|
|
| `ARCHIFY_SYNC_REDIRECT_SCHEMES` | `vscode,vscode-insiders,vscodium,cursor,windsurf` | Editors that may receive a sign-in code |
|
|
| `ARCHIFY_SYNC_EXTENSION_ID` | `root-at-skic.archify-vscode-ext` | Authority of the extension's callback URI |
|
|
| `ARCHIFY_SYNC_MAX_MESSAGE_BYTES` | `8388608` | Largest WebSocket frame |
|
|
| `ARCHIFY_SYNC_LOG_JSON` | `false` | JSON logs |
|
|
| `RUST_LOG` | `info` | Log filter |
|
|
|
|
## API
|
|
|
|
| Method and path | |
|
|
|---|---|
|
|
| `GET /v1/info` | Name, version, protocol, whether it is set up |
|
|
| `POST /v1/setup` | First admin with the first-start code |
|
|
| `POST /v1/auth/token` | `authorization_code` with PKCE, or `refresh_token` |
|
|
| `POST /v1/auth/logout` | Ends the session of a refresh token |
|
|
| `GET /v1/me` | The signed-in account |
|
|
| `GET, POST /v1/spaces` | Your spaces; create one (you own it) |
|
|
| `GET /v1/spaces/{id}` | One space |
|
|
| `GET /v1/spaces/{id}/members`, `DELETE …/members/{account}` | Members; leave, or remove someone (owner, admin) |
|
|
| `POST /v1/spaces/{id}/invites` | An invite link (owner, admin) |
|
|
| `POST /v1/invites/{token}/accept` | Join with a link while signed in |
|
|
| `GET, POST /v1/spaces/{id}/versions` | The history; publish the current state |
|
|
| `GET /v1/spaces/{id}/versions/{v}/files` | A version's files as text |
|
|
| `POST /v1/spaces/{id}/versions/{v}/restore` | Restore as a new version |
|
|
| `GET /v1/spaces/{id}/sync?v=1` | WebSocket, y-websocket protocol, `Authorization: Bearer` |
|
|
| `GET /healthz`, `GET /readyz` | Liveness; readiness checks PostgreSQL |
|
|
| `GET /setup`, `/authorize`, `/join/{token}` | The browser pages |
|
|
|
|
## Not yet
|
|
|
|
- **Model check.** The server does not validate files against the Archify schemas; each editor reports a file that two edits left broken in its Problems panel.
|
|
- **Structural compare.** The server returns a version's files, and the client compares them.
|
|
|
|
## Licence
|
|
|
|
MIT
|
|
|
|
## Support
|
|
|
|
Archify Sync is built and maintained in my spare time. If it helps you, a coffee keeps the releases coming:
|
|
|
|
[](https://buymeacoffee.com/root.at.skic)
|