Update Guide¶

Guide for updating calServer V2 to a new version.

In-app updates via the updater sidecar (recommended)¶

calServer V2 ships an integrated update system (ported from eforms.cloud): a separate updater container (calserver-updater) accepts authenticated update requests from the V2 API, checks the Docker registry for new version tags, pulls the new images and recreates only the V2 containers (calserver-api-v2, calserver-frontend). The V1 containers are untouched; V1 keeps deploying via IMAGE_TAG + deploy.sh.

Architecture¶

Component	Role
`calserver-updater` (Go sidecar)	Only container with a Docker-socket mount. Endpoints: `/check`, `/update`, `/rollback`, `/status`, `/changelog`. Writes `V2_IMAGE_TAG` to the host `.env`.
Laravel API (`/api/v2/admin/updates/*`)	Update check, trigger update/rollback, version log (`update_logs`), status polling. Permission: `administration_management`.
Admin UI (`/admin/updates`)	Current version, update check, one-click update/rollback, update history, changelog.
CI (`ci.yml`)	Only maintains branch pointers (`develop-latest` for develop, `release-candidate` for `release/*` hotfix builds). The immutable semver tags `v<VERSION>` — the foundation of the update check — and the rolling pointer `release-latest` come exclusively from the promote workflow.

Enabling¶

Generate a token and add it to the host .env (in the deploy directory, e.g. /var/calserver/.env):
```
echo "UPDATER_TOKEN=$(openssl rand -hex 32)" >> .env
```
Registry access: the calhelp/* images are private. The way in is registry.calhelp.de with the license key: once a key is stored under Administration → Subscription & Modules, deploy.sh logs in with it, test-pulls the API image and only then writes IMAGE_REGISTRY=registry.calhelp.de into .env. The updater fetches instance ID and key from the API container on every update (docker exec calserver-api-v2 php artisan license:registry-credentials); the key is never stored in the host .env.

Until every installation has moved, Docker Hub with DOCKERHUB_USERNAME and DOCKERHUB_PULL_TOKEN remains the fallback. Without a license key and without both variables the update check fails with HTTP 401; deploy.sh warns about it. IMAGE_REGISTRY=docker.io in .env keeps an installation on Docker Hub explicitly.

Subscription license: on installations running against a license service (LICENSE_SERVER_URL + LICENSE_PUBLIC_KEY set), an expired subscription blocks both the update check and applying an update. This page only displays the subscription state; the key is entered and checked under Administration → Subscription & Modules (/admin/subscription) — see License for details. Without a configured license service the check is inactive and changes nothing about the update path.
Deploy — deploy.sh activates the compose profile v2-updater automatically once UPDATER_TOKEN is set:
```
./deploy.sh
```

Without UPDATER_TOKEN nothing changes: the updater container is not started and /admin/updates stays read-only (history + changelog only).

Update flow¶

Check: /admin/updates → "Check now" asks the updater for newer v<x.y.z> tags in the registry (channel UPDATE_CHANNEL, default stable).
Apply: "Update" writes a version-log entry (update_logs), the updater pulls the images, writes V2_IMAGE_TAG to .env and recreates the V2 containers (docker compose up -d --no-deps). Migrations run automatically on container start (entrypoint).
Confirm: The UI polls the log status until the new tag is confirmed. After a successful update the updater prunes orphaned old V2 images (a rollback buffer is always kept).
Rollback: Every completed or failed history entry offers a rollback to the previously running tag.

Major updates: two-step confirmation¶

Crossing a major boundary (e.g. 5.64.x -> 6.0.0) does not run as an ordinary update. On /admin/updates the button reads "Prepare major update" instead of "Update": first the migration notes and the releases in between, then a backup confirmation plus typing the target version. Only then does the update start.

The updater sidecar and the V2 API require the confirmation independently of each other (409 and 422 respectively), so a system running an older updater image is still protected. The way back is never blocked: a rollback across the major boundary stays possible, but it only restores the containers, not the database schema. Hence the backup requirement.

ALLOW_MAJOR_UPDATE=false in the host .env blocks major updates fleet-wide, even when confirmed; true skips the prompt.

Also visible: the V1 offset. The updater only recreates the V2 containers; V1 still follows IMAGE_TAG + deploy.sh. The version card shows both tags.

Step by step: Migration to 6.0.

Beta channel¶

Releases go through the beta channel first. A test system sets UPDATE_CHANNEL=beta in the host .env and should run a real semver tag as V2_IMAGE_TAG (not develop-latest) — otherwise the running version cannot be determined and the comparison is only an indication.

Update ALLOWED_TAG_PATTERNS on existing systems

If you set the pattern yourself in the host .env, add the pre-release suffix — otherwise the check finds a beta tag and applying it fails with 403:

ALLOWED_TAG_PATTERNS=^v\d+\.\d+\.\d+(-[0-9A-Za-z.-]+)?$|^(develop|release|master|beta)-latest$|^latest$

Release process: CI "Promote to Prod"¶

There is no master branch anymore: develop is the integration and deploy branch (dev server), release/* branches serve hotfix builds (pointer release-candidate). New versions are cut exclusively via the manual CI workflow Promote to Prod (.github/workflows/promote-to-prod.yml, Actions → "Promote to Prod" → "Run workflow"):

The workflow can only be dispatched from develop or release/* (git tag and release depend on the branch state). Without an input the source is the newest immutable build tag <branch>-<sha7> of the dispatched branch (e.g. develop-33b1d26). ci.yml assigns it to all seven deploy images within the same run, so one tag names a complete, finished set. That is why you can promote at any time, even while develop is building: the promoted state is a different one from the state currently being produced. The target version then comes from the VERSION file at exactly that commit, not from the branch tip, which may have moved on (background: ADR 2026-08-15).

The moving branch pointers develop-latest / release-candidate remain selectable as a source — the CI gate applies in that case (see below).

Validation + digest pinning: all seven deploy images (calwebapp_bootstrap, report-runner, calserver-api-v2, calserver-frontend, calserver-updater, calserver-mcp, calserver-seal) are resolved to digests once under the source tag; every following step works with these digests only. A re-verify at the end checks that none of them moved between the first and the second resolve — a CI build running in parallel can no longer tag a mixed state into the version. The CI gate now only applies to the moving pointer sources develop-latest/release-candidate: no build may be running on the source branch and the last effective ci.yml run must be green. Effective means the run actually executed jobs. Waiting runs and runs displaced by the concurrency group do not count — ci.yml uses cancel-in-progress: false, GitHub keeps exactly one waiting run per group, and every new push cancels the previously waiting one. On an active develop the newest run is therefore usually one that has not moved a single pointer yet. For <branch>-<sha7>, release-latest, beta-latest and v<x.y.z> the gate does not apply: nothing moves at a finished state. A missing source tag aborts the promote. Since ADR 2026-11-09 this also holds for calserver-updater and calserver-seal, which used to fall back to :latest: no build pushes a bare latest any more.
Re-tag without rebuild: each image receives the immutable semver tag v<VERSION> via docker buildx imagetools create from the digest reference. What was validated ships byte for byte. If the semver tag already exists with a different digest, the promote aborts during validation — before any tag is written (semver tags are rollback targets of running installations).
Rolling pointers: only after all seven v-tags exist, release-latest (or beta-latest for pre-releases) is moved to the same digests — the pointer set never points at a half-tagged version.
GitHub Release: attached to the existing git tag <VERSION> (no v-prefix, created by the version bump) with generated release notes. 4b. Helm release chart: the chart is packaged at the release commit as version <VERSION> with appVersion: v<VERSION> and published through deploy-docs.yml at https://doc.calserver.com/charts. The image tags in the chart are empty and fall back to appVersion; clusters update on their own with helm upgrade --reuse-values <release> calwebapp/calserver --version <VERSION>.
Demo deploy: the demo/release server is updated to the freshly promoted state (release-latest) — formerly the master-build job in ci.yml, now part of the promote. An identity check verifies the V2 containers actually run release-latest afterwards (a V2_IMAGE_TAG pinned in the host .env turns the run red instead of silently undermining the promote).
Changelog push: all releases are pushed as a JSON snapshot via POST /internal/updater/changelog to the prod updater and land as storage/app/changelog.json in the V2 API — the source of the release notes on /admin/updates.
Prod trigger (optional): POST /internal/updater/update applies the version directly on the prod system (can be disabled via the dispatch option skip_prod_trigger) — only after a successful demo deploy, whose smoke test is the last gate before prod. If the updater's major gate blocks (HTTP 409), the run stays green with a warning: the update is then confirmed interactively via /admin/updates.

Steps 4+5 require the repo secrets PROD_APP_URL and PROD_UPDATER_TOKEN plus the nginx route /internal/updater/ (docker/nginx/vhost.d/default_location); without the secrets they are skipped with a notice. Customer systems then see the new version via the update check on /admin/updates and apply it themselves.

Apply the nginx route on existing servers

The CI deploy deliberately does not sync nginx/ to existing servers (host-managed configuration). On an existing system, copy the updated nginx/vhost.d/default_location from the repo to /var/calserver/nginx/vhost.d/ once and reload the proxy (docker exec nginx-proxy nginx -s reload).

Only after a promote does the update check (stable channel) find a new version — regular CI builds only maintain the branch pointers develop-latest/release-candidate.

Relevant environment variables (host `.env`)¶

Variable	Meaning	Default
`UPDATER_TOKEN`	Shared secret API ↔ updater (min. 32 chars)	— (unset = disabled)
`V2_IMAGE_TAG`	Active V2 image tag (managed by the updater)	falls back to `IMAGE_TAG`
`UPDATER_IMAGE_TAG`	Pin for the updater image itself — writable via Administration -> Updates	`release-latest`
`UPDATER_IMAGE_REPO`	Registry/repository of the updater image (for private mirrors)	`calhelp/calserver-updater`
`ALLOWED_UPDATER_TAG_PATTERNS`	Regex whitelist for the updater's self-pin	`v<x.y.z[-pre]>`, branch pointers, `release-candidate`, `updater-<hash>`; `latest` only for existing pins
`REPORT_RUNNER_TAG`	Pin for the report-runner (host `.env` only, no UI button)	empty = follows `V2_IMAGE_TAG`
`UPDATE_CHANNEL`	`stable` / `beta` / `dev`	`stable`
`UPDATE_SERVICES`	Compose services recreated by an update	`calserver-api-v2,calserver-frontend`
`UPDATER_COMPOSE_FILE`	The updater's compose file list (colon-separated) — align it when a custom SSL overlay (e.g. `docker-compose.ssl.yml`) configures the V2 services	`docker-compose.yml:docker-compose.v2.yml`
`ALLOWED_TAG_PATTERNS`	Regex whitelist of allowed tags	`v<x.y.z[-pre]>`, `develop/release/master/beta-latest`; `latest` only for existing pins
`ALLOW_MAJOR_UPDATE`	Major updates: empty = confirmation required, `true` = no prompt, `false` = blocked	empty
`UPDATE_MIGRATION_NOTES_URL`	Link offered by the major gate in the confirmation dialog	migration guide on doc.calserver.com
`CLEANUP_*`	Image cleanup (KeepLast, MinAge, MinFree)	sensible defaults

Failure pattern: "depends on undefined service … invalid compose project"¶

If an in-app update fails with a message like

compose pull: exit status 15 (…
service "calserver-api-v2" depends on undefined service "report-runner":
invalid compose project)

the compose files on the host are stale: an older docker-compose.v2.yml still carries a depends_on: report-runner entry while report-runner is not defined anywhere in the updater's file list (it lives in a newer docker-compose.yml or a separate overlay). Docker Compose then rejects the entire project before a single image is pulled — every in-app update fails.

Current updater versions self-heal this: the updater detects exactly this error, writes a generated overlay .updater-compose-compat.yml into the deploy directory (resetting the affected depends_on entries via !reset) and retries. This is safe because the updater always runs pull and up scoped to the V2 services with --no-deps, so depends_on is never acted on. deploy.sh does not load the overlay. Once the host files are refreshed, the updater removes the overlay automatically.

A host that is already affected still runs the old, pinned updater — bring it up to date once manually (this also refreshes the compose files, fixing the root cause):

cd /var/calserver   # the host's deploy directory
./deploy.sh

Alternatively, to renew only the updater while it still answers: Administration -> Updates -> Updater container -> Update now (see Pinning the updater container version).

By hand on the host: set UPDATER_IMAGE_TAG in the .env to a current version (or release-latest), run docker pull calhelp/calserver-updater:<tag> and recreate the container on its own (docker compose -f docker-compose.yml -f docker-compose.v2.yml up -d --no-deps calserver-updater still fails on an affected host with the same error — in that case remove the stale depends_on: report-runner from the local docker-compose.v2.yml first, or simply use deploy.sh).

Manual update via deploy.sh¶

The classic path keeps working unchanged and is the fallback when no updater sidecar is enabled.

Prerequisites¶

Backup before every update

Always create a complete backup before performing an update. See Backup and Recovery for details.

SSH access to the server
Docker and Docker Compose installed
Sufficient storage space for the new image

Update Process¶

1. Create Backup¶

# Database backup
php artisan backup:run

# Back up DMS files
tar -czf /backup/dms-$(date +%Y%m%d).tar.gz /var/www/storage/app/dms

2. Download New Version¶

cd /opt/calserver-v2
git pull origin main

3. Run Deploy Script¶

The deploy.sh script automates the entire update process:

./deploy.sh

The script performs the following steps:

Step	Description
Build Docker images	Create new container images
Restart containers	Stop old containers, start new ones
Run migrations	Update database schema
Clear cache	Invalidate all caches
Restart queue	Restart worker processes

4. Manual Steps (if needed)¶

If the deploy script is not used, the steps can be performed manually:

# Rebuild and start Docker containers
docker compose build
docker compose up -d

# Run database migrations
docker compose exec app php artisan migrate --force

# Clear cache
docker compose exec app php artisan cache:clear
docker compose exec app php artisan config:clear
docker compose exec app php artisan route:clear
docker compose exec app php artisan view:clear

# Restart queue workers
docker compose exec app php artisan queue:restart

5. After the update: did the schema actually finish?¶

The API container's entrypoint runs the migrations on every start and lets the container come up even when they fail — leaving a site dark would be the worse outage. The price: a broken run is invisible until someone asks. And "broken" does not mean "one is missing", it means "everything from there on is missing" — migrate stops at the first migration that fails.

What follows shows up far from the cause: an admin screen whose table does not exist answers with a 500; a grid view can no longer be saved because a column is missing. The search then starts at the screen, not at a migration run three weeks ago.

deploy.sh checks this itself since 5.64 and reports outstanding migrations as WARNING: MIGRATIONS ARE PENDING, with the list. By hand:

docker exec calserver-api-v2 php artisan migrate:status --pending=1

Exit code 0 means nothing is outstanding. If a list comes back, the reason is in the container's start-up log:

docker logs calserver-api-v2 | grep -A5 'Running migrations'

The same state is in the UI under Administration > Operations > Database state. To catch up:

docker exec calserver-api-v2 php artisan migrate --force

If the run fails again with the same error, it is a data migration caught on this installation's data (typically a value hitting a UNIQUE index). The error names the table and the column — take that line to support instead of guessing.

Token for the report-runner (from report-runner 2.2.0)¶

deploy.sh handles this itself

When REPORT_RUNNER_TOKEN is empty or absent in the host .env, deploy.sh generates a random value and writes it there — the same variable serves runner and API. One task remains: copy the generated value from the .env into the password manager (Vaultwarden, item "calServer report-runner Token", one per installation). The deploy reports it in one line: Generated REPORT_RUNNER_TOKEN and saved to .env.

Without a token the report-runner will not start

The runner compiles caller-supplied report templates and holds the PDF signing key. From 2.2.0 it therefore requires a shared secret and refuses to start while neither a token nor an explicit opt-out is configured. Background: ADR-041.

How to recognise this case: a container that fails to start has no DNS name on the compose network. The API then reports not "token missing" but cURL error 6: Could not resolve host: report-runner — for reports as well as for the seal credential sync. The address is not the problem here; docker logs report-runner --tail 20 names the reason.

If you drive the stack by hand (without deploy.sh, e.g. on Kubernetes), set the value yourself:

# 1. Generate a token and store it in the password manager (Vaultwarden,
#    item "calServer report-runner Token", one per installation)
openssl rand -hex 24

# 2. Add it to the host .env — the same value applies to API and runner
#    REPORT_RUNNER_TOKEN=<generated value>

# 3. Restart the services. calserver (V1) belongs in that list: it renders
#    reports through the same runner once the use_report_runner flag is set,
#    and reads the token from the same variable.
docker compose up -d report-runner calserver-api-v2 calserver

# 4. Verify the runner is protected
docker compose exec report-runner wget -qO- http://127.0.0.1:8090/ready
#    expected: "authentication":"token"

If the change cannot be made right away, set REPORT_RUNNER_ALLOW_ANONYMOUS=true for the runner to keep the previous behaviour. That is a deferral, not a fix: while it holds, anyone who can reach port 8090 can have an arbitrary document signed with this installation's identity. In Docker Compose the port is limited to the compose network; in Kubernetes every pod in the namespace reaches it.

Pinning the updater container version¶

The updater is the service whose version genuinely needs holding on an installation. It holds the Docker socket and the deploy directory with write access: a regression in it does not break a feature, it breaks the path every other fix travels on. And it hangs off the moving release-latest tag — a docker compose pull on the host carries it along without anyone having triggered an update.

Use Administration -> Updates -> Updater container:

Pin to a version (e.g. v6.0.0, or a CI tag such as updater-a1b2c3d4e5f6). The installation keeps updating, the updater stays put.
Release drops the pin and the updater follows release-latest again.
Update now re-pulls the configured tag and swaps the container. Without a pin that fetches the newest release-latest; with a pin, a fix pushed under the same tag.

The way back is the point: a pin you can only set forces you to the host .env next time, and not everyone allowed to make that call has shell access.

The updater swaps itself

A compose up from inside the updater would kill it: Compose stops the old container first, and the old container is the process running Compose. So the updater starts a short-lived helper container (calserver-updater-self-apply) from the image it is currently running, which performs the swap from the outside and survives the death of its parent. The updater is unreachable for a few seconds during this; the page may show "state unknown" while it happens.

The helper is deliberately left behind so its logs are available:

docker logs calserver-updater-self-apply

Order: pull first, write second

The sidecar pulls the image before it touches the .env. A typo in the tag therefore fails without consequences (HTTP 502, nothing changed) instead of leaving a container that cannot find its image — which would cut the installation's only remote-maintenance path.

A regular update does not touch the updater

UPDATE_SERVICES never contains the updater, even if someone adds it there. An installation should not swap its remote-maintenance path as a side effect of updating the application. The updater is renewed through this card only (or through deploy.sh).

Requires the updater sidecar

The section only appears when UPDATER_TOKEN is set. Without the sidecar, UPDATER_IMAGE_TAG from the host .env applies and the page is read-only here.

The report-runner version¶

The runner moves with the API: both hang off the same image tag, so an update moves them together. That is the behaviour since ADR 2026-08-14 and closes the gap the outage of that day came from — before it, the runner hung off the V1 tag and could move on its own while its configuration stayed put.

Administration -> Updates -> Report service shows its state: is it running, and is it held at a pinned version? A dead service surfaces there before someone tries to print a report.

Holding it back is a manual edit of REPORT_RUNNER_TAG in the host .env — an intervention, not a routine action with a button. A runner on the wrong version breaks reports; annoying, but repairable. An updater on the wrong version makes the installation unreachable, which is why the button sits there instead.

After the Update¶

Post-update check

Call the health check endpoint to verify the system status:

GET /api/v2/health

Check the application in the browser for correct functionality

Review the logs for error messages:

docker compose exec app tail -f storage/logs/laravel.log

Rollback¶

If the update fails:

# Restore database backup
php artisan backup:restore --latest

# Start previous container version
docker compose down
git checkout <previous-version>
docker compose up -d

Caution

Rolling back database migrations can lead to data loss. When in doubt, use the database backup.