sections.operations
Backup and restore
Nightly retention sweep, backup scripts under `scripts/backup/`, and a documented restore path for Mongo, Postgres, and MinIO.
Backup and restore
Skrum is the all-in-one AI project control plane — team chat, tasks and sprints, native video meetings, and inbound code activity in one workspace, with every AI action waiting for a human yes.
Backup is the operator's responsibility. Skrum ships scripts and a retention sweep — you supply the schedule, off-site destination, monitoring, and recovery drills.
Recovery objectives and schedule
Choose objectives from your own data volume and business requirements; the scripts cannot promise a universal recovery time.
- Recommended cadence: one complete backup every night, with a quarterly restore drill. Workspaces that cannot tolerate a day of lost writes should schedule every six hours or more often.
- RPO: at most the interval between successful complete backup sets plus the duration of the interrupted run. A nightly schedule therefore targets an RPO of no more than 24 hours only when every run is monitored and completes.
- RTO: establish it from a timed restore of production-sized data. A practical initial operating target is four hours, but the measured drill result is authoritative.
- Retention: as a starting policy, retain 7 daily, 4 weekly, and 12 monthly complete sets. Apply bucket lifecycle/immutability controls outside Skrum and keep at least one copy outside the application host and primary object store.
Alert on a missed schedule, non-zero exit, absent MANIFEST.txt, or failed off-site replication. Never delete the previous known-good set until the new set and its checksum sidecars are present.
Create a backup
./scripts/backup/backup-all.sh
The scripts read only backup-related values from the authoritative repository-root .env when those variables are absent from the invoking process. Explicit process variables — including an explicitly empty value — take precedence. The parser does not source or evaluate the file, and no credential value is printed.
Required values are MONGODB_URI, DATABASE_URL, S3_ENDPOINT, S3_ACCESS_KEY, S3_SECRET_KEY, and S3_BUCKET. BACKUP_TARGET=local writes the timestamped set under BACKUP_DIR; BACKUP_TARGET=s3 uploads a hashed bundle under BACKUP_S3_PREFIX and prints its restorable s3:// URI. The orchestrator hashes the Mongo, Postgres, and MinIO archives plus the complete manifest before any restore is allowed.
Mongo, Postgres, and object storage are dumped sequentially. For a transactionally quiet recovery point, stop incoming writes and pause workers for the short backup window; otherwise records created across stores during the run may land on different sides of the recovery point. Do not stop the database or object-store services themselves.
Example scheduler entry for the nightly recommendation, run from the repository root:
15 2 * * * cd /srv/skrum && ./scripts/backup/backup-all.sh >> /var/log/skrum-backup.log 2>&1
Use a scheduler/secret manager appropriate to your platform and restrict backup directories, logs, and root .env to the operator account.
Retention sweep
A daily retention-sweep BullMQ job archives prune-eligible messages, task comments, and work-graph events losslessly to cold storage under archive/{accountId}/{kind}/{sha256}.ndjson.gz before deletion. Retention days per tier follow historyDays (30 / 90 / 365 / unlimited). Unlimited tiers no-op.
Restore expectations
Restore is destructive: Mongo uses --drop, Postgres uses --clean --if-exists, and object storage mirrors with removal. Schedule a maintenance window, stop API/worker writes, take a safety copy of the current stores when possible, and confirm the destination identifiers before proceeding.
./scripts/backup/restore-all.sh <BACKUP_SET_DIR_OR_S3_URI>
The command validates the complete manifest and every archive checksum before mutating any datastore, then restores Mongo, Postgres, and MinIO. A successful command means the archives were accepted by their restore tools; it does not by itself prove application-level recovery.
Before reopening writes:
- Boot the API and worker against the restored stores and wait for health checks.
- Verify sign-in, workspace/project/task/message reads, and one known uploaded-file download.
- Confirm queue depth, migration state, and recent audit/work-graph records.
- Record start/end time, restored set URI, byte counts, and smoke-test result in the recovery log.
Keep the system in maintenance mode and investigate if any check fails. A quarterly drill should restore into an isolated environment, prove seeded Mongo/Postgres/object data by content, and update the measured RTO.
FAQ
- Managed hosting? Backups are our responsibility on the managed tier.
- Can I override the retention window? Yes —
retention_policiesper account clamps downward only. - Does application retention replace disaster-recovery backup retention? No. The retention sweep governs product records; complete backup-set lifecycle is an operator policy.
See also: Skrum documentation