--- title: "Previews" description: "A running copy of every pull request, with its own database and bucket." url: https://docs.skiffr.findmalek.engineer/docs/previews markdown: https://docs.skiffr.findmalek.engineer/docs/previews.md --- # Previews Every pull request from this repository gets a preview of all apps on the self-hosted Dokploy: the landing page, the dashboard, the api and the docs (and the e2e playground once it has a Dockerfile). One sticky comment on the pull request lists the links, a test login and the extension build. Closing or merging the pull request deletes all of it. Nothing is created until the repo variable `PREVIEW_ENABLED` is `true`. Cleanup and the sweep do not check it, so turning previews off still removes what exists, and they run against the real Dokploy as soon as the secrets are set, even before previews are enabled. After merging, run the `preview-sweep` workflow once with `dry_run` and read what it would remove before you enable previews. Pull requests from forks and from dependabot get no preview: they cannot read the secrets. ## What a push does 1. **plan** asks turbo which packages the pull request changes (`turbo ls --affected`, dependents included). An app is built when it is in that list. A change to a shared package such as `@skiffr/orpc` or `@skiffr/db` rebuilds every app that uses it. If turbo fails or its output cannot be read, every app is built. 2. **build** builds the changed apps on GitHub runners with docker buildx and pushes `ghcr.io/findmalek/skiffr-:pr--`. It uses the same build args as the production build, so the layers and the turbo remote cache (`TURBO_API`, `TURBO_TEAM`, `TURBO_TOKEN`) that `main` already warmed are reused. 3. **dokploy** creates or updates the preview in the Dokploy project `skiffr-previews`: a bucket, a postgres, then the apps, the api first. An app that was not built runs the production image `ghcr.io/findmalek/skiffr-:main`, wired to the preview's own env. The images are env agnostic: the Next apps read `API_URL` per request, so the production image works behind a preview api. 4. it waits for the deploys, checks `/health` of the api (the database answers and `env` is `pr-`) and every other app, then edits the sticky comment and the GitHub deployment `preview-pr-`. A new push cancels the running one. An app that is behind `main` is not rebuilt: a preview mixes the pull request's changed apps with the latest production images of the rest. ## What a preview owns | What | Name | | --- | --- | | apps | `skiffr--pr-` at `https://pr--.` | | database | a Dokploy postgres `skiffr-db-pr-` on `pgvector/pgvector:pg17`, its own user and password, its own volume | | bucket | `skiffr-pr-` on the S3 api (the api also prefixes keys with `pr-/`) | | env | its own `BETTER_AUTH_SECRET`, `ADMIN_PASSWORD`, `DATABASE_URL`, urls, `DEPLOY_ENV=pr-` | The api migrates the database when it starts and creates the admin from `ADMIN_EMAIL` and `ADMIN_PASSWORD`. A dedicated postgres per preview needs nothing but the Dokploy api to create and delete, and removing it frees the data. A shared postgres server with a database per preview would be lighter, but creating and dropping databases needs a connection from the runner, and the server is not reachable from GitHub. Nothing from production is shared. Previews get no model key, no Google client and no GitHub OAuth client, so the AI, Gmail import and GitHub sign in stay off. The job source and Gmail schedulers are off. The optional secret `PREVIEW_GROQ_API_KEY` (a key with its own small budget, never the production key) turns the model on with `AI_DAILY_CALLS=20` and `AI_GLOBAL_DAILY_CALLS=200`. `DEPLOY_ENV=pr-` makes the web and docs apps answer `noindex`. `EXTENSION_ORIGINS` is not set on previews: an unpacked extension has no stable id, so any extension origin may call the preview api with a token. ## The seeder The preview api seeds its empty database on start. It sets `SEED=preview` when `packages/env/src/api.ts` accepts that value, and `SEED=demo` until then. The contract for the seeder is: `SEED=preview` seeds a realistic, deterministic dataset into an empty database only, never touches a database that already has data, and runs inside the api start (after the migrations and `ensureAdmin`). The test login in the comment is the env admin. Any extra seeded accounts belong in the seeder's own documentation. ## The extension The comment links the CI run that holds the artifact `extension-chrome-pr-` (built by the `extension` job of every pull request). Unzip it twice, load the folder unpacked, set the api url in the options to the preview api (`https://pr--api.`), then create a token in the preview dashboard under Extension and paste it. The extension has no build time default for the api url. ## Cleanup - `preview-cleanup` runs when a pull request is closed or merged. It deletes the apps (their domains go with them), the postgres and its volume, the bucket with its objects, the `pr--*` image tags on ghcr (best effort, a failure only leaves registry clutter), and marks the GitHub deployments inactive. It is safe to run twice and a cleanup in progress is never cancelled. A bucket that holds an object key with a dot segment (`..` or `.`) is refused and stays, which turns the nightly sweep red until someone empties and deletes it by hand. A failed `up` leaves its apps and postgres in place, they count toward `PREVIEW_MAX` until the pull request closes or the idle limit hits. - `preview-sweep` runs daily, or by hand with `dry_run` to only list. It removes previews of closed pull requests the cleanup missed, previews idle for more than `PREVIEW_TTL_DAYS` (default 7) even when the pull request is open (a push brings it back), and the leftover postgres volumes and buckets of those pull requests. A github answer other than closed or 404 (an outage, a rate limit) stops the sweep with an error instead of deleting anything. Dokploy's `postgres.remove` leaves the data volume behind, so both paths remove it through `dockerVolume.removeVolume`. - at most `PREVIEW_MAX` previews (default 6) run at once. A new one over the limit is skipped, the comment says so, and it starts on the next push once there is room. Image layers of old previews stay in the box's docker cache: turn on Dokploy's daily docker cleanup in Settings, Server. Both scripts take `--dry-run`: ```bash DOKPLOY_URL=... DOKPLOY_TOKEN=... GITHUB_TOKEN=... GITHUB_REPOSITORY=FindMalek/skiffr \ node .github/scripts/preview.mjs sweep --dry-run ``` ## One-time setup DNS, one record: `*.preview.skiffr` CNAME to `home.findmalek.engineer`, TTL 60. Every preview host is one label under it (`pr-12-web.preview.skiffr.findmalek.engineer`), so a single wildcard certificate covers all of them. Without it each host orders its own Let's Encrypt certificate, and the weekly limit of 50 certificates per domain is reached after a few pull requests. Certificate, once, on the Dokploy box: put this in `/etc/dokploy/traefik/dynamic/preview-skiffr-wildcard.yml`. Traefik orders the wildcard certificate with the DNS challenge resolver `vercel` and every preview router reuses it. ```yaml http: routers: preview-skiffr-wildcard-cert: rule: Host(`preview.skiffr.findmalek.engineer`) entryPoints: [websecure] service: noop@internal tls: certResolver: vercel domains: - main: "*.preview.skiffr.findmalek.engineer" ``` Preview domains are created with `certificateType: custom` and `customCertResolver: vercel` (the variable `PREVIEW_CERT_RESOLVER`; `letsencrypt` uses the HTTP challenge and one certificate per host instead). S3, once: create a key that may create buckets and nothing else, so the runner needs no admin access. On Garage: `garage key create skiffr-preview-key`, then `garage key allow --create-bucket skiffr-preview-key`. A key owns the buckets it creates and can empty and delete them. | Where | Name | Value | | --- | --- | --- | | variable | `PREVIEW_ENABLED` | `true` | | variable | `PREVIEW_DOMAIN` | `preview.skiffr.findmalek.engineer` | | variable | `DOKPLOY_URL`, `PRODUCTION_DOMAIN` | the same as for the production deploy | | secret | `DOKPLOY_TOKEN` | the same Dokploy API key | | variable, optional | `PREVIEW_S3_ENDPOINT`, `PREVIEW_S3_REGION` | the S3 api (`https://s3.findmalek.engineer`) and its region (`garage` unless configured). Without them previews have no uploads | | secret, optional | `PREVIEW_S3_ACCESS_KEY_ID`, `PREVIEW_S3_SECRET_ACCESS_KEY` | the bucket creating key | | variable, optional | `PREVIEW_MAX`, `PREVIEW_TTL_DAYS`, `PREVIEW_CERT_RESOLVER` | 6, 7 and `vercel` | | variable, optional | `TURBO_API`, `TURBO_TEAM` | the turbo remote cache for the builds | | secret, optional | `TURBO_TOKEN`, `GHCR_READ_TOKEN`, `PREVIEW_GROQ_API_KEY` | the cache token, a `read:packages` token (ghcr packages are private unless made public), and a model key with its own budget | Never log Dokploy into ghcr with a workflow token: it expires with the run and every later pull on the box fails. `GHCR_READ_TOKEN` is the only registry login the scripts send. ## Limits - previews are public: anyone with a link can sign in with the test login. Only same-repo pull requests get one, the data is fake and the model and mail are off, but do not put anything real in a preview. - one S3 key owns all preview buckets, and a bucket has no quota. The daily sweep and the idle limit bound the space. There is no isolation between previews: the key sits in every preview api env, so pull request code running in one preview can read, write and delete the buckets of all the others. - the previews are public on the internet with a published test login, and each one runs five containers with no resource limits. - the preview of an app that was not rebuilt runs the latest `main` image, which can be newer than the pull request's base. - the `web` and `docs` builds bake the production site url into their metadata. The previews tell search engines not to index. ## Threat model Under `pull_request` the workflow file comes from the merge ref, so anyone who can push a branch to this repository can change the workflow or the scripts and read `DOKPLOY_TOKEN`, `PREVIEW_S3_*` and `GHCR_READ_TOKEN`, whatever the checkout ref is. Checking out `main` for the scripts does not change that, and `pull_request_target` would be worse. Forks and dependabot are skipped, and nothing a branch name or title carries reaches a shell. - `DOKPLOY_TOKEN` is a full admin key, the same power as production. Whoever can push a branch can delete `skiffr-db`. If Dokploy lets you create a key scoped to the `skiffr-previews` project, use that one. Otherwise keep write access to the repository tight. - all previews share one S3 key (see the limits) and are public.