Skip to main content
The API container runs prisma migrate deploy on boot. A migration that cannot apply to your data stops the container rather than warning, and every later boot stops at the same point, so the API never serves a schema that is partly migrated. Getting the stack back is a deliberate step, described in If a migration fails. This page states which versions upgrade directly, and how that claim is tested.

Supported upgrade baselines

1.0.0 is the first release, so it is also the oldest baseline the project verifies. Migrations are additive and forward only.

What the upgrade does

Migrations run in order against your existing rows. They never rewrite history:
  • Existing rows are preserved. The drill fails the build if any table’s row count changes across an upgrade.
  • _prisma_migrations records what has already been applied, so a restart never replays a migration.
  • An applied migration is never edited. A correction ships as a new forward migration.
A migration that adds an index builds it with a plain CREATE INDEX, which blocks writes to that table until it finishes. add_listing_market rebuilds the snapshot and change event indexes, and AppSnapshot is the largest table of an instance with a long history, so expect that step to take longer there. It runs inside prisma migrate deploy, before the API starts, so no request or job waits on it.

Before you upgrade

Take a backup. See Back up your data. A backup taken shortly before the daily pipeline window is the cheapest point, because the least work is in flight. Then pull the new image and restart. Watch the API container’s first log lines: prisma migrate deploy reports each migration it applies.

What a restart does to a job in flight

docker stop, docker compose restart and a host reboot all send SIGTERM. The API answers it by taking no further jobs and letting the job that is already running finish, then disconnecting from PostgreSQL and Redis. A running job is drained, not abandoned. PostgreSQL is disconnected last, once every request and job that was still running has finished, so neither loses its database half way. One stage is cut short rather than waited out. A job that has not yet been given a proxy endpoint stops waiting for one the moment the signal arrives, instead of sitting out the PROXY_ACQUIRE_TIMEOUT_MS budget, which is longer on its own than the whole shutdown allowance below. See Configuration reference. That job fails its attempt and the queue retries it with backoff. What is left to wait for is the store work itself. Compose allows the API 90 seconds through stop_grace_period, and only sends SIGKILL afterwards. The number comes from the slowest single job: a Google Play rank check at depth 200 is roughly eight sequential store requests under SCRAPE_GPLAY_RPM, so about a minute in the worst case. Docker’s ten second default is shorter than that, which is why the stack sets its own. While Redis cannot be reached the stop is shorter, because nothing that waits on Redis can finish. The API tries Redis three times, a third of a second apart, closes its queue and worker connections at once instead of draining them, and then gives requests that are still open two seconds before it cuts them. A short request that only needs PostgreSQL finishes in that time. Every request still open after those two seconds is cut, whatever it waits on: the queue dashboard, which waits on Redis, and also a long export or a refresh that is still reaching a store. A direct client sees a closed connection, and a browser using the web app sees its 502 answer. A database query that such a request left running is abandoned two seconds later and rolled back by PostgreSQL. The whole stop takes a few seconds, well inside Docker’s ten second default. With Redis answering nothing is cut and a request in flight completes. A job that still has not finished when the budget runs out is killed with the process. Nothing is lost, but the recovery is slower than a drain: the job keeps its lock until the lock expires, and the stalled job check on the next start returns it to the queue. A job that stalls a second time is failed rather than retried forever, so it appears in the failed list on the queue dashboard instead of cycling. You do not need to drain the queues by hand before an ordinary restart. You do need to for the release named below.

Drain the queues before upgrading to a tenancy-aware release

Every queued job now names the workspace that owns it. A job queued by an older release carries no workspace, so the new workers reject it as unrecoverable: it fails once, without retries, and appears in the failed list on the queue dashboard. Nothing is silently retried and no work runs against the wrong workspace. Let the queues empty before you swap the image. The cheapest window is well after the daily pipeline has finished, when the waiting and delayed counts on the queue dashboard are zero. See The queue dashboard and the OpenAPI surface for how to reach it. If you upgrade with jobs still queued, the affected jobs fail immediately and the next daily run schedules their replacements, so the only cost is one lost day for the drained jobs. Alert jobs are the exception worth watching: their source events may already be marked flushed, so a batch that fails this way is not sent again. Check the failed list after the first boot and clear it once you have read it.

Move WEB_PUBLIC_URL to the root .env

The Compose stacks pass WEB_PUBLIC_URL to the API from the root .env and default it to http://localhost:3001, so a value in apps/api/.env no longer reaches them. If you set it there, move the line to the root .env before you restart. Otherwise every email links to localhost, and the API logs a warning at startup that names the address. See Hosting behind TLS and a reverse proxy.

How the upgrade path is proven

Every pull request runs an upgrade drill in CI. It builds a database at the schema of the oldest supported baseline, loads a fixture carrying multi store and multi market data, applies every migration added since, and then asserts:
  • prisma migrate status reports the database up to date.
  • prisma migrate diff reports no drift between the migrations and the schema.
  • Per table row counts are identical before and after. A migration that resolved a collision by deleting rows fails here.
  • The API boots against the upgraded database and serves authenticated reads across apps, keywords, rankings, reviews, changes, actions and alerts.
  • Data retention prunes only what its windows allow, and never prunes open or snoozed actions by age.
The baseline is a variable, so the same drill covers a new baseline once one is declared. While the baseline tag is the current commit there is nothing to upgrade from, and the drill says so and skips:
The script refuses to run unless the database name ends in _upgrade, because it drops the public schema before it starts.

If a migration fails

Prisma does not run a migration inside a transaction. When a statement fails, every statement before it in the same migration stays applied, and the migration is recorded as failed in _prisma_migrations. prisma migrate deploy then refuses to apply anything while a failed migration is recorded, whichever image runs it. The API container exits on every boot, and starting the previous image tag fails at the same check, so changing the tag alone does not bring the stack back. The web app waits for a healthy API, so the site stays down until the API starts. When you run the published images, add -f docker-compose.pull.yml to every command below and keep ASOBEAST_IMAGE_TAG in the root .env, so each command runs the image you mean.

Read the failure

The first boot after the upgrade names the migration and the database error.
Every later boot reports P3009, migrate found failed migrations in the target database, and names the same migration. Stop the API so it stops restarting, then read the full error from the database, which keeps it after the container log is gone.

Return to the previous release

This is the recovery to choose when the error comes from the migration meeting your data, such as a constraint or a type that existing rows violate, and the safe choice whenever you are unsure. Follow Upgrade and roll back: switch back to the previous version, then restore the backup you took before the upgrade. The archive predates both the partial changes and the failed record, so nothing needs resolving afterwards. Report the failure with the migration name, the database error code and the error message.

Undo the migration by hand

Use this when the cause lies outside the migration, such as a full disk or a lost connection, and you want to retry the same release once it is fixed, or when you have no backup from before the upgrade. A data error fails again on the same release, so after undoing it start the previous version.
1

Back up the current state

Take a backup of the database as it is now, so a mistake in the next step can be undone. See Back up your data.
2

Reverse what the migration applied

Read the migration from the image of the release that failed.
The database error names a table, a column or a constraint. Find the statement that touches it: every statement before it is applied, and the failing one and everything after it never ran. Reverse the applied statements in the opposite order with psql. A statement that changed or removed rows, such as an UPDATE, a DELETE or a DROP, cannot be reversed this way, so restore the backup from before the upgrade instead. Without one, report the failure before you change anything else.
3

Mark the migration rolled back

This changes only the record in _prisma_migrations, never the schema. Without the previous step, the next attempt fails again on the first statement that already ran, for example with column already exists.
4

Start the API

Start the same release once the cause is fixed, and it applies the migration again from its first statement.
To start the previous version instead, set the previous ASOBEAST_IMAGE_TAG, or check out the previous revision and add --build to the command above. It boots against the schema it expects.
Last modified on October 28, 2026