Skip to content
Pyrula
Pre-release: the packages are not on PyPI yet, so pip install does not resolve. Build from source to try it today.

Postgres, not Redis

Pyrula does not run on Postgres alone today. Postgres is an archive for finished runs, and the live state (journals in flight, leases, timers, signals) lives in the store a worker claims from: SQLite on one host, or Valkey for anything larger (Choosing a store, Known limitations). Valkey is the production store. This page does not promise a Postgres-only live store.

Choosing a store lists SQLite as built for development, demos, CI and low traffic on one host. It fits a small deployment when all of these hold:

  • Every process that opens the file runs on one machine. The store refuses a second host, a network or shared filesystem, and Windows.
  • Traffic is low. Writes are one at a time; the page estimates a ceiling near 40 to 120 five-step runs a second on a network-attached disk, and says that figure is not yet measured.
  • You run on Linux. macOS does not flush the drive cache on fsync.
  • You are fine with finished runs kept 7 days by default (retention_seconds).

Several worker processes on the host share the file; the single-host Compose recipe scales them with --scale. A store is created on first open, and pyrula store backup writes a consistent copy while workers run. Copying the .db file alone is not a backup.

Terminal window
export PYRULA_STORE_URL=sqlite:///recipe.db
python -c "from pyrula.workflows import open_store; open_store()" # creates the file
pyrula store backup recipe-backup.db
pyrula store info

pyrula store info prints the size, retention, runs by status and the hosts that opened the file. There is no replication: a lost disk is lost runs back to the last backup.

Valkey is the production store. Move to it when you need a second host, when the single writer is the ceiling, or when you need replication. Valkey 7.2+ or Redis 6.0+ with maxmemory-policy noeviction is the supported store. Managed Valkey and Redis lists, service by service, whether the default policy can be set to noeviction and whether the store runs on it. From that table, as of this release:

ServiceRuns the store
Render Key ValueYes, on Valkey 8 and on legacy Redis 6; set noeviction in the dashboard or with render kv update
RailwayYes; noeviction is the default
DigitalOcean, Aiven for ValkeyYes; noeviction is the default (Aiven: non-clustered plans)
AWS ElastiCache, node-basedYes, with a custom parameter group
AWS ElastiCache ServerlessRuns; unverified on AWS, and its fixed volatile-lru policy only warns
Upstash, Fly.io RedisNot yet tested: not stock Redis

This page does not compare prices; check each provider’s current plans. Whatever you pick, check it from the machine the workers will run on:

Terminal window
pyrula db check --store "$PYRULA_STORE_URL"

With Valkey as the live store, you can add a Postgres archive so finished runs outlive Valkey’s retention (a day by default). ArchivingStore wraps a ValkeyStore with a PostgresArchive and archives a run’s stream when it reaches a terminal event. In-flight runs stay only in Valkey, so the archive needs Valkey and does not replace it (Deployment modes).

Install the extra (pip install "pyrula[postgres]"), then create the archive’s tables with pyrula db migrate before the first run is archived:

from pyrula.workflows import make_archiving_postgres
store = make_archiving_postgres(
dsn="postgresql://user:pass@localhost:5432/pyrula_events",
valkey_url="valkey://localhost:6379/0",
)

The pyrula worker --workflow entry point takes the archive from the environment: PYRULA_STORE_URL for Valkey and PYRULA_POSTGRES_DSN for Postgres 15+. PYRULA_POSTGRES_DSN is for the workflow entry point only; the agent worker does not read it (Deploy a worker). --postgres-dsn with a SQLite store is refused.

If you already run Postgres for your application, it is a good place for finished-run history and a poor place to hold in-flight runs today. The honest options are SQLite while one host is enough, a small managed Valkey beside your Postgres when it is not, and the archive if you need history to last.

View this page as Markdown