Managed Valkey and Redis

The Valkey store works best with maxmemory-policy noeviction. An allkeys-* policy evicts live run journals and queues, so while maxmemory is above zero ValkeyStore refuses it at start and names the fix for the service its URL points at. A volatile-* policy evicts only keys with a TTL, and no key that keeps two workers off one run, or stops a duplicate run or schedule fire, carries one, so the store starts with a warning instead (what it can evict). That includes ElastiCache Serverless, which fixes volatile-lru.

Check with the URL the workers will use:

pyrula dev --check --store "$PYRULA_STORE_URL"
pyrula db check --store "$PYRULA_STORE_URL"   # the check a worker host runs

On a service that blocks CONFIG:

valkey-cli -u "$PYRULA_STORE_URL" INFO memory | grep -E 'maxmemory(_policy)?:'

To accept the risk of allkeys-*, set PYRULA_VALKEY_EVICTION_CHECK=warn (logs a warning) or off in the worker’s environment, or pass ValkeyStore(..., eviction_policy_check="warn"). off also silences the volatile-* warning.

If the store’s user may not run INFO, the check cannot read the policy. The store then logs one valkey_eviction_policy_unread warning at startup and keeps starting. When cluster is not set, the same warning also says cluster mode was not detected (pass cluster= to set it). Eviction can then drop run state without notice, so confirm maxmemory-policy is noeviction another way (the console, or CONFIG GET from an admin user), or allow INFO for the store’s user.

The store needs Valkey 7.2+ or Redis 6.0+ with SCRIPT LOAD and EVALSHA open to its user. Services that run it once eviction is set:

Service	Default	Settable to noeviction	Runs the store
AWS ElastiCache (node-based)	volatile-lru	Yes, custom parameter group	Yes
AWS ElastiCache Serverless	volatile-lru, fixed	No	runs; unverified on AWS
Amazon MemoryDB	noeviction	Yes, custom parameter group	Yes, single-Region; Multi-Region not tested
Google Memorystore for Valkey	volatile-lru	Yes, gcloud	Yes, cluster mode disabled
Google Memorystore for Redis, Redis Cluster	volatile-lru	Yes, gcloud	Yes
Azure Managed Redis	volatile-lru	Yes, az redisenterprise	Yes, NoCluster or one shard
Azure Cache for Redis (Basic, Standard, Premium)	volatile-lru	Yes, portal	supported, Redis 6.0, retiring 2028-09-30
Redis Cloud	volatile-lru	Yes, console	Yes, single shard
Upstash (and Fly.io Redis)	eviction off	Yes (default)	Not yet tested: not stock Redis
DigitalOcean	noeviction	Yes (default)	Yes
Aiven for Valkey	noeviction	Yes (default)	Yes, non-clustered plans
Render Key Value	allkeys-lru	Yes, dashboard or render kv update	Yes, on Valkey 8 and on legacy Redis 6
Railway	noeviction	Yes (default)	Yes
Heroku Key-Value Store	noeviction	Yes (default)	Premium and up; Mini no
Dragonfly	noeviction	Default, not settable	Yes, v2.0.2 (below)
Garnet	varies	varies	No (below)
KeyDB	varies	varies	Not tested (below)

The refusal recognises the service from the endpoint host. Memorystore and private IPs get the generic fix with a link here.

What a volatile policy can evict

Under memory pressure a volatile-* server evicts keys that have a TTL, and only those. The store’s keys:

Key	TTL	If evicted
Run lock, shard owners (shard_owners:*, one hash per workflow)	None: the deadline is in the value, read against the server clock	Not evictable
Active runs, queues, the journal and lifecycle of a live run	None	Not evictable
Idempotency claim, body hash and pending marker	None: deadlines in the shard’s idempotency_expiries set	Not evictable
Wake token lists (wake:*, push wake-ups)	None: at most 64 tokens per workflow and version, trimmed on every push; a retired workflow keeps its lists	Not evictable
Schedule fired slots	None: one set per schedule, each slot scored by its deadline	Not evictable
Finished-run history (journal, lifecycle, completed marker, chain record, result, signal buffers)	The retention window	The history goes before its retention ends. A resubmit under the same run_id without an idempotency key starts the run again
A run’s shard locator while its submit is in flight	Seconds	A submit retried in that window can place the run in another shard
A cancel or terminate sent before its run was submitted	Until the submit	The cancel is lost and the run starts
An update’s stored result	The result window	A redelivered update can be settled again. It is never applied twice
Reader heartbeats (cancel_on_reader_disconnect)	A few seconds	A run that someone is watching can be cancelled as if they had left
LLM rate limit buckets (rate_limit=LLMRate(...))	Until the bucket is full again, plus a second	The shared limit restarts full, so calls can burst past the provider’s rate limit and come back rate limited
The idempotency sweep’s gate	Up to 300 s	One extra sweep pass
cache_set entries	The caller’s TTL	A cache miss

Keys without a TTL are bounded by what deletes them:

• A shard owner is a field of its workflow’s owners hash, deleted when its worker stops and taken over when its deadline passes. A worker that crashed while serving a workflow name that no worker serves any more leaves one hash of at most shard_count fields for that name, a few dozen bytes each.
• A schedule’s fired set holds the slots of the last SCHEDULE_FIRE_TTL (300 s) and is deleted with the schedule, in the same call.
• Idempotency keys end at their window. Every write and every orphan scan deletes expired keys, a bounded number per call. Those only run for a workflow name a worker serves, so every name with idempotency keys is also registered in idempotency_names, with the highest shard count it was written with. A maintenance pass walks that registry, at most 64 units of work per pass, one pass per interval for the whole fleet (a gate key with a server-side expiry lets one worker in; the others skip with one command): every second while work remains, otherwise every 300 s. Each pass goes on from the name the last one finished. It prunes every shard each name was ever written in, so a retired or renamed workflow and the shards above a lowered shard_count are emptied within one idempotency window plus the passes it takes. A name with no keys left is dropped from the registry once no process has registered it for 10 minutes. The registry holds one entry per name in use.

Alarm on evicted_keys (INFO stats) either way: a non-zero count means finished history is going early.

Push wake-ups and the connection budget

An idle worker does not poll the server every 100 ms. When a run becomes claimable (a submit, a signal or update, a requeue, a lock released over a queued run) the call that made it claimable also pushes one token onto a short list for that workflow and version, and an idle worker blocks on those lists. One token wakes one worker, in the order the workers started waiting, so a submit does not wake the fleet. A claim that finds nothing also learns when a rate gate refills or a held lock’s deadline passes, and the worker looks again then.

A token is a hint, never the work. A lost token, a dead reader connection or a server that refuses blocking commands leaves the worker claiming: with a healthy reader an idle worker looks again at least every 10 s (plus or minus 20 %); with none it polls at claim_poll_interval (100 ms) as before. A worker started without a version takes any run, so it waits on every version’s list that is registered; a version another process registers is heard from the worker’s next wait.

What it costs, per process, on top of the command pool:

• standalone: one extra connection, the reader’s, which blocks and sends one command per 10 s while a worker waits, and nothing while none does;
• cluster: the same single connection, because every wake list is in one slot per namespace ({wake:<namespace>});
• one extra round trip on the call that queues a run (a pipeline of LPUSH and LTRIM per version), sent from the calling thread after the script commits.

200 worker processes therefore hold 200 more connections. Check that against the plan’s connection limit (below, for each service): a 256-connection cap leaves room for about 100 processes once each has its command pool.

An idle worker’s other standing cost is its shard leases: one script call per workflow name every shard_lease_interval (10 s), which checks the worker in, renews what it holds and takes or gives back shards toward an even share, whatever shard_count is.

The service has to allow BLPOP. A proxy that does not forward blocking commands, or an ACL user without +@blocking, makes the first wait log one WARNING and the process polls; grant the user +blpop (and +lpush +ltrim, which the push uses) to turn push on. The wake lists have no expiry; their length cap bounds them.

AWS ElastiCache

Default parameter groups use volatile-lru and can’t be edited. Create a group with noeviction and attach it. The family matches the engine: valkey7, valkey8, redis6.x or redis7.

aws elasticache create-cache-parameter-group \
  --cache-parameter-group-name pyrula-noeviction \
  --cache-parameter-group-family valkey8 --description "pyrula: noeviction"
aws elasticache modify-cache-parameter-group \
  --cache-parameter-group-name pyrula-noeviction \
  --parameter-name-values ParameterName=maxmemory-policy,ParameterValue=noeviction
aws elasticache modify-replication-group --replication-group-id <your-group> \
  --cache-parameter-group-name pyrula-noeviction --apply-immediately

In the console: ElastiCache, Parameter groups, Create, edit maxmemory-policy, then Modify the cluster and select the group. The change applies immediately. ElastiCache blocks CONFIG, so verify with INFO memory.

AWS ElastiCache Serverless

The service fixes maxmemory-policy at volatile-lru. The store starts with the same volatile-* warning as any other host, valkey_eviction_policy_risky, logged at each start without asking the server, and pyrula dev --check reports it as [warn]. Its fix line reads:

the service fixes maxmemory-policy at volatile-lru and it cannot be changed. Keep no data-storage maximum and alarm on evicted_keys (INFO stats) and BytesUsedForCache.

Serverless blocks FUNCTION and FCALL, which the store doesn’t use. Every Serverless cache is cluster-mode enabled, and the store builds the cluster client for a Serverless host, so cluster=True is not needed. A call that meets NOSCRIPT after a restart or scale event reloads the script.

The store has not run against a real Serverless cache. Its suite passes against a local Valkey set up like one (FUNCTION, FCALL and CONFIG disabled, volatile-lru, a 1 GiB limit), and a test fills such a server at 32 MB until it evicts and checks that no run lock, shard owner or dedupe key went. Still open: whether Serverless accepts a script that touches a key it did not declare in the same hash slot, what its INFO reports, and whether the script survives a scale event.

For production, prefer a node-based ElastiCache cluster, which can be set to noeviction.

Amazon MemoryDB

The default is noeviction; the refusal appears only when a custom parameter group changed it. Multi-Region clusters are untested. Set the policy back:

aws memorydb update-parameter-group --parameter-group-name <your-group> \
  --parameter-name-values ParameterName=maxmemory-policy,ParameterValue=noeviction

To move a cluster from a default group to a custom one: aws memorydb update-cluster --cluster-name <name> --parameter-group-name <your-group>.

Google Memorystore

All three products default to volatile-lru, and changes apply without a restart. Google documents only gcloud for this setting:

# Memorystore for Valkey
gcloud memorystore instances update <instance> --location=<region> \
  --update-engine-configs=maxmemory-policy=noeviction
# Memorystore for Redis
gcloud redis instances update <instance> --region=<region> \
  --update-redis-config maxmemory-policy=noeviction
# Memorystore for Redis Cluster
gcloud redis clusters update <cluster> --region=<region> \
  --update-redis-config maxmemory-policy=noeviction

Create Memorystore for Valkey instances with cluster mode disabled.

Azure Managed Redis

The default is volatile-lru. Set it on the database:

az redisenterprise database update --cluster-name <cache-name> \
  --resource-group <group> --eviction-policy NoEviction

Use the NoCluster policy or a single shard. Sharded Azure Managed Redis is untested.

Azure Cache for Redis

The Basic, Standard and Premium tiers run Redis 6.0, the oldest server the store is tested on. The default is volatile-lru. In the Azure portal, open the cache, then Settings, Advanced settings, set Maxmemory policy to noeviction, and select Save.

Microsoft is retiring the service on 2028-09-30 in favor of Azure Managed Redis.

Redis Cloud

The default is volatile-lru. In the Redis Cloud console, open the database, then Configuration. Select Edit, set Data eviction policy to No eviction, and select Save database. Use a single shard; sharded databases are untested.

Upstash

Eviction is off by default. If it was turned on, turn it off in the console (database, Configuration, Eviction) or through the API:

curl -X POST -u <email>:<api-key> \
  https://api.upstash.com/v2/redis/disable-eviction/<database-id>

Fly.io’s Redis is Upstash (fly-<name>.upstash.io): run fly redis update <name> and turn eviction off when it asks. If INFO reports Upstash’s optimistic-volatile policy, the store only warns.

Upstash implements the Redis protocol itself and the store has not been run against it; test on a staging database first.

DigitalOcean

The default is noeviction. To restore it, use the control panel (cluster, Settings, Eviction policy, Edit) or the API:

curl -X PUT -H "Authorization: Bearer $DIGITALOCEAN_TOKEN" \
  -H "Content-Type: application/json" -d '{"eviction_policy":"noeviction"}' \
  https://api.digitalocean.com/v2/databases/<cluster-uuid>/eviction_policy

Aiven

The default is noeviction. If it was changed:

avn service update <service> -c valkey_maxmemory_policy=noeviction

Render

New Key Value instances run Valkey 8 and legacy ones Redis 6; the store runs on both. The default is allkeys-lru, so the store refuses a new instance until you switch it:

render kv update <red-id-or-name> --memory-policy noeviction

Or change the policy in the dashboard, or set maxmemoryPolicy: noeviction in a Blueprint.

Railway

Railway’s Valkey and Redis templates default to noeviction. If a start command passes --maxmemory-policy, remove it, or on a running service use CONFIG SET maxmemory-policy noeviction then CONFIG REWRITE. The templates save snapshots only; add --appendonly yes to the start command so a restart keeps the last second of runs.

Heroku

Heroku Key-Value Store defaults to noeviction. If it was changed:

heroku redis:maxmemory <addon-name> --policy noeviction -a <app>

Use a Premium plan or above; Mini has no persistence.

Dragonfly

The store runs on Dragonfly v2.0.2 (tested with --proactor_threads=1 and no other flag). Dragonfly refuses a script that touches an undeclared key unless the script’s first line allows it, so the store’s script starts with this line:

--!df flags=allow-undeclared-keys

The store declares every key it can name before it calls a script. The rest are names a script builds from data it reads, such as a claim candidate’s lock, and each is built from a prefix declared in the same call, so it lives in the slot of the call’s other keys. The exception is the idempotency keys a script deletes when their deadline has passed: they are names stored in the shard’s expiries set, not built from a prefix, and live in the same slot as that set. Dragonfly does not see any of these as declared, which is why the line is needed.

Other servers read that line as a Lua comment, and Dragonfly needs no --default_lua_flags. Differences from Valkey:

• Dragonfly pauses other operations while a script with undeclared keys runs, so expect lower throughput (not measured).
• Dragonfly reports noeviction but rejects CONFIG SET maxmemory-policy.
• Durable restart is untested, so check its persistence settings.

Dragonfly Cloud has not been tested.

Garnet

The store does not run on Garnet, which cannot call stream commands from Lua. When Garnet answers the script commands with unknown command, the store names Garnet (from INFO server) and refuses to start. Use Valkey 7.2+ or Redis 6.0+.

KeyDB

KeyDB tracks Redis 6.2. The store is untested on it and doesn’t refuse it.

When the server refuses scripts

The store loads its script with SCRIPT LOAD at start and calls it with EVALSHA. If the user in the URL may not run them, the server answers NOPERM and the store says so. Grant the commands:

valkey-cli ACL SETUSER <user> +eval +evalsha +script|load

A script runs under its caller’s key patterns, so an ACL on ~<namespace>:* still confines the store to its namespace. On a managed service, give the user an access string or role that includes the commands. A server that answers INFO without maxmemory_policy gets a warning at start.

Self-run Valkey or Redis

valkey-cli CONFIG SET maxmemory-policy noeviction
valkey-cli CONFIG REWRITE   # keep it across restarts

Or put maxmemory-policy noeviction in valkey.conf, or pass --maxmemory-policy noeviction to valkey-server. Turn on persistence too (appendonly yes) (Production checklist).