Share a quota across workers

This guide moves a valve's budget into Redis or Valkey, so every worker that calls a provider with the same credential shares one limit.

Before you beginCopy Anchor Link

• A Redis 6.2+ or Valkey 7.2+ server reachable from every worker.
• An ioredis or node-redis client. See Installation.
• The provider's documented quota for your credential.

Configure the valveCopy Anchor Link

import { Redis } from 'ioredis';
import { redisStore } from 'valvejs/redis';

export const redis = new Redis(process.env.REDIS_URL!);
export const store = redisStore({ client: redis });

import { createValve } from 'valvejs';

export const hubspot = createValve({
  id: 'hubspot:portal-123',
  rate: { limit: 190, interval: '10s' },
  concurrency: 10,
  store,
});

import { hubspot } from './valves';

const hubspotFetch = hubspot.wrapFetch();
const res = await hubspotFetch(
  'https://api.hubapi.com/crm/v3/objects/contacts',
);

process.on('SIGTERM', async () => {
  await hubspot.close({ drain: true });
  await redis.quit();
});

Redis key layoutCopy Anchor Link

Keys look like PREFIX:{ID}:.... The {ID} hash tag places every rule, the pause, and the leases for one valve in the same Redis Cluster slot, so one Lua script can update them atomically. For this reason, ids and prefixes must not contain braces.

Use a different prefix to separate environments that share a Redis server:

redisStore({ client: redis, prefix: `valve:${process.env.NODE_ENV}` });

Configuration driftCopy Anchor Link

The first store admission pins a versioned configuration fingerprint at PREFIX:{ID}:config. Every later admission checks it atomically before spending rate budget or creating leases, including while the quota is paused.

The fingerprint covers names and scope dimensions when configured, plus each normalized rule's algorithm, interval, limit, and burst, plus global concurrency scope, limit, and lease TTL. Rule order and equivalent durations such as '1s' and 1000 match. Local concurrency, queue settings, and lease owners can differ between workers.

When Redis reports a mismatch, the incompatible worker's jobs reject with INVALID_OPTIONS; its callbacks do not run. This applies to all onStoreError policies, including 'open' and 'wait'. Restore the pinned configuration to resume that worker. Compatible workers keep using the same budget during a rolling deployment.

An outage or timeout can hide Redis's mismatch response. With 'open', jobs then run without shared admission checks, so configuration compatibility and rate bounds are not guaranteed. Use 'wait' or 'throw' when these checks must hold.

The config key has no TTL. Expired rate state, released leases, resume(), and close() do not clear it. Valve leaves one config key per quota id. Keep it while that quota is in use, including during idle periods.

Reliable shared quota enforcement and mismatch detection require preserving Redis state. Use a non-evicting deployment, such as maxmemory-policy noeviction. An allkeys-* eviction policy can remove the config key despite its lack of a TTL. The next admission then pins its caller's fingerprint, allowing a worker with different settings to admit jobs without a mismatch error. Eviction can also remove active rate and lease state; volatile-* policies protect the config key but can evict those TTL keys.

Changing a shared quotaCopy Anchor Link

Intentional changes to rate rules or global concurrency require a coordinated migration. Before resetting the fingerprint:

1. Stop producers and drain every worker using this Redis database, prefix, and id with await valve.close({ drain: true }). Disable restarts of old workers and ensure all outbound jobs have actually finished, including jobs whose leases expired. A store pause alone does not stop already-issued grants.

2. Let outstanding Redis commands settle and close the old clients. Account for timed-out commands and offline queues as described in Timeouts and reconnection.

3. Wait until all old shared PREFIX:{ID}:r:* and scoped PREFIX:{ID}:s:* rate keys have expired and the provider's quota has fully recovered. GCRA debt can exceed one interval when burst > limit. Confirm that PREFIX:{ID}:leases is empty or expired and that no old jobs can start or continue. Leave any provider cooldown in place.

4. Delete only PREFIX:{ID}:config, using an administrative Redis connection. For example, with the default prefix and the id above:

   Copy Text

   redis-cli -u "$REDIS_URL" DEL 'valve:{hubspot:portal-123}:config'

5. Start all workers with the same new settings. The first admission pins the new fingerprint. Verify it by running a job and checking for INVALID_OPTIONS.

Do not delete the config key while workers are active. Changing the prefix or id creates an independent budget: old and new workers can exceed a provider quota when they overlap. A namespace change requires the same drain and recovery steps, or an external mechanism that enforces a common budget during migration.

Adding, renaming, or removing rulesCopy Anchor Link

Prepare the complete new policy before the coordinated migration above:

• Add a scoped rule: update every producer to supply its required identity at the cutover. For example, adding { name: 'recipientPacing', scope: 'recipient', limit: 1, interval: '6s' } requires scopes: { recipient: stableId } on every job. Stage the code behind a deployment gate, then switch policy and callers together after draining. Old policies reject extra dimensions; new policies reject missing ones.
• Rename a rule or dimension: change all worker configurations and, for a dimension rename, every caller's scopes property. A name selects state keys; a rename is not a cosmetic edit and does not transfer the old counter.
• Remove a rule: remove its identity from callers only if no remaining rule uses that dimension. Wait for the old quota to recover before starting the reduced policy; removing a rule does not cancel provider restrictions.
• Change an algorithm, interval, limit, or burst: use the same migration, even for a limit increase. Valve cannot infer whether the provider permits reinterpreting existing debt or counts under the new settings.

Routine rolling deployments with an unchanged normalized policy retain their budgets. Mixed-policy rolling deployments are unsupported: new workers receive fingerprint conflicts until the coordinated cutover. If uninterrupted processing is required during a policy change, use external coordination that enforces the common provider budget. Do not delete the fingerprint or switch IDs while old workers are still admitting jobs. Normal inactive-scope expiry needs no migration or administrative deletion.

Upgrading older Valve workersCopy Anchor Link

Older versions do not check the fingerprint. A rolling library upgrade with unchanged quota settings can retain existing counters and leases, but detection is complete only after all workers run a version with this guard. The first guarded admission pins its caller's configuration; it cannot infer the settings that older workers used. Keep settings identical throughout that upgrade. Use the coordinated migration above before changing them.

Redis ClusterCopy Anchor Link

Pass an ioredis Cluster instance, or a node-redis cluster client, as client. No other configuration is needed. The integration suite runs both client families against a three-primary cluster.

Verify it locallyCopy Anchor Link

Start Redis, build the workspace, and run the shared example. It creates six valve instances against one Redis budget.

docker compose -f compose.test.yml up -d
pnpm build
REDIS_URL=redis://127.0.0.1:6379 pnpm --filter @valve/example-shared start

Next stepsCopy Anchor Link

• Respect Retry-After across workers
• Redis store reference