Independent keyed groups
Give tenants, credentials, or models independent budgets and cooldowns with a bounded local cache.
Use createValveGroup when each tenant, provider credential, or model needs its
own policy, queue, concurrency, and feedback cooldown. Keys are independent;
a group does not spend budgets across keys atomically.
Choose a model
| Model | Use when | Admission guarantee |
|---|---|---|
| One valve with atomic scoped rules | One request spends a hard shared parent quota and a per-key quota. | All applicable budgets are spent together, or none. |
| Independent group | Keys have separate quotas, tiers, concurrency, or 429 cooldowns. | Each key admits independently. No shared parent budget. |
Sequential stages with runSequential | Separate budgets must admit in order and earlier charges should stand. | Earlier admissions stay spent if a later stage denies. |
For a hard parent quota plus a per-tenant quota on the same request, use
scoped rate limits. Neither a group nor
runSequential gives atomic admission across
separate Valve IDs.
Resolve tenant tiers
import { createValveGroup } from 'valvejs/group';
const tiers = new Map([
['tenant-42', { limit: 40, concurrency: 4 }],
['tenant-7', { limit: 10, concurrency: 2 }],
]);
const tierFor = (tenant: string) =>
tiers.get(tenant) ?? { limit: 5, concurrency: 1 };
const tenants = createValveGroup({
id: 'crm:tenants',
maxSize: 500,
idleTtl: '10m',
resolve: (tenant) => {
const tier = tierFor(tenant); // Synchronous application lookup.
return {
rate: { name: 'tenant', limit: tier.limit, interval: '1s' },
concurrency: tier.concurrency,
maxQueue: 100,
maxWait: '10s',
limits: 'managed',
};
},
});
try {
const result = await tenants.run('tenant-42', () => 'sent', {
priority: 5,
cost: 1,
signal: AbortSignal.timeout(5000),
});
console.log(result);
} finally {
await tenants.close({ drain: true });
}Import createValveGroup from valvejs/group. It is a separate entry point,
like valvejs/redis and valvejs/headers, and is not exported from valvejs.
resolve(key) returns ValveOptions without id or store. It runs
synchronously on first local use and again after eviction, never per job.
It may run before room is available to obtain the default maxWait, even if
the call later aborts or expires without creating a child. Concurrent calls
waiting for the same key share that configuration snapshot. A promise, or a result containing id or store,
is rejected with INVALID_OPTIONS. So is any call to the same group made
from inside resolve, option getters, or fetch option resolvers. These must
return configuration without calling back into the group. Options are copied when resolved; mutating the returned
object later has no effect. Load tier configuration before using the
group and keep rule structure consistent across workers.
Without store, the group creates one private memory store shared by its
children. It does not share state across processes.
Keep a member safely
get(key) validates the key and returns a ValveGroupMember without creating
a child. Keep the member or its wrappers for as long as needed:
import type { ValveGroup } from 'valvejs/group';
async function sendMessage(tenants: ValveGroup, message: string) {
const tenant = tenants.get('tenant-42');
const send = tenant.wrap((text: string) => Promise.resolve(text));
return send(message);
}Call this helper with an open group. Members expose key, id,
run, wrap, wrapFetch, observe, pause, resume, getLimits,
updateLimits, and stats(). They are not raw child Valve handles. Each call
pins its child until that call settles. After eviction, the next call
transparently creates a new child. Submission snapshots run options and scope
identities before waiting for room. Fetch option resolvers run once per call
before that wait; retries keep the resolved identities. stats() returns undefined when no local
child exists and does not create one.
Share provider credential budgets with Redis
Use opaque credential IDs as keys, never the credentials themselves. All
workers must use the same Redis namespace, group ID, and key mapping.
This example requires ioredis and REDIS_URL:
import { Redis } from 'ioredis';
import { createValveGroup } from 'valvejs/group';
import { redisStore } from 'valvejs/redis';
const redis = new Redis(process.env.REDIS_URL!, { enableOfflineQueue: false });
const store = redisStore({ client: redis });
const credentials = createValveGroup({
id: 'partner:credentials',
store,
resolve: () => ({
rate: { name: 'requests', limit: 20, interval: '1s' },
concurrency: 2,
limits: 'managed',
}),
});
try {
const limitedFetch = credentials.get('credential-42').wrapFetch();
const response = await limitedFetch('https://api.example.com/contacts', {
headers: { Authorization: `Bearer ${process.env.PARTNER_TOKEN!}` },
signal: AbortSignal.timeout(5000),
});
console.log(response.status);
await credentials.get('credential-42').updateLimits({
requests: { limit: 40 },
});
} finally {
await credentials.close({ drain: true });
await store.close();
await redis.quit();
}The same key on another worker shares its stored budget and pause. Distinct keys have independent queues, budgets, cooldowns, and concurrency. Numeric concurrency is local; use a global concurrency option when the key needs a shared lease limit. Queues and stats stay local, with no cross-worker fairness.
Close the group first, then the supplied store, then the Redis clients you own. A group never closes a supplied store. Its default private memory store is closed by the group. If a store is shared with other valves or groups, close all of them before closing it.
Wrap model endpoints
Only separate models when their provider quotas are independent. These rates are illustrative. If models share a quota pool, use one key for that pool.
import { createValveGroup } from 'valvejs/group';
type Model = 'small' | 'large';
const models = createValveGroup<Model>({
id: 'inference:account-42',
resolve: (model) => ({
rate: { limit: model === 'small' ? 20 : 5, interval: '1s' },
concurrency: 2,
}),
});
const largeFetch = models.get('large').wrapFetch(fetch, { retries: 0 });
const unsubscribe = models.on('pause', (event, key) => {
console.warn({ key, ...event });
});
try {
const response = await largeFetch('https://api.example.com/models/large', {
signal: AbortSignal.timeout(5000),
});
console.log(response.status);
} finally {
await models.close({ drain: true });
unsubscribe();
}Keep feedback on its key
A member's wrapFetch observes responses and applies 429 cooldowns only to
that key. Its retry behavior matches Valve.wrapFetch:
GET, HEAD, OPTIONS, PUT, and DELETE retry by default. POST and PATCH require
explicit opt-in after checking the provider's idempotency contract. Disabling
retries still applies feedback.
For a custom client, pass observe in that key's run options, or call the
member's observe(response). observe returns void; feedback writes happen
in the background. Do not wrap the same request in both group.run and the
member's wrapFetch, which would admit it twice.
The model example uses group.on('pause', (event, key) => ...) to listen to
all current and future children. It returns an unsubscribe function. Register
listeners before submitting work. pause and resume affect only
the member's ID, across workers sharing the store. Pauses do not cancel work
already admitted. The group cannot infer a provider's quota scope from a URL
or a response.
Update managed limits
With limits: 'managed', resolver limit and GCRA burst values seed only a
key the store does not know yet. Stored values remain authoritative after
local eviction. Use a member's getLimits() and updateLimits() to read and
change them. Each call needs the key's local child, so on a key with no child
it takes a cache slot like any other call and can reject with QUEUE_FULL when
every child is busy. Keep a little headroom in maxSize if limit changes must
succeed during saturation:
import type { ValveGroup } from 'valvejs/group';
async function upgradeCredential(credentials: ValveGroup, key: string) {
const credential = credentials.get(key);
const before = await credential.getLimits();
return credential.updateLimits(
{ requests: { limit: 60 } },
{ expect: { requests: { limit: before.rules.requests.limit } } },
);
}Call this helper with the Redis group above before shutdown. The update affects that
key on all workers and preserves spent budget. Changing the tier lookup alone
does not overwrite stored limits. Names, scopes, algorithms, intervals, and
global concurrency stay fixed. In immutable mode, workers resolving different
rules for the same key are rejected by the store with INVALID_OPTIONS, just
as with createValve. See Runtime limits.
Map IDs and protect keys
A child's ID is exactly `${id}:${encodeURIComponent(key)}`. For example,
crm:tenants and tenant-42 produce crm:tenants:tenant-42. Migrating from
hand-built IDs crm:tenants:${tenant} preserves stored state for plain keys
like this when the store namespace and rules match. Keys containing characters
that need encoding produce different IDs from an unencoded mapping.
The group ID must contain 1 to 198 characters without { or }. Keys must be
nonempty strings with valid URI encoding, and the derived child ID must fit
200 characters. Invalid keys reject with INVALID_OPTIONS; get() validates
synchronously, while run() rejects its promise. Encoding is injective and
contains no :, so different keys and group namespaces do not collide.
Hash long opaque keys yourself with a stable scheme on every worker:
import { createHash } from 'node:crypto';
const keyFor = (opaqueId: string) =>
createHash('sha256').update(opaqueId).digest('hex');Allow room for the 64-character hash and separator in the group ID. Keys appear in Redis key names. Encoding is not anonymization: use opaque non-secret IDs, not API tokens, emails, or phone numbers. For sensitive source values, use a keyed hash rather than a plain hash. Changing the mapping creates new budgets.
Bound local children and handle capacity
maxSize defaults to 1000 and bounds live local children, busy and idle. It
is a positive integer. At capacity, the least recently used idle child leaves
the cache at once and closes in the background, so the new key's child is
created immediately, including for maxWait: 0. Idle children that are not
paused go first. An idle child with a local paused estimate is evicted only
when no other idle child can make room; its shared pause stays in the store,
so the recreated child still honors it, and only local backoff counters reset.
Children with calls in progress or queued or running jobs are never evicted.
A child that is closing blocks only calls for its own key, which wait for the
close so a key never has two children. At most maxSize evicted children close
at once, so together with live children a group holds at most 2 × maxSize.
When that many closes are in flight, calls needing a new child wait for one.
A call that waits keeps its abort signal, and
the wait counts toward its maxWait (the run option, or the key's default). A
member's wrapFetch applies maxWait to each attempt, including retries after
a shared pause. A wait that uses the whole limit rejects with
MAX_WAIT_EXCEEDED, but a shorter wait does not shorten later attempts.
If every live child is busy, a call needing a new child rejects with
QUEUE_FULL. This is group capacity backpressure, separate from maxQueue,
which bounds each child's local job queue. Existing keys can still accept work
subject to their own queue bounds. Apply upstream backpressure rather than
retrying in a tight loop:
import { isValveError } from 'valvejs';
import type { ValveGroup } from 'valvejs/group';
async function tryTenant(tenants: ValveGroup, tenant: string) {
try {
return await tenants.run(tenant, () => 'sent');
} catch (error) {
if (isValveError(error) && error.code === 'QUEUE_FULL') {
return 'busy'; // Let the caller defer or reject the request.
}
throw error;
}
}maxPending defaults to 1000 and bounds calls waiting for a closing child,
whether their own key's or one freeing room. Overflow rejects with QUEUE_FULL, separately from each child's
maxQueue. Aborted or expired calls immediately leave this queue.
idleTtl defaults to '10m', measured after the last call settles. Idle
cleanup skips paused children until their pause ends, and children with work. Cleanup uses one unref'd timer.
Eviction frees local memory only: budgets, pauses, managed limits, and shared
cooldown remain in the store under its normal retention rules. Recreated
children continue that shared state; local stats and backoff counters reset.
group.stats() returns { size, active }: live children (not those still
closing) and children with a call in progress in this process. It is not a distributed total.
Shut down
close() stops admission, rejects queued jobs with CLOSED, closes children,
and waits for running calls to settle. close({ drain: true }) lets queued
work run first. Either way, calls still waiting for a closing child reject
with CLOSED, since no child has queued them yet. Repeated calls return the same promise. Later member operations
that need a child reject with CLOSED; retained members do not reopen the group.
await using is also supported and calls close() without draining.
For a caller-owned Redis store, follow the close order shown above. Stop incoming application work before draining so shutdown can finish.