@studnicky/cache
Capacity-bounded LRU cache with per-entry and default TTL, O(1) promotion on read.
What it is
@studnicky/cache is a composable in-process retention primitive: a bounded LRU store with TTL and stale thresholds, plus keyed memoization with single-flight coordination. It holds derived values close to one process; it does not become a database, distributed lock, or application cache product.
What it is for
Use it when a Northstar Books process needs to reuse a recently computed catalogue projection, protect an expensive lookup from concurrent duplicate work, or bound memory while keeping freshness policy explicit. The consuming application owns keys, invalidation, authoritative writes, and cross-process coordination.
Northstar Books examples
- Observed memoization — cache and single-flight lifecycle maps concurrent requests for the same book-order summary to one loader invocation. It proves that Northstar can reuse a successful result, coalesce matching in-flight requests, and observe hits, misses, and coalescing without making memoization an order system.
- Observed cache — lifecycle hook trace maps a small title-card cache to catalogue browsing. It proves the lifecycle events Northstar can emit when a card is stored, read, replaced, evicted, expired, deleted, or cleared while the catalogue remains authoritative elsewhere.
Install
pnpm add @studnicky/cacheRequires @studnicky:registry=https://npm.pkg.github.com in .npmrc.
@studnicky/cache/node exports runtime cache operations, @studnicky/cache/memoize exports keyed memoization, and schemas remain available from @studnicky/cache/entities.
Northstar Books catalogue reads
Northstar serves a frequently viewed title catalogue without treating an in-process cache as the source of truth. Cache a read model by stable catalogue query or title identifier, choose capacity and freshness bounds for that response, and refill it from the authoritative store after a miss. LruCache guarantees bounded memory, least-recently-used eviction, and lazy TTL removal; callers retain invalidation and cross-process coherence policy.
Usage
Northstar Books wants to keep a handful of hot lookups — like a shopper's running order score — warm without letting memory grow without bound. Spin up an LruCache with a fixed capacity, then work with it like a slightly smarter Map: set a value in, get it back out, check it with has, delete a single entry, or clear the whole thing when you're done.
import { LruCache } from '../src/index.js';
const cache = LruCache.create<string, number>({ 'capacity': 10 });
// set and get a value
cache.set('score', 99);
console.log('get score:', cache.get('score'));
// has returns true for a stored key
console.log('has score:', cache.has('score'));
// size reflects the number of entries
cache.set('level', 3);
console.log('size after two sets:', cache.size);
// delete removes the entry and returns true
const deleted = cache.delete('score');
console.log('deleted score:', deleted);
console.log('has score after delete:', cache.has('score'));
console.log('get score after delete:', cache.get('score'));
// size decrements after delete
console.log('size after delete:', cache.size);
// clear removes all entries
cache.clear();
console.log('size after clear:', cache.size);
console.log('has level after clear:', cache.has('level'));Keyed memoization
Memoize composes cache retention with keyed single-flight execution. Supply the callback, cache capacity and TTL policy, and an explicit key derivation function. Successful results, including undefined, are cached; rejected calls are not. Concurrent calls for one derived key share the in-flight callback.
Memoize does not define application identity or persistence policy. The caller chooses a key whose result identity matches the callback’s semantics.
import { Memoize } from "@studnicky/cache/memoize";
const loadOrder = Memoize.create(
async (orderId: string) => fetchOrder(orderId),
{ capacity: 100, ttlMs: 30_000 },
{ keyDeriver: (orderId: string): string => orderId },
);
const order = await loadOrder.call("order-42");Two checkout pages can easily ask for the same order summary within milliseconds of each other, and Northstar doesn't want to run that expensive lookup twice. invalidate(...args) evicts one derived key when the underlying order changes, while clear() wipes every cached result at once. The example below calls a loader back-to-back for order-42: the first call misses and fetches, the second call hits the cache for free, and after an explicit invalidate the loader runs again — all observable through the onMemoHit/onMemoMiss/onMemoCoalesced hooks without wiring in a logger.
LRU eviction
Northstar only keeps its two hottest catalogue records warm at a time, so when a third title shows up something has to go. Below, Clean Code and Design Patterns are cached first; a checkout lookup then reads Clean Code, which promotes it to most-recently-used. When Domain-Driven Design arrives and pushes the cache over capacity, Design Patterns — now the least-recently-used of the three — is the one evicted, not Clean Code:
import { LruCache } from '../src/index.js';
const cache = LruCache.create<string, string>({ 'capacity': 2 });
// Northstar Books keeps its hottest catalogue records in a bounded cache.
cache.set('978-0132350884', 'Clean Code');
cache.set('978-0201633610', 'Design Patterns');
// A checkout lookup for Clean Code promotes that catalogue record to MRU.
const cleanCodeBeforeEviction = cache.get('978-0132350884');
console.log('catalogue hit for Clean Code:', cleanCodeBeforeEviction);
// Reading a third title evicts the least recently used catalogue record.
cache.set('978-0321125217', 'Domain-Driven Design');
const cleanCodeAfter = cache.get('978-0132350884');
const domainDrivenDesignAfter = cache.get('978-0321125217');
const designPatternsAfter = cache.get('978-0201633610');
console.log('cached Clean Code:', cleanCodeAfter);
console.log('cached Domain-Driven Design:', domainDrivenDesignAfter);
console.log('evicted Design Patterns:', designPatternsAfter);
console.log('catalogue size:', cache.size);TTL expiry
Some values — a short-lived auth token, say — shouldn't outlive their welcome even if nobody ever deletes them. Pass ttlMs when setting a value and the cache treats it as expired once that window passes, though eviction is lazy: the entry isn't actually swept out until the next get or has touches it. Below, a token cached with a 10ms TTL reads back fine immediately, but after waiting past that window the very same get call returns undefined and quietly removes the stale entry on its way out:
import { LruCache } from '../src/index.js';
const cache = LruCache.create<string, string>({ 'capacity': 10, 'ttlMs': 10 });
cache.set('token', 'abc123');
// Before expiry, the entry is present
const beforeExpiry = cache.get('token');
console.log('get token before expiry:', beforeExpiry);
console.log('has token before expiry:', cache.has('token'));
// Wait 50ms — the 10ms TTL will have elapsed
await new Promise<void>((resolve) => { setTimeout(resolve, 50); });
// After expiry, get returns undefined (lazy eviction on access)
const afterExpiry = cache.get('token');
console.log('get token after expiry:', afterExpiry);
console.log('has token after expiry:', cache.has('token'));Local replay and single-flight
A shopper double-clicks "place order," or a flaky network makes the browser retry a checkout request that already went through — Northstar needs to recognize "I've seen this exact request before" and hand back the original result instead of charging twice. The recipe below composes LruCache with Coalesce at the application boundary: it validates the incoming JSON payload once, snapshots it immutably, and uses deep structural equality to decide whether a replay matches the original request or is a different payload wearing the same key, which it rejects outright. Concurrent calls for the same key share one in-flight execution rather than racing each other, and a loader that legitimately resolves to undefined still replays as undefined instead of looking like a miss.
This is process-local coordination only — it does not survive a process restart, coordinate across multiple servers, or substitute for an authoritative write boundary that actually guarantees the charge happened once.
class IdempotencyReplay {
static readonly #replayCache = LruCache.create<string, ReplayEntryInterface>({
'capacity': 100,
'ttlMs': 60_000
});
static readonly #coalesce = Coalesce.create<ReplayEntryInterface>();
static readonly #inFlightPayloads = new Map<string, InFlightPayloadInterface>();
static async execute(
key: string,
payload: unknown,
operation: () => Promise<unknown>
): Promise<unknown> {
if (!JsonValue.is(payload)) {
throw new TypeError('idempotency payload must be finite, acyclic JSON');
}
const payloadSnapshot = ImmutableSnapshot.from(payload);
const cached = IdempotencyReplay.#replayCache.tryGet(key);
if (cached.found) {
const cachedEntry = cached.value;
if (cachedEntry === undefined) {
throw new Error(`idempotency cache entry for "${key}" is missing`);
}
IdempotencyReplay.requireMatchingPayload(key, cachedEntry.payload, payloadSnapshot);
return cachedEntry.result;
}
const inFlight = IdempotencyReplay.#inFlightPayloads.get(key);
if (inFlight !== undefined) {
IdempotencyReplay.requireMatchingPayload(key, inFlight.payload, payloadSnapshot);
} else {
IdempotencyReplay.#inFlightPayloads.set(key, { 'payload': payloadSnapshot });
}
try {
const entry = await IdempotencyReplay.#coalesce.run(
key,
async (): Promise<ReplayEntryInterface> => {
const result = await operation();
const completedEntry: ReplayEntryInterface = { 'payload': payloadSnapshot, 'result': result };
IdempotencyReplay.#replayCache.set(key, completedEntry);
return completedEntry;
}
);
IdempotencyReplay.requireMatchingPayload(key, entry.payload, payloadSnapshot);
return entry.result;
} finally {
if (inFlight === undefined) {
IdempotencyReplay.#inFlightPayloads.delete(key);
}
}
}
private static requireMatchingPayload(
key: string,
expectedPayload: unknown,
actualPayload: unknown
): void {
if (!Predicates.areDeeplyEqual(expectedPayload, actualPayload)) {
throw new Error(
`idempotency key "${key}" is already associated with a different JSON payload`
);
}
}
}Deterministic cache time
LruCache.create takes schema-validated config as its first argument and a LruCacheCollaboratorsInterface as its second. The collaborator's clock measures both TTL expiry and the soft staleMs threshold, so virtual time tests do not wait for wall time.
import { VirtualClockProvider, VirtualTimeCounter } from "@studnicky/clock/node";
import { LruCache } from "@studnicky/cache/node";
const counter = VirtualTimeCounter.create({ startMs: 0 });
const cache = LruCache.create<string, string>(
{ capacity: 10, ttlMs: 1_000 },
{ clock: VirtualClockProvider.create(counter) },
);Try it
Lifecycle hooks
Say Northstar's platform team wants to watch a cache's behavior in production — every hit, miss, and eviction — without baking a logging dependency into LruCache itself. TracingCache below subclasses LruCache and overrides all eight lifecycle hooks (onHit, onMiss, onSet, onUpdate, onEvict, onExpire, onDelete, onClear) to record each event. With capacity pinned to 2, trace the exact sequence as it happens: two sets, a hit, an update, an eviction when a third key arrives, a miss for the now-evicted key, a delete, another set, and a clear — plus a second scenario proving that an expiring TTL entry fires onExpire before it ever reports as a miss.
Observability hooks
LruCache exposes protected lifecycle hooks that a subclass can override to add logging, timing, or metrics without any changes to the caller. The base class never calls any logger or metrics library. All hooks are no-ops by default.
| Hook | When it fires | Args |
|---|---|---|
onHit(key, value) | get() finds a live, non-expired entry | key: K, value: V |
onStale(key, value) | get() finds a live entry past its staleMs threshold | key: K, value: V |
onMiss(key) | get() returns undefined (key absent or entry expired) | key: K |
onSet(key) | set() inserts a new key | key: K |
onUpdate(key) | set() overwrites a value for an existing key | key: K |
onEvict(key, reason) | An entry is removed to make room at capacity | key: K, reason: 'capacity' |
onExpire(key) | get() or has() encounters an entry past its TTL and lazily removes it — fires before onMiss | key: K |
onDelete(key) | delete() removes an entry that existed — not called for absent keys | key: K |
onClear(count) | clear() empties the cache | count: number (entries present before wipe) |
import { EventRecorder } from '@studnicky/errors/node';
import assert from 'node:assert/strict';
import { LruCache } from '../src/index.js';
class TracingCache extends LruCache<string, number> {
readonly #recorder = new EventRecorder<{ 'event': string; 'key'?: string }>();
get events(): readonly { 'event': string; 'key'?: string }[] { return this.#recorder.events; }
constructor(options: { 'capacity': number; 'ttlMs'?: number }) {
super(options);
}
protected override onHit(key: string, value: number): void {
this.#recorder.record({ 'event': 'hit', 'key': key }, `[cache] hit key=${key} value=${value}`);
}
protected override onMiss(key: string): void {
this.#recorder.record({ 'event': 'miss', 'key': key }, `[cache] miss key=${key}`);
}
protected override onSet(key: string): void {
this.#recorder.record({ 'event': 'set', 'key': key }, `[cache] set key=${key}`);
}
protected override onUpdate(key: string): void {
this.#recorder.record({ 'event': 'update', 'key': key }, `[cache] update key=${key}`);
}
protected override onEvict(key: string, reason: 'capacity'): void {
this.#recorder.record({ 'event': 'evict', 'key': key }, `[cache] evict key=${key} reason=${reason}`);
}
protected override onExpire(key: string): void {
this.#recorder.record({ 'event': 'expire', 'key': key }, `[cache] expire key=${key}`);
}
protected override onDelete(key: string): void {
this.#recorder.record({ 'event': 'delete', 'key': key }, `[cache] delete key=${key}`);
}
protected override onClear(count: number): void {
this.#recorder.record({ 'event': 'clear' }, `[cache] clear count=${count}`);
}
eventNames(): string[] {
const result: string[] = [];
const length = this.events.length;
for (let index = 0; index < length; index += 1) {
const event = this.events[index]!;
result.push(event.event);
}
return result;
}
}
// Capacity-2 cache; demonstrates set, hit, miss, update, evict
const cache = new TracingCache({ 'capacity': 2, 'ttlMs': 5_000 });
cache.set('a', 1); // onSet(a)
cache.set('b', 2); // onSet(b)
cache.get('a'); // onHit(a, 1)
cache.set('a', 99); // onUpdate(a)
cache.set('c', 3); // onEvict(b, capacity) then onSet(c)
cache.get('b'); // onMiss(b) — evicted
cache.delete('c'); // onDelete(c)
cache.set('d', 4); // onSet(d)
cache.clear(); // onClear(2)
// TTL expiry scenario
const ttlCache = new TracingCache({ 'capacity': 10 });
ttlCache.set('ttl-key', 7, { 'ttlMs': 1 }); // 1 ms TTL
await new Promise<void>((resolve) => { setTimeout(resolve, 5); });
ttlCache.get('ttl-key'); // onExpire then onMissThe base class never calls any logger or metrics library. All hooks are no-ops by default.
API
| Export | Type | Description |
|---|---|---|
LruCache<K, V> | class | LRU + TTL cache; generic key and value types |
Memoize<Args, Result> | class | Keyed successful-result cache with keyed single-flight execution |
LruCacheCollaboratorsInterface | interface | Typed collaborators LruCache.create accepts alongside schema-validated config — currently an optional clock provider |
CacheError | class | Base package error |
CacheConfigError | class | Invalid cache configuration |
Memoize<Args, Result>
| Member | Signature | Description |
|---|---|---|
create | static create<Args, Result>(callback, cacheConfig, { keyDeriver }): Memoize<Args, Result> | Constructs memoization from a callback, schema-validated cache config, and explicit derived key |
call | (...args: Args) => Promise<Result> | Returns a cached successful result or runs/coalesces the callback |
invalidate | (...args: Args) => void | Evicts one result by its derived key |
clear | () => void | Evicts every cached result |
LruCache<K, V>
| Member | Signature | Description |
|---|---|---|
create | static create<K, V>(config: unknown, collaborators?: LruCacheCollaboratorsInterface): LruCache<K, V> | Constructs a cache from schema-validated config and a typed clock collaborator |
size | get size(): number | Current entry count |
get | (key: K) => V | undefined | Returns value; promotes to MRU; evicts expired |
tryGet | (key: K) => { found: boolean; value: V | undefined } | Distinguishes a miss from a stored undefined value in one traversal |
set | (key: K, value: V, options?: { staleMs?: number; ttlMs?: number }) => void | Stores a value with optional per-entry staleness and expiry thresholds |
has | (key: K) => boolean | True if key exists and has not expired |
delete | (key: K) => boolean | Removes entry; returns whether it existed |
deleteWhere | (predicate: (key: K, value: V) => boolean) => number | Removes matching entries and returns the removal count |
clear | () => void | Removes all entries |
Entities
@studnicky/cache/entities exports cache option and node-timing schemas.
import { LruCacheOptionsEntity } from "@studnicky/cache/entities";Public entrypoints
| Import path | Use it when |
|---|---|
@studnicky/cache/node | A Northstar Books server or worker needs the runtime LRU cache, errors, and cache operations for derived catalogue or order reads. |
@studnicky/cache/browser | A Northstar Books browser bundle needs the same in-memory cache primitive for local derived view state; it is a runtime alternative to /node, not a second cache product. |
@studnicky/cache/memoize | A book-detail or order-summary loader needs successful-result reuse and keyed single-flight work. |
@studnicky/cache/interfaces | A consumer needs the typed cache collaborator contract, such as a deterministic clock, while composing its own application policy. |
@studnicky/cache/entities | A consumer needs cache configuration and timing schemas as contracts at its configuration boundary. |
Exports
| Symbol | Purpose | Import path |
|---|---|---|
LruCache | Stores bounded least-recently-used values with optional expiry. | @studnicky/cache/node |
Memoize | Caches successful callback results and coalesces matching concurrent calls. | @studnicky/cache/memoize |
CacheConfigError | Represents invalid cache configuration. | @studnicky/cache/node |
CacheError | Base error for cache failures. | @studnicky/cache/node |
LruCacheCollaboratorsInterface | Defines the typed clock collaborator create accepts alongside config. | @studnicky/cache/interfaces |