Feature flags in Firestore: kill switches that cost nothing
A feature-flag vendor is more than a small team needs. One Firestore document, a short in-memory cache and safe defaults give you kill switches and rollouts.
How long does it take to turn a feature off? If the answer is "one deploy", the switch is a slow rollback with extra steps. Take a small team running three Cloud Run services, one of which calls a third-party integration that has started timing out. The fix they want is not a code change, it is a switch that takes effect in seconds and keeps working while their own infrastructure is misbehaving.
This is a design review of the smallest system that provides that switch: one Firestore document, an in-memory snapshot, and defaults compiled into the code. It goes from requirements to the parts that break under load, outage and change, including two bugs that are easy to write in the first version of the module.
Requirements, in priority order#
Most flag discussions start with targeting rules and dashboards. For a small team the list is shorter, and the order matters.
- A kill switch has to work when everything else is broken. If the flag store is down or slow, the service keeps running with a sane behaviour. It does not crash and it does not hang waiting.
- Reads must be cheap. A flag is checked on hot paths. A network call per check adds a dependency and a latency tax to every request.
- Changes propagate in seconds. A switch that takes ten minutes to reach every instance is a deploy.
- Defaults are explicit. Every flag has a value in code, so a missing document, a typo or a first boot never changes behaviour silently.
- Flags are temporary. A flag is debt from the moment it exists, and the system should make old ones visible.
Per-user targeting rules, experiment analysis and audit-grade history are not on the list. They matter at some scale, and below it they are why a team ends up with a flag platform nobody opens.
Why one document#
| Approach | Propagation | Read cost | When the store is down | Moving parts |
|---|---|---|---|---|
| Environment variable | A full deploy | None | Not applicable | None |
| Config file in the repository | A full deploy | None | Not applicable | None |
| One Firestore document, cached | Seconds (the cache TTL) | One read per instance per TTL | Last good value, then code defaults | One document, one small module |
| Firebase Remote Config with the server SDK | Seconds to minutes | Local after fetch | Cached template | Another product and its console |
| Hosted flag platform | Seconds | Local after streaming | Depends on the vendor SDK | Account, SDK, billing |
Environment variables and files suit settings that change with releases. They are the wrong tool for a switch you flip mid-incident. A hosted platform solves the problem well and adds a vendor, an SDK and a bill. If the services already run on Firestore, the third row meets the requirements with infrastructure the team already pays for and understands, and the data model is one document that can be read, diffed and written from a script.
All flags live in a single document, config/flags. A flag is a small object so it can grow without changing the shape:
{
"checkout-v2": { "enabled": true, "rolloutPercent": 25 },
"partner-sync": { "enabled": true },
"ai-summaries": { "enabled": false, "owner": "checkout-team", "removeBy": "2026-12-15" }
}One read returns every flag, so the cost of reading does not grow with the number of flags. The document limit is 1 MiB, far more than a few hundred small flags need. The trade-off is write contention, since one document sustains a limited rate of writes, but flags change a few times a day. The owner and removeBy fields answer requirement five: they cost nothing, and a ten-line script that lists flags past their date makes the debt visible.
The flag module, and the two bugs it invites#
The module has three jobs: hold typed defaults, refresh a snapshot lazily, and answer isEnabled. It avoids a background timer on purpose, for the Cloud Run reason covered further down.
import { createHash } from "node:crypto";
import { Firestore } from "@google-cloud/firestore";
import { logger } from "../logger";
const db = new Firestore();
const FLAGS_DOC = "config/flags";
const TTL_MS = 30_000;
const LOAD_TIMEOUT_MS = 1_500;
export interface Flag {
enabled: boolean;
rolloutPercent?: number; // 0-100, only meaningful when enabled
}
const FLAG_NAMES = ["checkout-v2", "partner-sync", "ai-summaries"] as const;
export type FlagName = (typeof FLAG_NAMES)[number];
// Every flag has an explicit default, and the compiler enforces it.
const DEFAULTS: Record<FlagName, Flag> = {
"checkout-v2": { enabled: false },
"partner-sync": { enabled: true },
"ai-summaries": { enabled: false },
};
let snapshot: Partial<Record<FlagName, Flag>> = {};
let loadedAt = 0;
let inflight: Promise<void> | null = null;
function withTimeout<T>(p: Promise<T>, ms: number): Promise<T> {
let timer: NodeJS.Timeout;
const timeout = new Promise<never>((_, reject) => {
timer = setTimeout(() => reject(new Error(`flag read timed out after ${ms}ms`)), ms);
});
return Promise.race([p, timeout]).finally(() => clearTimeout(timer));
}
async function load(): Promise<void> {
try {
const doc = await withTimeout(db.doc(FLAGS_DOC).get(), LOAD_TIMEOUT_MS);
snapshot = (doc.data() ?? {}) as typeof snapshot;
loadedAt = Date.now();
} catch (err) {
// Keep the last good snapshot (or the defaults on a cold start).
// Retry after a short delay, not on every call.
logger.warn({ err }, "flag refresh failed, serving last good values");
loadedAt = Date.now() - TTL_MS + 5_000;
}
}
export async function refreshFlagsIfStale(): Promise<void> {
if (Date.now() - loadedAt < TTL_MS) return;
inflight ??= load().finally(() => {
inflight = null;
});
await inflight;
}
function bucket(flag: FlagName, userId: string): number {
const digest = createHash("sha256").update(`${flag}:${userId}`).digest();
return digest.readUInt32BE(0) % 100;
}
export function isEnabled(name: FlagName, userId?: string): boolean {
const flag: Flag = snapshot[name] ?? DEFAULTS[name];
if (!flag.enabled) return false;
const percent = flag.rolloutPercent ?? 100;
if (percent >= 100) return true;
if (!userId) return false; // partial rollout needs a stable identity
return bucket(name, userId) < percent;
}The first bug is the missing timeout. Awaiting inflight with no limit means that on a cold start, with Firestore slow or unreachable, every request waits for the read. The client library retries transient failures on its own schedule, so the wait can be long. That breaks requirement one at the worst moment: the switch exists, and the service hangs before it can consult it. The timeout caps the wait at 1.5 seconds and falls through to the last good snapshot, or the defaults on a first boot. During an outage one request per instance pays that wait every few seconds, which is the price of retrying at all.
The second bug is a type error that hides in plain sight. Declare the defaults as const and TypeScript infers { readonly enabled: false } for each, with no rolloutPercent, so snapshot[name] ?? DEFAULTS[name] is a union that does not have the property and the rollout line fails to compile. Declaring DEFAULTS as Record<FlagName, Flag> fixes it and adds a guarantee: adding a name to FLAG_NAMES without a default is a compile error, and a typo at a call site is too.
Three more properties are worth stating.
- The snapshot survives a failed read. A kill switch flipped before an outage still applies. A brand new instance uses
DEFAULTS, which is why the defaults are the safe values, not the hopeful ones. - The refresh is deduplicated.
inflightmakes concurrent requests share one read. Without it, the moment the TTL expires on a busy instance every request in flight issues its own. - The bucket is deterministic. Hashing the flag name with the user ID gives each user a stable position per flag. Raising a rollout from 25 to 50 percent keeps the first 25 percent enabled and adds more, and different flags do not all enable for the same unlucky slice of users.
Call refreshFlagsIfStale once per request, early, then use the synchronous isEnabled anywhere in the handler:
import Fastify from "fastify";
import { isEnabled, refreshFlagsIfStale } from "./flags/flags";
const app = Fastify();
app.addHook("onRequest", async () => {
await refreshFlagsIfStale(); // at most one Firestore read per instance per TTL
});
app.get("/checkout", async (req) => {
const userId = req.headers["x-user-id"] as string | undefined;
return isEnabled("checkout-v2", userId) ? checkoutV2(userId) : checkoutV1(userId);
});Only the request that finds an expired snapshot waits, and that wait is one document read with a ceiling.
Why not a snapshot listener#
Firestore's realtime listeners give push-style propagation, and the Node.js client exposes them as doc.onSnapshot(callback, errorCallback) (realtime updates guide). For a long-lived process that is a good option. On Cloud Run with the default request-based billing it fits badly, because the billing settings page says CPU is only allocated during request processing. A listener callback that fires while the instance is idle may not run until the next request, which makes propagation unpredictable, the opposite of what a kill switch needs. Instance-based billing allocates CPU outside requests and makes a listener work as expected, at the price of paying for idle instances.
Listener reads are also billed in ways that are easy to underestimate. The Firestore pricing page says you are charged one read when a document in the result set is added or updated, and that after a disconnect longer than 30 minutes you are charged as if you had issued a new query. For one small document that is cheap, but it is a cost the TTL cache does not have, and the TTL cache works under both billing modes.
Flipping a flag#
Writes come from a script, not a custom admin UI. Server client libraries authenticate through IAM and bypass Firestore security rules, so the permission to flip a flag is the permission to write that document.
import { Firestore } from "@google-cloud/firestore";
const [name, enabled, percent] = process.argv.slice(2);
if (!name || !["on", "off"].includes(enabled)) {
console.error("usage: set-flag <name> <on|off> [rolloutPercent]");
process.exit(1);
}
const db = new Firestore();
const update: Record<string, unknown> = { [`${name}.enabled`]: enabled === "on" };
if (percent !== undefined) update[`${name}.rolloutPercent`] = Number(percent);
await db.doc("config/flags").update(update);
console.log(`${name}: ${enabled}${percent ? ` at ${percent}%` : ""}`);The dotted field path updates one field of one flag and leaves the rest alone. Grant the right to run it with a dedicated role on the on-call group, as in least-privilege IAM, instead of broad Firestore write access for everyone who can deploy.
What breaks under load, failure and change#
Each instance reads the document at most once per TTL window while it is receiving requests. With a 30 second TTL, one busy instance performs at most 2,880 reads a day, and 10 instances at most 28,800, whatever the request rate. An idle instance that receives no requests performs none. Firestore bills per document read with a daily free quota; check the pricing page for current rates. To cut reads further, raise the TTL for non-critical flags or split a small short-TTL document for kill switches from a larger one for rollout settings.
Propagation is bounded by the TTL. A 30 second TTL means the slowest instance takes up to 30 seconds to see a change, and a cold instance reads fresh on its first request. Pick the TTL from the incident you are designing for. If 30 seconds is too slow for a specific switch, that switch belongs in the service's own circuit breaker.
Instances disagree briefly. During the window, one instance serves the old value and another the new one. Do not gate changes that must be atomic across instances, such as a schema switch. Use a migration with a compatible intermediate state.
A flag read mid-request can change the answer. Read it once at the start of the handler and pass the boolean down, or a request can begin on the old path and finish on the new one if the snapshot refreshes in between.
Flags rot. The slow failure is a codebase where nobody knows which of forty flags are live. The owner and removeBy fields, plus a CI script that fails when a flag is past its date, turn that into routine cleanup. Delete the flag from the code first, then from the document.
Both branches must stay tested. A flag doubles the paths through the code. Keep tests for the off state for as long as the flag exists, or the kill switch will be broken on the day it is needed. Running the suite twice with different defaults for flags that guard risky paths is cheap.
The kill switch needs a drill. Flip a switch in staging every few months and watch the effect arrive. A switch never flipped is a hypothesis.
Where this design stops fitting#
Do not build this if you need targeting by arbitrary user attributes, percentage rollouts coupled to experiment analysis, per-environment approval workflows or an audit trail a compliance team will accept. A hosted platform earns its price there, and rebuilding it in Firestore costs more than the subscription. It is also the wrong place for secrets or per-environment configuration, which belong in the setup from configuration and secrets for Node services. And if services are few and deploys are fast and safe, an environment variable and a quick rollout may be all the switch you need. The aim is to shorten the dangerous minute of an incident, not to add a system to maintain for its own sake.
Related articles
All articles →A practical split for small teams: coarse per-client limits at the edge, per-user and per-cost limits in the service, and the 429 behaviour both must share.
11 min
Ordering keys give per-key order and take away parallelism. Learn what they guarantee, how one failure stalls a key, and when versioned events are a better fix.
9 min
Payment webhooks arrive twice, late and out of order. A handler that verifies, stores, acknowledges fast and applies effects idempotently, shown with Stripe.
11 min