Documents and graphs: when a second database earns its place

Firestore is superb at keyed reads and poor at friends-of-friends. When a graph database earns its place, and how to keep two stores honest.

Published

Updated

Reading time

12 min

A second database is justified by one thing: a question your first database can only answer by shipping you half the dataset. For a document store that question is multi-hop, "people you may know", "mutual connections", "followers of people you follow who also like this". Once you have such a question on a hot path, add a graph database, and give it exactly one job: it answers "which ids", while the document store stays the only place where any fact is written. Every fact has one owner. Everything else is a projection that can be thrown away and rebuilt.

That rule is the whole argument. The rest of this piece shows why the question breaks a document store, what the graph actually changes, and which parts of the pipeline you cannot skip if you accept the rule. The examples use Firestore and Neo4j because that is the pairing I know best, but nothing in the reasoning is specific to them.

Where the document store stops being cheap#

Firestore is excellent when you know the key of what you want. users/{uid} is one read with constant cost no matter how many users exist. A feed item can carry the author's display name so a list renders from one query. Realtime listeners come with the product. The shape it rewards is "give me this document" or "give me the documents where an indexed field matches".

Friends-of-friends does not have that shape. Say friendships are stored as users/{uid}/friends/{friendUid} and you want people you may know, ranked by mutual friends:

pymk-firestore.tsts
import { getFirestore } from "firebase-admin/firestore";
 
const db = getFirestore();
 
export async function peopleYouMayKnow(uid: string, limit = 20) {
  // Hop 1: my friends. One read per friend document.
  const mine = await db.collection(`users/${uid}/friends`).get();
  const myFriends = new Set(mine.docs.map((d) => d.id));
 
  // Hop 2: each friend's friends. One query per friend, one read per result.
  const counts = new Map<string, number>();
  await Promise.all(
    [...myFriends].map(async (friendId) => {
      const theirs = await db.collection(`users/${friendId}/friends`).get();
      for (const d of theirs.docs) {
        if (d.id === uid || myFriends.has(d.id)) continue;
        counts.set(d.id, (counts.get(d.id) ?? 0) + 1);
      }
    }),
  );
 
  return [...counts.entries()]
    .sort((a, b) => b[1] - a[1])
    .slice(0, limit)
    .map(([id, mutuals]) => ({ id, mutuals }));
}

The code is correct, and it is a cost trap. Take a hypothetical user with 300 friends who each have 300 friends. One request is 300 reads for hop 1 and up to 90,000 for hop 2, and Firestore bills per document read. The exact numbers are made up; the shape is not. Cost is the product of two fan-outs, social graphs have long tails, and so your most active users, the ones you most want to keep, are the most expensive to serve. Filtering blocked users or pending requests adds still more reads.

You can mitigate it by capping hop 2 at the fifty most recent friends, sampling, or caching the answer for a day. For a while you should. Each mitigation makes the answer worse, though, and none of them changes the underlying fact: the database ships you the entire second hop so your application can do the join.

What the graph changes, and what it does not#

A graph database stores relationships as records with direct references between nodes. In Neo4j, moving from a node to its neighbours follows stored pointers rather than doing an index lookup per hop. The engine does the join near the data and returns only the answer, in Cypher:

pymk.cyphercypher
MATCH (me:User {id: $uid})-[:FRIENDS_WITH]-(f:User)-[:FRIENDS_WITH]-(fof:User)
WHERE fof <> me
  AND NOT EXISTS { (me)-[:FRIENDS_WITH]-(fof) }
  AND NOT EXISTS { (me)-[:BLOCKED]-(fof) }
RETURN fof.id AS id, count(DISTINCT f) AS mutuals
ORDER BY mutuals DESC
LIMIT toInteger($limit)

One round trip, twenty rows. A third hop, a weighting by shared interests or a new exclusion rule changes the pattern instead of adding another loop to application code. That expressiveness is the real gain.

Two things it does not change. A traversal through a node with millions of edges is still expensive, so cap expansion per hop or leave very high-degree accounts out of recommendation patterns. And the graph is not a better place to store the data, only a better place to ask one kind of question about it.

The toInteger($limit) is not decoration. The JavaScript driver sends a plain JavaScript number as a Cypher float, and LIMIT and SKIP want an integer. Either wrap the parameter as above or pass neo4j.int(20) from the driver. It is the first error most people hit, and it was mine. A paginated graph query that had worked for weeks started returning 500s one night. It took an evening to corner: my driver was configured to return plain JavaScript numbers, so it also sent them, and the database received the page size as 25.0, a float, and refused to LIMIT by it. The fix was one line, wrapping pagination values in the driver's explicit integer type.

Firestore owns the record, Neo4j owns the traversal#

Once two stores hold overlapping facts, the dangerous question is "which one is right when they disagree". Settle it before writing any code:

  • Firestore owns profiles, settings, content and the canonical record of every relationship: one friendships/{pairId} document per pair, block lists, follow records. Per-user subcollections like users/{uid}/friends can exist, but as denormalised views for the UI. API writes go here and nowhere else.
  • Neo4j owns nothing. It holds a projection of relationship facts plus the few node properties traversals filter on: an id, a visibility flag, maybe a coarse region. It never holds display names or avatars.
  • Derived answers (recommendations, mutual counts) come from Neo4j as lists of ids and are hydrated from Firestore.

Consequences follow without further design. If the stores disagree, Firestore wins and the graph is repaired. If the graph is lost, you rebuild it from Firestore. If you are tempted to write a fact into Neo4j first, you are about to create a second source of truth, and the system will eventually tell you which of the two it believes by doing something wrong.

Figure 1— Firestore owns every fact; Neo4j is a rebuildable projection fed by events.

Keeping the projection honest#

One owner per fact is cheap to say and expensive to enforce. The pipeline from the owner to the projection has to survive duplicates, reordering and crashes.

Emit events from the commit, not from the API#

You need a signal whenever a relationship document changes. Two options on Google Cloud:

  • Firestore triggers through Cloud Run functions and Eventarc. The event fires after the write commits, so there is no dual-write problem. Delivery is at-least-once and unordered.
  • A transactional outbox. In the same Firestore transaction that writes the friendship, write an outbox/{eventId} document. A trigger or poller publishes outbox entries to Pub/Sub and marks them sent. You get a durable, inspectable log and control over the payload.

Never publish to Pub/Sub from the API after writing to Firestore. If the process dies between the two calls, the graph silently misses a fact and nothing tells you.

The event needs enough information to be applied in any order, any number of times:

events.tsts
export type RelationshipEvent = {
  type: "friendship";
  a: string;           // lower uid of the pair, so the edge id is stable
  b: string;           // higher uid
  active: boolean;     // false = removed
  version: number;     // monotonic per pair, stored on the Firestore doc
};

Increment version in the same transaction that changes the relationship. It is what makes out-of-order delivery harmless.

Upsert with a version guard#

At-least-once delivery means duplicates, and no ordering means stale events. MERGE absorbs the duplicates and the version guard rejects the stale ones:

sync-worker.tsts
import neo4j from "neo4j-driver";
import type { RelationshipEvent } from "./events";
 
// Create once per process and reuse: the driver pools connections.
const driver = neo4j.driver(
  process.env.NEO4J_URI!,
  neo4j.auth.basic(process.env.NEO4J_USER!, process.env.NEO4J_PASSWORD!),
);
 
const APPLY = `
MERGE (a:User {id: $a})
MERGE (b:User {id: $b})
MERGE (a)-[r:FRIENDS_WITH]->(b)
  ON CREATE SET r.version = -1
WITH r
WHERE r.version < $version
SET r.version = $version,
    r.active  = $active
`;
 
export async function applyEvent(evt: RelationshipEvent): Promise<void> {
  await driver.executeQuery(
    APPLY,
    { a: evt.a, b: evt.b, active: evt.active, version: evt.version },
    { database: "neo4j" },
  );
}

Create the uniqueness constraint before the first write, so MERGE is safe under concurrency and has an index behind it:

schema.cyphercypher
CREATE CONSTRAINT user_id IF NOT EXISTS
FOR (u:User) REQUIRE u.id IS UNIQUE;

Three details carry most of the weight:

  1. Direction is canonical. Friendship is symmetric, so always write a -> b with a < b and query without direction. One edge per pair.
  2. Removals are soft. If an unfriend arrives and you DELETE the edge, a delayed older "befriend" event recreates it, because the version that would have rejected it went with the edge. Keep the edge with active = false, filter on r.active in traversals, and garbage-collect inactive edges older than your longest redelivery window. This is the single most common way a version guard is defeated.
  3. Ack after the write succeeds. Pub/Sub redelivers on failure. Configure a dead-letter topic so one poison message does not retry forever, and alert on its depth. The consumer-side rules are the same ones described in idempotent Pub/Sub consumers.

With soft removals the recommendation query filters both hops:

pymk-active.cyphercypher
MATCH (me:User {id: $uid})-[r1:FRIENDS_WITH]-(f:User)-[r2:FRIENDS_WITH]-(fof:User)
WHERE r1.active AND r2.active AND fof <> me
  AND NOT EXISTS { (me)-[x:FRIENDS_WITH]-(fof) WHERE x.active }
RETURN fof.id AS id, count(DISTINCT f) AS mutuals
ORDER BY mutuals DESC
LIMIT toInteger($limit)

Backfill and reconcile#

To introduce the graph to an existing product, start the live consumer first, then run a backfill that reads the friendships collection in pages ordered by document id and applies the same upsert in batches with UNWIND $rows AS row. Because both paths carry versions they can overlap safely. Run a reconciliation pass before enabling the read path behind a flag.

Reconciliation should not stop at launch. Pipelines drift: a deploy with a bug, a dead-lettered message nobody replayed, a manual fix in one store. A scheduled job that samples users, compares their active edges in both stores and re-emits events for differences turns drift into a metric. If that metric is not near zero, the pipeline has a bug, and you want to learn it from a dashboard rather than from a user. I run two mechanisms for this: scheduled sampling that logs every mismatch between owned facts and their shadows, and on-read healing, where a service that notices a shadow contradicting its owner repairs it in passing instead of serving the stale value. Both have caught real bugs before users did.

Reading across two stores#

The graph answers "which ids"; Firestore answers "what do they look like". Hydrate with one batched read, not one get() per id:

recommendations.tsts
import { getFirestore } from "firebase-admin/firestore";
 
const db = getFirestore();
 
export async function hydrate(ids: string[]) {
  if (ids.length === 0) return [];
  const refs = ids.map((id) => db.doc(`users/${id}`));
  const snaps = await db.getAll(...refs);   // one round trip, one read per doc
  return snaps.filter((s) => s.exists).map((s) => ({ id: s.id, ...s.data() }));
}

Two rules for this path follow from the ownership rule. First, the projection lags, so anything security-relevant (blocks, account deletion, privacy settings) is re-checked against Firestore during hydration. The graph can suggest; only the owner can permit. Second, the graph is a secondary store and must never take down a primary page. Put a short timeout on the graph call and degrade to an empty list or a cached result. Cache the final list per user for minutes or hours, which also absorbs most of the traversal load; recommendations do not need to be fresh to the second.

The strongest objection: do you need a graph at all#

The honest counter-argument is that a second store costs more than the reads it saves, and often it does. You take on a sync pipeline with its own bugs and backlog, a consistency gap (a user can unfriend someone and still see them suggested for a moment), two failure modes on every read path, and a second query language and set of credentials, backups, upgrades and connection limits to run. A serverless platform that scales out fast opens a driver pool per instance, so connection limits need deliberate sizing.

Managed tiers add their own surprises. Neo4j documents that AuraDB Free instances are paused automatically after 72 hours of inactivity. A rarely used staging environment is the usual casualty: the first request after a quiet weekend fails with a connection or DNS error that looks like a network bug and not a paused database. Health-check the instance and alert on it, and know that a free instance left paused long enough is eventually deleted.

So compare the real alternatives before committing:

ApproachMulti-hop query costConsistencyExtra operations
Firestore fan-out at read timeGrows with friends times friends-of-friendsStrongNone
Precomputed recommendations in documentsOne read at request time, heavy batch writesStale by designBatch jobs
Postgres self-join or recursive CTEModerate, index-dependentStrong if Postgres owns the dataOne relational database
Firestore plus Neo4j fed by eventsOne bounded traversalEventualSecond database plus pipeline

If you already run Postgres, a self-join answers friends-of-friends well:

pymk.sqlsql
SELECT f2.friend_id AS id, count(*) AS mutuals
FROM friendships f1
JOIN friendships f2 ON f2.user_id = f1.friend_id
WHERE f1.user_id = $1
  AND f2.friend_id <> $1
  AND NOT EXISTS (
    SELECT 1 FROM friendships x
    WHERE x.user_id = $1 AND x.friend_id = f2.friend_id
  )
GROUP BY f2.friend_id
ORDER BY mutuals DESC
LIMIT 20;

With an index on (user_id, friend_id) and edges stored in both directions, this is fast at considerable scale, with strong consistency and no pipeline. Recursive CTEs cover variable-depth traversal. If Postgres is not on the menu and the graph is small or slow-moving, a nightly job that writes one recommendation document per user is hard to beat: one read at request time. If the query is rare, run it offline in BigQuery against an export.

Where the argument stops#

The position holds when three things are true together: the question needs two or more hops, it sits on a user-facing path that runs often, and a few seconds of staleness is acceptable. Drop any one and the case weakens. A one-hop question (my followers, my friends) is a plain indexed query. A rare or admin-only multi-hop question belongs in a batch job. A question that needs strong consistency cannot come from a lagging projection. In my pipeline the lag is a matter of seconds, and for social-topology shadows that is a trade nobody has noticed from the outside.

There is also an organisational condition that is easy to overlook: someone has to own the pipeline and its on-call. A graph store without an owner decays into a second source of truth that nobody trusts, which is worse than not having it.

Almost every domain contains relationships. Very few need to traverse them several hops on a hot path. Add the graph when the numbers say so, give it one job, and leave the facts where they already live.

Share
All articles →

The Firestore limits that bite as a database grows, and the schema choices that avoid them: hot documents, index fan-out, ramp-up, pagination, rules and TTL.

13 min