Cal Henderson: our patron saint of scaling web apps

Scaling means serving more useful work while keeping response times, correctness, reliability and cost within an explicit budget. For that job, our editorial choice is Cal Henderson.

Henderson led engineering at Flickr and later co-founded Slack. His 2006 book, Building Scalable Web Sites, connects application architecture, database queries, caching, load balancing, replication, partitioning, storage and monitoring. That breadth makes him our pick: the whole application and its operation, rather than one fashionable component. See his biography and the publisher’s contents and overview.

The book is historical, not a current deployment manual. We are borrowing its breadth and practical questions, not copying its PHP/MySQL-era settings. This profile uses the public biography, book overview and contents; it does not claim a fresh cover-to-cover review. Our application to Ruby follows this site’s existing guidance and the framework documentation linked below.

Who else belongs on the shelf?

These are complementary perspectives, not a ranking of people’s worth. Henderson earns this particular editorial title for the application-wide scope of his work.

1. Name the workload

“A million users” is not a capacity plan. Record peak requests per second, read/write mix, costly endpoints, concurrent connections, uploads, job arrivals, data growth and tenant skew. A browsing catalog and a burst of inventory updates have different limits.

Choose a latency percentile and measurement window, correctness requirements and maximum age for urgent jobs. Load-test the actual mix with realistic data, including an unusually busy tenant. Find where throughput stops increasing and queues grow. These are our planning practices; projected traffic remains a hypothesis, not proof that distribution is needed today.

2. Separate waiting from working

Distinguish queue time, application execution, database work and downstream waits. More web processes can reduce waiting for a worker; they cannot repair a missing database index. Faster requests also occupy resources for less time. Berkopec explains the queueing distinction.

Track percentiles as well as averages. With many dependencies, an occasional slow response can become an ordinary problem for the overall request. Every synchronous service adds latency and failure modes to that path. Read Dean and Barroso’s The Tail at Scale. Record workload, bottleneck, change, before-and-after results and cost; stop when the target is met.

3. Reduce database work

Use bounded queries, suitable indexes and only the data the page needs. Remove N+1 queries with deliberate eager loading, but inspect the SQL and allocations it creates. Batch maintenance rather than loading an entire table into Ruby. Active Record’s query guide describes the tools.

Check plans against representative data. EXPLAIN ANALYZE executes the statement: consider the effects of writes and expensive production queries before using it. Indexes add storage and write-maintenance cost. Keep transactions short, distinguish lock waits from connection-pool waits, and preserve tenant scoping and constraints. See PostgreSQL’s EXPLAIN documentation.

4. Give every cache a correctness contract

Specify what can be stale, for how long, and what invalidates it. Include tenant and relevant permission context when results differ. A shared fragment must not cross an access boundary. A cached result cannot replace a current authorization check unless it safely represents that permission.

Rails supports fragments, versioned keys and low-level fetching. Prefer cached primitives or IDs to long-lived Active Record objects. Measure hit rate, lookup latency and rebuild cost; test cold starts and simultaneous expiry. A warm-cache benchmark hides important load. See Caching with Rails.

5. Own the work moved into a queue

Email, reports and media processing often belong in jobs. That shortens the request without removing the work. Define the waiting state and how failure becomes visible. Active Job supplies an interface; execution behavior depends on the backend.

Side effects must tolerate retries. Sidekiq promises at-least-once execution, not exactly once. Use durable idempotency records or provider-supported keys where appropriate; a casual “already done?” check can race. Measure oldest-job age, failure and processing rates. Isolate urgent work from bulk jobs. A growing queue is deferred overload.

6. Budget processes, memory and connections together

Keep the Rails application together until a measured need justifies splitting it. A monolith can use multiple web processes and separate job workers. Durable state belongs in backing services, not process memory.

On CRuby, threads can overlap I/O while the GVL constrains parallel Ruby execution within a process. Processes consume memory; concurrency can overwhelm the database. Budget connection pools across web and job processes, allowing deployment overlap and administration. There is no universal worker/thread count. Use Rails’ deployment tuning guide and measurement. A bigger machine can be the simplest next step; multiple machines add coordination while potentially improving capacity or resilience.

7. Distribute data with a correctness plan

Replicas can offload suitable reads, but lag can make recent writes appear missing. Rails’ delay-based database switching provides limited read-your-own-write behavior, not a guarantee of fresh reads for everyone. Read Multiple Databases with Active Record.

Decide which reads may be stale: a popularity count and permission revocation have different needs. Use authoritative reads and transactional enforcement for invariants. Before sharding, explain the key, hot tenants, cross-shard operations, rebalancing, migration and recovery. Distribution must earn those obligations. Vogels’ Eventually Consistent provides useful context.

8. Design for overload and recovery

Bound queues, retries and timeouts. Defer or reject excess work deliberately; preserve critical user journeys when optional work fails. Google SRE’s overload chapter explains why unlimited acceptance is not resilience.

Practice restores, rollback and dependency failure. Redundancy is not proof of recovery; two processes on one machine do not survive losing that machine. Hamilton’s operational guidance belongs beside throughput measurements.

9. Count money and attention

Join request/job outcomes with latency, saturation and errors. Correlate traces, metrics and logs without recording secrets or sensitive payloads. OpenTelemetry’s signals support our existing portable observability approach.

Our cost measure is useful completed operations, including headroom, storage, transfer and external calls. Add backups, incidents and engineering effort to the decision. Self-hosting does not make maintenance free. A new service must earn its complexity.

Rails scaling checklist

  1. Write the workload, latency target, correctness requirements and growth assumption.
  2. Capture a production-like baseline, including a busy tenant and cold cache.
  3. Identify the limiting resource before changing the architecture.
  4. Fix wasteful SQL, excessive allocations and unbounded work first.
  5. Specify cache keys, access boundaries, invalidation and rebuild behavior.
  6. Make jobs retry-safe; measure their age and failed outcomes.
  7. Budget Puma workers, job concurrency, RAM and database connections together.
  8. Document stale-read behavior before adding replicas or shards.
  9. Test overload, dependency failure, restoration and rollback.
  10. Report before-and-after latency, throughput, errors and cost. Stop when the target is met.

Scaling different kinds of platforms with open source

These are proposed designs, not reconstructions of any company's private architecture or measured capacity promises. They put Henderson's application-wide perspective to work. Record request mix, data size, concurrency, hardware, cache state and latency targets before comparing results. Start with a measured Rails monolith; add a component when it removes an observed constraint and someone can operate it.

The common foundation

Use Rails, PostgreSQL and background workers. Size web and worker processes independently while keeping business rules together. Solid Queue provides durable database-backed jobs; its default separate database does not make enqueueing atomic with an application transaction. Deferring enqueue until commit prevents premature execution but leaves a crash gap between commit and enqueue. For indispensable work, persist an outbox record alongside business changes and dispatch it with retries. Consumers need stable idempotency keys and database uniqueness constraints. See Rails job semantics.

Budget every web, worker, queue, Cable, dispatcher and scheduler connection pool, plus migrations, administration and overlapping deployments. Size pools for measured concurrent database use. PgBouncer can reduce server connection pressure, but transaction pooling changes session semantics: verify LISTEN, session advisory locks and prepared-statement configuration against its compatibility matrix.

Set latency, error, freshness and recovery objectives. Measure request percentiles, SQL and lock waits, connection checkout, CPU, memory, GC, oldest-job age, retry rate, replication lag, disk/WAL growth and cost per useful operation. Bound concurrency, request bodies, queues and retries. Keep sensitive payloads out of logs. Test backup restoration. Our identity and service boundaries remain the defaults: tenant-scoped records and authorization, interchangeable providers and private object storage.

1. Multi-tenant SaaS

Workload and invariants. Interactive CRUD shares resources with imports, exports and scheduled work. One customer's bulk task must not starve others. Tenant isolation must hold across requests, jobs, caches and search.

Start. Use tenant-scoped composite indexes, database constraints and Solid Queue for exports and integrations. Follow this site's organization/rooftop model and PostgreSQL row-level security; verify the application role and pooled connection context cannot bypass or leak it. Carry tenant identity into jobs and cache keys. Use keyset pagination and remove N+1 queries.

Measure and grow. Compare resource use and latency by tenant. When an import drives connection waits or job age, impose tenant concurrency limits and fair scheduling, then isolate heavy workers. Put the queue database on independent resources when it competes with transactions. Use replicas only for stale-tolerant reporting, keeping immediate read-after-write paths on the primary. Split exceptionally large tenants only when storage, isolation or contention warrants routing and migration machinery.

Tradeoff and failure test. Sharding complicates reports, moves and recovery. Run a large export alongside ordinary edits with a skewed tenant distribution, then restart workers. Assert other tenants' latency, no cross-tenant records, bounded queue age, resumable output and authorization after a role change.

Queue progress needs its own promise

For the SaaS workload above, define acceptable job age by class and tenant before choosing a scheduler. The pinned Solid Queue documentation gives earlier configured queues precedence over later ones; a job’s numeric priority does not override that queue order. A continuously busy first queue can keep later work waiting. Start with the existing Rails stack and compare dedicated capacity for each class before building custom tenant scheduling. Reserved workers may sit idle and still share database resources; class isolation alone does not establish tenant fairness.

The same revision’s concurrency controls do not guarantee execution order. Semaphore expiry can admit another job while the original holder still runs; duration is not a cancellation deadline. Longer leases reduce overlap risk but delay recovery from a lost holder. Protect a database-only business effect with a transactional state condition or uniqueness constraint. External effects need the provider’s idempotency and reconciliation contract.

PostgreSQL 18 documents that SKIP LOCKED skips unavailable row locks and produces an inconsistent view useful for queues; ordinary table locking still applies. An empty poll does not prove there is no work, and skipping locks does not establish fairness. Observe ready, blocked, claimed, failed and completed jobs, including blocked-job age, rather than aggregate throughput alone.

Proposed tests, not run here: with one execution slot and queues ordered interactive then export, keep interactive work ready for a fixed number of barrier-controlled claims; verify export starvation even with higher scheduling priority (a lower numeric value), then compare dedicated class capacity against a predeclared age budget. Hold a same-key job before its write, expire its semaphore in a disposable test database and run maintenance; verify overlapping attempts still produce one durable effect in the correct tenant. This controlled expiry does not validate real-clock timing. In separate PostgreSQL sessions, lock the oldest row, then all eligible rows, and confirm skipped work resumes after release; separately demonstrate a conflicting table lock with a bounded lock timeout. Use the application’s pinned dependencies and real database, not the in-memory job adapter or mocked SQL. A finite exercise is not a universal progress guarantee. See our testing standards.

Documentation checked 16 October 2026. These are engineering proposals, not installed-version, benchmark or runtime test results.

2. Ecommerce and marketplaces

Workload and invariants. Browsing is read-heavy; checkout concentrates writes on scarce inventory. Never promise the same unit twice, duplicate an order on retry, or interpret a payment timeout as proof of failure.

Start. PostgreSQL owns orders, reservations and inventory. Use short transactions, conditional updates or row locks, constraints and checkout idempotency keys. Reservations need explicit states, expiration and reconciliation. Keep provider requests outside held locks. Use hosted payment fields or checkout, never collect card numbers. Verify webhook signatures, persist event identifiers and tolerate duplicates and out-of-order delivery. Review PostgreSQL locking and deadlocks.

Measure and grow. Cache repeated public catalog reads; optional Valkey is never the inventory authority. Separate checkout and email work. When popular items cause lock waits, shorten transactions, order locks consistently and cap concurrent checkout. Admission control during a sale beats an uncontrolled retry surge. Add OpenSearch when relevance or faceting needs it; revalidate price and stock at checkout.

Tradeoff and failure test. Search and replicas can show stale stock; reservations temporarily withhold it. Race buyers for the last unit, repeat submissions and webhooks, and lose a payment response after success. Assert stock invariants, one logical transition per key, eventual reconciliation and a usable sold-out response. A cache outage must not stampede the database.

3. Social feeds and chat

Workload and invariants. Feeds amplify reads and fan-out; chat adds sockets, reconnect bursts and per-conversation ordering. Acknowledged durable messages must survive reconnects. Membership, blocking and deletion apply to stored and live content.

Start. Keep posts, messages and membership in PostgreSQL. Use indexed feed queries and keyset pagination. Action Cable delivers notifications; durable history and cursor-based catch-up stay in the database. A broadcast is not the permanent message record. The Action Cable guide covers separate servers and worker/database implications.

Measure and grow. Track sockets, outbound bytes, reconnects, delivery delay and fan-out job age separately from HTTP. Separate Cable capacity when it contends with requests. If feed queries exceed their budget, materialize entries asynchronously. Ordinary accounts may use fan-out-on-write while very large audiences use read-time merging; choose from measured write amplification. Bound client buffers, coalesce disposable updates and disconnect slow consumers with resumption cursors.

Tradeoff and failure test. Materialized feeds increase storage and delay edits/deletes. Presence may be approximate; durable history cannot. Test a high-follower post, busy room, slow clients and mass reconnect. Assert no duplicate logical messages, defined conversation order, complete catch-up and prompt permission revocation. Keep message bodies out of broad diagnostic traces.

4. Uploads, media and video delivery

Workload and invariants. Bytes, transformations and bandwidth dominate. Accepted originals must remain recoverable. Untrusted files must not become executable or public accidentally.

Start. Rails/Active Storage tracks metadata and authorization. Follow this site's S3-compatible private storage and direct-upload convention, using tenant-scoped keys and expiring authorization; app replicas do not share local disks automatically. Back up originals independently. Isolate FFmpeg workers, quarantine unprocessed files, validate actual type, size, duration and decompression limits, and cap time/memory. See Active Storage and FFmpeg build and license considerations.

Measure and grow. Track upload completion, transcode age, CPU, temporary disk, playback startup/rebuffering and origin egress. Give media workers dedicated capacity before multiplying web servers. Use immutable derivative keys and publish completed renditions atomically. Add adaptive streaming when playback needs justify storage costs. A CDN distributes cached content near viewers; an object store is the durable origin. Self-hosted storage alone is not a global CDN.

Tradeoff and failure test. Ceph Object Gateway is an S3-compatible option when distributed durability and experienced operators justify substantial infrastructure. It is not a mandatory starter dependency. Hosted storage/CDNs are optional service boundaries, not open-source components. Test interrupted uploads, duplicate callbacks, malformed video, full disks, worker death and cold-cache demand. Assert originals survive, partial renditions stay hidden, retries are bounded, orphan uploads are cleaned up and private media never enters public shared caches. Plan deletion and CDN purge.

5. Search and knowledge archives

Workload and invariants. Document size, ingestion bursts, relevance and access control matter. Preserve provenance and stable IDs. Revoked access must not remain visible in stale snippets or aggregations.

Start. Use PostgreSQL full-text search, appropriate indexes, explicit language configuration and a judged query set. Keep canonical documents outside any disposable search projection. Extract asynchronously with limits on hostile or oversized files.

Measure and grow. Track latency, relevance, zero-result rates and source-to-index freshness. Add OpenSearch when corpus size, faceting, relevance or workload isolation exceeds the simpler design. Use an outbox/replayable pipeline, versioned upserts and deletion tombstones. Bulk index and benchmark shards against the actual corpus. Rebuild into a new index, replay intervening changes, then switch readers.

Tradeoff and failure test. Another engine adds eventual consistency, memory/disk operations, snapshots and schema migrations. Filter retrieval and aggregations by authorization, and recheck current access before revealing protected content. Embeddings require a separate relevance experiment and model-license/privacy review. Test long-tail queries, huge documents, concurrent ingestion, outages and rebuilds. Revoke access during lag and replay duplicates/out-of-order changes; deleted content must not reappear.

6. Analytics and event ingestion

Workload and invariants. Append-heavy events meet wide scans. Define accepted-event durability, allowed loss/duplication, event versus arrival time, retention and deletion.

Start. At modest volume, use PostgreSQL event tables, batched writes and scheduled aggregates. Bound and version payloads, assign event IDs and minimize personal data. Product transactions can write outbox events so requests do not depend on an analytics service.

Measure and grow. When event writes, scans or storage impair transactions, move analytical copies into ClickHouse while keeping orders/accounts in PostgreSQL. Benchmark sort keys, batch sizes and retention. Batch inserts or use asynchronous inserts that wait for flush acknowledgment. Fire-and-forget acknowledgment can hide errors and undermine backpressure; see insert behavior. Retry deduplication depends on engine, settings and finite windows; it is not unlimited end-to-end exactly-once delivery.

Tradeoff and failure test. Add a replicated event log only when independent consumers, retention or outage buffering earns another operated system. Monitor oldest event, buffer disk, failed batches, merge backlog and query contention. Test duplicates, late arrivals, malformed batches and sink outages/recovery. Assert accepted-event accounting, bounded storage, explicit backpressure and correct replayed aggregates. Deletion policy must cover raw events, aggregates and retained backups.

Open source is a versioned decision

These checks describe source projects, not blanket permission for every distribution, plugin or hosting product. Before selection, verify the exact version/license, maintained releases, advisories, client compatibility and an upgrade/recovery rehearsal.

A repeatable scaling gate

Record the failing objective, profile evidence, smallest proposed fix, resource budget and rollback. Use realistic skewed traffic, cold caches and failures. An arrival-rate load model can expose growing queues that a slow-client benchmark hides. Compare tail latency, correctness, freshness, recovery and cost. Keep the simpler design when it meets the targets; microservices and Kubernetes are not default prerequisites.

Provided as is, for information only, with no warranty and no liability. Not legal or professional advice. Security and radio material is for authorized, lawful use only. Full disclaimer.

Visits are logged on our own server: IP address, browser and device details, and pages viewed. We use them to understand traffic and spot attacks, and never share or sell them.