PostgreSQL Active-Active Across 3 Regions | Production HA

Three writable PostgreSQL regions. No single global primary - all three accept writes at once. That is active-active, with pgEdge Spock. When one region dies, the survivors keep writing.

Watch. Cape Town disappears - the entire region. Virginia and Frankfurt remain writable. Nothing went read-only. But Spock does not move application traffic. Something else must.

So here is the plan. We build this from one server, layer by layer - then we break a database node, lose etcd quorum, and remove an entire region on purpose.

Start where everyone starts: one PostgreSQL 18 server in Northern Virginia. It takes every read and every write. For users on the US east coast it is fast. For someone in Frankfurt, every query crosses an ocean twice. And if this box dies, everything stops. That is the problem we are solving.

Second region: Frankfurt, in EU-Central. Another full PostgreSQL node - and notice, it is not a replica. It's a second active server, read and write, sitting close to European users so their queries never have to cross an ocean. Two independent regions now, each serving its own corner of the world.

And a third, in Cape Town - AF-South. Same idea: a fully writable node, close to users in the southern hemisphere. So now we have three active PostgreSQL servers on three continents. The catch? Right now, they know nothing about each other. Three islands. Let's connect them.

This is where it gets interesting. We turn on pgEdge Spock - logical, multi-master replication for Postgres. Every node subscribes to every other node, in both directions. So the moment Virginia commits a change, that change flows out to Frankfurt and Cape Town. When Frankfurt writes, it flows back to Virginia and Cape Town. It's a full mesh - no single global leader, no one primary that every region depends on. Each region is a first-class citizen, and changes replicate continuously in the background.

There is no global primary to hunt for. A user in Mumbai writes to Cape Town; a user in New York writes to Virginia. Each gets a fast local commit, and Spock carries the change to the other regions. Everybody writes locally. Changes replicate continuously, and regions converge as replication lag clears.

Active-active does not eliminate conflicting writes. Spock resolves supported conflicts with its configured policy, normally last-update-wins - which needs commit timestamps configured correctly. Non-resolvable update-missing, update-exists, and secondary unique-constraint conflicts enter the exception log. Production systems need globally unique keys, regional write ownership, conflict monitoring, and coordinated schema changes.

The mesh keeps the regions connected - but inside one region, we still have to survive a server failure. So each active node gets two local hot standbys, kept current through physical streaming replication. Place those members across separate availability zones. If the active node fails, nearby copies remain promotion candidates. Clients connected to the failed process must reconnect.

But copies aren't enough - something must decide who is in charge when a node fails. That's Patroni: an agent on every database member in the region, watching health. Continuously verify that each promotion candidate has failover-ready logical slots. Then, when the active fails and etcd still has quorum, Patroni promotes an eligible candidate. That turns a pile of servers into one self-managing regional cluster.

Patroni must never guess who the leader is. Each region runs a three-member etcd cluster across three availability zones. As long as two agree, the region has exactly one leader. Use watchdog or fencing so an isolated former leader cannot keep serving writes after losing authority. Consensus stays inside the region - nobody crosses an ocean just to hold an election.

Applications need stable regional endpoints. HAProxy instances span separate zones and query Patroni's role API on each database member. Port five thousand forwards to the current writer's PgBouncer; port five thousand one to eligible replicas. Above them, global DNS uses application-level readiness - not plain TCP. HAProxy routes within a region; it does not perform global failover.

PgBouncer runs on each database host - or beside each member - not as one shared pool in front of the whole cluster. HAProxy selects the member that matches the Patroni role, then that member's PgBouncer pools connections into its local PostgreSQL. Thousands of application sessions become a controlled number of backends. Routing stays role-aware. Pooling stays aligned with the selected member.

Now the architecture is complete: three writable regions in a Spock mesh. Each has zoned physical standbys, Patroni, etcd, HAProxy, and per-member PgBouncer. Global DNS directs clients toward a healthy regional endpoint. Every layer has one job. And together, the surviving regions need enough spare capacity to absorb one region's traffic. Now let's test failures.

First, the normal path. An application reaches HAProxy on port five thousand. HAProxy follows Patroni's role check to the current primary's PgBouncer, then into PostgreSQL, which commits at local-region latency. Spock publishes that change to the other regions asynchronously. The application gets a fast local response without pretending a remote copy is already durable.

Reads scale the same way. Point read-only traffic at port five thousand one, and HAProxy fans it to replica members through their local PgBouncer. Analytics and reports can use those standbys. But replica reads may lag. Keep read-after-write and consistency-sensitive traffic on the regional writer.

Now let's break something. The active node in one region dies. Patroni confirms etcd still has quorum, then selects a promotion candidate whose failover slots were already verified. HAProxy sees the new role and sends fresh connections to that member's PgBouncer on the same regional endpoint. Existing TCP sessions break, so a retry-aware application must reconnect. Local RPO depends on standby lag and the configured synchronous replication policy. This is controlled recovery with a brief interruption - not magical zero downtime.

Here's a subtler failure. Suppose one region loses two of its three etcd members. The remaining member cannot form quorum, so Patroni must not establish new leadership there. What happens to an existing leader depends on fencing and watchdog policy, but an unsafe promotion is refused. The other regions use independent consensus groups, so they can continue accepting writes while operators restore the affected region's quorum.

Now the big one: an entire region goes dark. Cape Town leaves the mesh, but Virginia and Frankfurt stay active, writable, and connected. A global traffic layer must detect that with application-level health checks and send new connections to a surviving endpoint. How fast that happens depends on health-check intervals, DNS TTL, resolver caching, and application retries. Spock does not move client traffic. In-flight sessions fail; retry-aware clients recover against the survivors.

When Cape Town returns, do not reopen traffic immediately. Restore a healthy local Patroni cluster, catch up Spock from retained replication state, drive logical lag to zero or an accepted threshold, clear unresolved conflicts, and pass application smoke tests. If retained WAL or replication-slot state is unavailable, rebuild or resynchronize the regional node first. Only then mark the region ready for traffic.

This design can also make planned maintenance predictable. Work one region and one replica at a time: drain it, upgrade it, validate compatibility, and let it catch up before moving forward. Major PostgreSQL upgrades still require a supported migration plan, and extension compatibility must be tested carefully. With healthy capacity elsewhere, applications can keep working through controlled routing changes and brief reconnects instead of one global maintenance outage.

Spock meshes three active regions; Patroni and etcd own local leadership; HAProxy and per-member PgBouncer protect each edge. If this was useful, subscribe - and I'll see you in the next build.