Vladimir Krinitsyn
The CAP theorem states that a distributed system can guarantee at most two of three properties: Consistency, Availability, and Partition Tolerance. For two decades, this theorem has been treated as a hard ceiling - a reason to accept painful trade-offs. YT Cluster challenges that framing not by breaking the theorem, but by making the trade-off programmable, situational, and transparent to the application. This article explains how, and walks through a live demonstration.
Cloud-native databases have taught us to accept their trade-offs. DynamoDB gives you availability and partition tolerance but eventual consistency by default. Google Spanner gives you consistency and availability but only inside Google’s private WAN. CockroachDB, TiDB, and YugabyteDB give you CP systems with impressive engineering - but at the cost of complexity, cloud lock-in, or a licensing model that punishes growth.
For organizations running on-premises or at the edge - manufacturing plants, hospitals, financial institutions, gaming backends, regional compliance environments - none of these options fit cleanly. You cannot run Spanner on-prem. You cannot afford CockroachDB’s licensing at scale. You cannot accept Cassandra’s eventual consistency for financial transactions.
The real problem is not the CAP theorem itself. The problem is that existing systems force you to pick a fixed point on the CAP triangle at design time, and live with that choice forever. YT Cluster’s answer is to make consistency level a runtime, per-transaction decision.
YT Cluster is a distributed database system built on top of standard PostgreSQL. It is implemented in Rust and delivered as a PostgreSQL extension (ytsetup). It transforms any set of PostgreSQL instances into a peer-to-peer distributed cluster - with no dedicated master node, no centralized coordinator, and no single point of failure.
Key facts:
Source: github.com/DBinvent/yt-cluster-tests
Product page: dbinvent.com/cluster
YT Cluster uses a Super-Shard-Cell (SSC) model for data partitioning. Unlike static consistent hashing (used by most distributed databases), SSC partitioning is dynamic: shards are reorganized at runtime based on load, access patterns, and node availability. An AI-assisted partitioning layer monitors query patterns and proposes rebalancing without downtime or data migration.
This is a fundamental departure from systems like Citus or Greenplum, where shard rebalancing requires explicit operator intervention and often temporary performance degradation.
Distributed transactions are coordinated using a RAFT consensus variant - but not in the traditional sense of a single RAFT group for the entire cluster. YT Cluster implements per-shard RAFT groups, so the transaction coordinator overhead scales horizontally with the number of shards. There is no global serialization bottleneck.
When a node is elected leader for a given term, it logs:
Term: 3, for 6sec., Leader
Non-leader nodes record the current leader’s address and port, routing write requests accordingly without a centralized proxy.
This is YT Cluster’s most distinctive architectural feature. Rather than offering a single consistency model cluster-wide, it exposes Programmable Synchronization Rules - per-table or per-transaction policies that define:
Example policy semantics:
-- Strong consistency: all replicas must ack
SYNCHRONIZE TABLE accounts WITH QUORUM = ALL;
-- Eventual: write proceeds when one replica acks
SYNCHRONIZE TABLE user_sessions WITH QUORUM = ONE;
-- Regional: write must be acked by at least one node per region
SYNCHRONIZE TABLE gdpr_data WITH QUORUM = REGIONAL;
This means a single cluster can simultaneously be:
The CAP trade-off is no longer a cluster property - it becomes a data classification problem.
The cluster supports hierarchical multi-region deployment. Nodes in the same data center form a local RAFT group. Data centers form a higher-level synchronization group. This allows:
This architecture directly addresses GDPR and data residency requirements: a table’s synchronization rule can enforce that its quorum is satisfied only by nodes in a specific geographic region.
| Scenario | Consistency | Availability | Partition Tolerance | YT Cluster behavior |
|---|---|---|---|---|
| Normal operation | Strong | Full | N/A | Synchronous replication, any node writable |
| Node failure (minority) | Strong | Full | Yes | Remaining nodes form quorum, no downtime |
| Network partition (minority isolated) | Strong | Partial | Yes | Majority partition continues; minority serves reads only |
| Network partition (50/50 split) | Degraded | Configurable | Yes | Per-table policy: block writes or accept local commits |
| Single-node “cluster” | Strong | Full | N/A | Standard PostgreSQL behavior, no overhead |
The crucial insight: partition tolerance is always guaranteed (you cannot opt out of it in a real distributed system). The programmable quorum rules let you choose, per table, per transaction, where on the C-A spectrum you want to land when partitions occur.
| System | Consistency model | On-prem | Postgres-native | Write scale |
|---|---|---|---|---|
| Citus | Strong (coordinator bottleneck) | Yes | Extension | Limited by coordinator |
| Greenplum | Strong (MPP) | Yes | Forked PG | Good for analytics |
| CockroachDB | Serializable (CP) | Yes ($$) | No | Linear |
| YugabyteDB | Tunable (CP default) | Yes | API-compatible | Linear |
| Spanner | External consistency | No | No | Linear |
| YT Cluster | Programmable | Yes | Extension | Linear |
YT Cluster’s key differentiators:
YT Cluster is part of a broader toolchain developed by Vladimir Krinitsyn and DBinvent:
sge) - open-source (MIT) PostgreSQL schema migration: versioned SQL scripts and declarative YAML, tracked by checksum per environment, built on the schema_guard library. On a cluster a change is rolled node by node while the cluster keeps serving.What is working today (MVP):
ytsetupcargo make)2025-2026 Roadmap:
Installation today (Debian/Ubuntu):
# Add DBinvent package repository
# Install PostgreSQL + YT Cluster extension
apt install yt-cluster
ytsetup # enables the extension in PostgreSQL
# Start a 3-node cluster for testing
cargo make --env PG_VER=14 --env CNT=3
This section outlines the live demonstration structure. Total time: 60 minutes.
./docker.sh for isolated environmentTalk track: Open with the CAP triangle on a whiteboard or slide. Ask the audience: “Which two would you pick?” Walk through why each classic answer hurts.
Demo: Show a traditional PostgreSQL streaming replication setup. Cause a failover. Show the promotion gap - the window where the system is unavailable. Ask: “What happened to CAP here?”
Demo:
# Terminal 1: start node 1
NID=1 ./start_node.sh
# Terminal 2: start node 2
NID=2 ./start_node.sh
# Terminal 3: start node 3
NID=3 ./start_node.sh
Show leader election output:
Term: 1, for 6sec., Leader # on one node
Term: 1, Leader: 127.0.0.1:5433 # on the other two
Talk track: No master configured. No pg_hba.conf primary/standby dance. The cluster elects a leader for each term dynamically.
Demo: Strong consistency write
-- On node 1
BEGIN;
INSERT INTO accounts (id, balance) VALUES (1, 1000);
COMMIT;
-- Immediately on node 3 (different terminal)
SELECT * FROM accounts WHERE id = 1;
-- Should return 1000 - synchronous replication
Demo: Eventual consistency write
-- Configure session table with ONE quorum
SYNCHRONIZE TABLE user_sessions WITH QUORUM = ONE;
-- Write on node 1
INSERT INTO user_sessions (token, uid) VALUES ('abc', 42);
-- Read immediately on node 3 - may or may not be there yet
SELECT * FROM user_sessions WHERE token = 'abc';
Talk track: Same cluster, same PostgreSQL, two different consistency behaviors. The CAP choice is now a schema design decision, not a deployment decision.
Demo: Kill node 3 (minority failure)
# Stop node 3
kill $(cat /tmp/pg_node3.pid)
Show that nodes 1 and 2 continue serving reads and writes. Show new leader election if node 3 was the leader:
Term: 2, for 6sec., Leader # new leader elected
Demo: Reconnect node 3
NID=3 ./start_node.sh
Show it catching up - no manual intervention, no data migration.
Demo: Network partition (advanced)
Use iptables or Docker network rules to split nodes 1+2 from node 3. Show the 50/50 behavior under different quorum policies.
Talk track: This is where CAP theorem bites traditional systems hard. With YT Cluster, you’ve already declared your preference per table. The cluster enforces your policy automatically.
Demo:
# A rolling schema change, no downtime: the same migration on each node in turn,
# followers first, the leader last, while the application keeps writing
for cfg in node2.cfg node3.cfg node1.cfg; do
sge -c "$cfg" --source_dir=./schema migrate || break
done
Show that the cluster keeps serving through the roll; that during it a migrated node reports the table as “schema differs” (select * from yt_info('W')) and is held out of row comparison rather than compared across two shapes; and that every node has the same schema once the roll ends. A rolled change is an expand step - a nullable column, a column with a default, an index - and the contract step comes only after every node has it. The roll stops at the first node that fails, and each node’s sge_history says exactly how far it got.
Talk track: Distributed schema changes are one of the hardest operational problems. Schema Guard treats the schema as a versioned artifact with its history in each node’s own database, and the cluster checks that every node agrees on it. Queuing the roll per node from a single deploy is designed, not yet released.
Suggested questions to prompt:
DBinvent - Making distributed databases accessible to organizations that cannot move to the cloud.