RockLake¶

What is RockLake?¶

RockLake is a DuckLake catalog server that stores your entire lakehouse metadata — every schema definition, every table registration, every file path, every column statistic — directly in object storage through SlateDB, an embedded LSM-tree key-value store built for the cloud. You point DuckDB at RockLake exactly the way you would point it at a PostgreSQL-backed DuckLake catalog, and everything just works. The difference is what you no longer need: no database server to provision, no connection pools to tune, no WAL archives to manage, no failover replicas to monitor. Your catalog becomes as durable and as available as your S3 bucket itself.

RockLake implements the full DuckLake catalog protocol — all 28 catalog tables, snapshot-based MVCC, atomic schema mutations — and exposes it over the PostgreSQL wire protocol. DuckDB connects, issues its catalog queries, and receives the responses it expects. Behind the scenes, RockLake translates those SQL statements into key-value reads and writes against SlateDB, which persists everything as immutable sorted-string-table files in your object store. The result is a catalog that inherits the durability, availability, and virtually unlimited capacity of cloud object storage without any of the operational overhead of running a database.

Why RockLake?¶

Every lakehouse needs a catalog — something that knows which Parquet files belong to which tables, what schema each table has, how the schema has evolved, and what the data looked like at any point in time. The question is where that catalog lives and what it costs you to keep it running.

RockLake wins on simplicity, durability, and horizontal read scale. It is intentionally slower per individual operation (object-store round trips cost 20–50 ms vs. 1–5 ms for local PostgreSQL), but for DuckLake's typical usage pattern — one catalog lookup per analytical query, then DuckDB reads Parquet files directly — that latency is negligible compared to the query execution time. If your workload involves hundreds of short catalog lookups per second, PostgreSQL is likely a better fit. If your workload is analytical queries against large tables with infrequent schema changes, RockLake eliminates an entire category of infrastructure without meaningful performance compromise.

The Architecture at a Glance¶

graph LR
    DDB[DuckDB + ducklake extension] -->|PostgreSQL wire protocol| SD[RockLake sidecar]
    SD -->|Key-value operations| SDB[SlateDB LSM-tree]
    SDB -->|GET / PUT / List| OBJ[S3 / GCS / Azure Blob]
    DDB -->|Direct Parquet I/O| OBJ

The separation of concerns is clean and deliberate. RockLake handles metadata — it knows where your data lives and what schema it has. DuckDB handles data — it reads and writes Parquet files directly from the object store. RockLake never sees your actual data, and DuckDB never needs to understand how the catalog is stored. This means a compromised catalog cannot corrupt your data files, and a compromised data-plane credential cannot corrupt your catalog.

Key Properties¶

Truly serverless. RockLake is a single static binary. You point it at a bucket path, and it opens (or creates) a catalog there. There is no schema migration step, no initialization script, no connection string to a backing database. When you are done, you stop the process. The catalog remains in the bucket, ready for the next writer to open it.

Infinite time travel. Every mutation to the catalog creates a new immutable snapshot. Old snapshots are never deleted unless you explicitly choose to run garbage collection followed by excision. You can query the catalog as it existed at any point in its history — last week, last month, last year — with a single SQL clause. This is not a premium feature bolted on after the fact; it is the natural consequence of the storage model.

Horizontal read scale-out with zero coordination. Because catalog data is stored as immutable files in object storage, any number of reader processes can open the catalog concurrently without communicating with each other or with the writer. There is no replication lag because there is no replication — every reader sees the same durable state directly from the object store. Adding ten more readers requires no configuration change, no leader election, and no additional infrastructure.

Crash safety by construction. SlateDB guarantees that a write either completes atomically or does not happen. There is no possibility of a partial update leaving the catalog in an inconsistent state. If the process crashes mid-transaction, the next startup sees a clean, consistent catalog with all previously committed transactions intact.

Drop-in DuckLake compatibility. RockLake supports exactly the SQL statement shapes that DuckDB's ducklake extension emits. Switching from a PostgreSQL-backed catalog to RockLake requires changing a connection string. No client code changes, no migration scripts, no schema adjustments.

Get Started¶