Composable CRDTs
Convergent by Construction

crdtlib is a Lean 4 library with reusable combinators that can be used to construct conflict-free replicated data types without needing to prove convergence at every step. The combinators each preserve convergence, so whatever you assemble from them is a CRDT by construction: verified in Lean and executable.

Examples

Each tab below is a CRDT assembled from the combinators. Watch an operation travel down through the construction, transform at each combinator, and the interpreted state come back up to the application.

Two CRDTs side by side

A task with tags and a priority

A grow-only set of tags next to a counter. disjoint_product keeps both: an operation .inl goes to the set, and .inr to the counter.

disjoint_product (gset String) counter
  1. .inl "urgent"→({urgent}, 0)
  2. .inr 2→({urgent}, 2)
  3. .inl "ux"→({urgent, ux}, 2)
  • disjoint_product

DisjointProduct Reference

A task with tags and a priority: disjoint_product (gset String) counter. Built from gset String, counter.app reads({urgent, ux}, 2)Go to referencedisjoint_producttwo CRDTs, side by sidecounterpriority2gset Stringtags{urgent, ux}

Conflict resolution on read

A last-writer-wins register

A multi-value register keeps all concurrent writes. On read, map_interpretation uses a user-defined total order to determine the latest among those values and returns it.

map_interpretation largest (mv_regₜ String)
  1. write "blue"→"blue"
  2. write "red" ∥→"red"
  3. write "green"→"green"
  • map_interpretation

LWW Register Reference

A last-writer-wins register: map_interpretation largest (mv_regₜ String). Built from mv_regₜ String.app reads"green"Go to referencemap_interpretationreads the largestmv_regₜ Stringconcurrent writes{green}

A keyed family of CRDTs

A counter for every element

The PN-Set: associate turns one counter into a counter per element. Adds count up, removes count down. The set keeps only the elements with a positive counter on interpretation.

map_interpretation positive (map_op ±1 (associate σ counter))
  1. add "milk"→{milk}
  2. add "eggs"→{milk, eggs}
  3. remove "milk"→{eggs}
  • map_interpretation
  • map_op
  • associate

PN-Set Reference

A counter for every element: map_interpretation positive (map_op ±1 (associate σ counter)). Built from counter.app reads{eggs}Go to referencemap_interpretationkeeps counts above 0Go to referencemap_opadd ↦ +1, remove ↦ −1Go to referenceassociate σa copy per elementcountercount{milk ↦ 0, eggs ↦ 1}

Observed-remove semantics

Concurrent adds survive removes

Adds carry a unique token, and a remove cancels only the tagged adds it has seen using their unique tokens. On interpretation, the set contains elements who's unique token was not removed.

map_interpretation (adds \ removes) (map_op route (disjoint_product union_set union_set))
  1. add milk #1→{milk}
  2. remove {(milk, 1)}→{}
  3. add milk #2→{milk}
  • map_interpretation
  • map_op
  • disjoint_product

OR-Set Reference

Concurrent adds survive removes: map_interpretation (adds \ removes) (map_op route (disjoint_product union_set union_set)). Built from union_set (σ × tok), union_set (σ × tok).app reads{milk}Go to referencemap_interpretationadds minus removesGo to referencemap_opadd → left, remove → rightGo to referencedisjoint_producttwo CRDTs, side by sideunion_set (σ × tok)removes{(milk, 1)}union_set (σ × tok)adds{(milk, 1), (milk, 2)}

A partially ordered list

An insert adds a node and, through traverse, two edges to a graph. Removals are tracked in a tombstone set. Reading the list traverses the graph in order.

map_interpretation po_traversal (disjoint_product (map_op _ (joint_product (gset σ) (po_edges σ))) (map_op _ (gset σ)))
  1. insert ⊥ H ⊤→"H"
  2. insert H i ⊤→"Hi"
  3. remove H→"i"
  • map_interpretation
  • disjoint_product
  • map_op
  • joint_product
  • traverse
  • associate

List Reference

A partially ordered list: map_interpretation po_traversal (disjoint_product (map_op _ (joint_product (gset σ) (po_edges σ))) (map_op _ (gset σ))). Built from gset σ, gset σ, gset σ.app reads"i"Go to referencemap_interpretationtraverses the graph in orderGo to referencedisjoint_productinsert → left, remove → rightGo to referencemap_opunwraps the elementgset σtombstones{H}Go to referencemap_opkeeps the elementGo to referencejoint_productupdates both, at onceGo to referencetraverseone insert, two edgesGo to referenceassociate σsuccessors per nodegset σedges{⊥→H, H→⊤, H→i, i→⊤}gset σnodes{H, i}

A tree of Tic-tac-toe games

Games fork into a tree, keyed by path, and every node has a board and a threaded chat. traverse turns one chat message into two list operations: create a message node, then write the text.

traverse route (associate (List String) (disjoint_product (mv_lwwₜ Board) (polist_with_valuesₜ Cursor (mv_lwwₜ String))))
  1. place [] ⊞→{[] ↦ (⊞, [])}
  2. chat [] "gg"→{[] ↦ (⊞, [gg])}
  3. fork [b] ⊞′→{[] ↦ (⊞, [gg]), [b] ↦ (⊞′, [])}
  • traverse
  • associate
  • disjoint_product

Paper Reference

A tree of Tic-tac-toe games: traverse route (associate (List String) (disjoint_product (mv_lwwₜ Board) (polist_with_valuesₜ Cursor (mv_lwwₜ String)))). Built from mv_lwwₜ Board, polist_with_valuesₜ.app reads{[] ↦ (⊞, [gg]), [b] ↦ (⊞′, [])}Go to referencetraverseone operation, several updatesGo to referenceassociate (List String)a game per pathGo to referencedisjoint_productboard and chatpolist_with_valuesₜthe chat{[] ↦ [gg]}mv_lwwₜ Boardthe board{[] ↦ ⊞, [b] ↦ ⊞′}

A JSON document, like Automerge

Objects are keyed by id, and each one is a map or a list, chosen by a tagged union. One edit fans out to several objects, and the document is read off the objects from root. It runs 2.4 to 6 times faster than Automerge 0.7 to 0.9.

traverse route (associate Oid (tagged_unionₜ (mapₜ Oid Key) (listₜ Oid Cursor)))
  1. put_list root "todos" o₁→{root ↦ {todos: o₁}, o₁ ↦ []}
  2. insert o₁ ⊥ "milk"→{root ↦ {todos: o₁}, o₁ ↦ [milk]}
  3. insert o₁ p₁ "eggs"→{root ↦ {todos: o₁}, o₁ ↦ [milk, eggs]}
  • traverse
  • associate
  • tagged_unionₜ

Paper Reference

A JSON document, like Automerge: traverse route (associate Oid (tagged_unionₜ (mapₜ Oid Key) (listₜ Oid Cursor))). Built from mapₜ Oid Key, listₜ Oid Cursor, mv_lwwₜ Choice.app reads{root ↦ {todos: o₁}, o₁ ↦ [milk, eggs]}Go to referencetraverseone edit, several updatesGo to referenceassociate Oidan object per idGo to referencetagged_unionₜa map or a listmv_lwwₜ Choicemap or list?{o₁ ↦ list}listₜ Oid Cursorlists{o₁ ↦ [milk, eggs]}mapₜ Oid Keymaps{root ↦ {todos: o₁}}
  • Principal combinator
  • Derived combinator
  • Primitive CRDT
  • CRDT built elsewhere from combinators
  • ωOperation
  • γInterpretation of the state

The model

The CRDT Type

Replicas apply operations as they arrive, in whatever order the network delivers them. A CRDT defines what an operation does to the local state, and what the application reads from it.

  • effect applies an operation to the state σ.

  • interpret turns the state into the interpreted state γ for the application.

  • commutative guarantees that events can be applied in any order and produce the same state.

Replicas that have seen the same operations end in the same state, without coordination: strong eventual consistency.

Read about the model →

structure CRDTₜ (τ : Type) [PartialOrder τ] (ω σ γ : Type*) where effect : Event τ ω → σ → σ interpret : σ → γ commutative : ∀ (e₁ e₂ : Event τ ω), Concurrent e₁.t e₂.t → (effect e₂) ∘ (effect e₁) = (effect e₁) ∘ (effect e₂)

Convergent by construction

Compose without a convergence proof burden

Two replicas converge Replica A applies add x and then add y. Replica B applies add y and then add x. Both end with the set containing x and y. Replica A Replica B add x add y {x} {x, y} {y} {x, y} same state
Replica A applies add x then add y; replica B the reverse. The two operations are concurrent, so commutative guarantees that applying them in both orders result in the same state.

Writing a CRDT by hand means proving commutative for the whole data type, and every change to it means proving it again.

Instead, each combinator automatically constructs a proof of commutativity (and thus convergence) for each constructed CRDT. This holds for any level of nesting, since the CRDT type carries the commutativity proof.

Next steps

Compose your first CRDT.