What is Ankurah?

Ankurah is a state-management framework for synchronizing data between connected nodes with built-in observability.

It supports multiple storage engines and per-field merge backends so applications can choose representations that fit their data.

Note: This project is beta status. It works, but be careful with production use.

Key Features

• Schema-First Design: Define data models using Rust structs with an ActiveRecord-style View/Mut interface
• Content-filtered pub/sub: Subscribe to changes on a collection using a SQL-like query
• Real-Time Observability: Signal-based pattern for tracking entity changes
• Distributed Architecture: Multi-node synchronization with event sourcing
• Flexible Storage: Implementations for Sled, SQLite, Postgres, and IndexedDB in the browser (with beta-level maturity)
• Shared model code: The same Rust models and query semantics compile for native servers and browser clients; React has maintained hooks and templates, while the Leptos bridge remains experimental

Core Concepts

• Model: A struct describing fields and types for entities in a collection (data binding)
• Collection: A group of entities of the same type (similar to a database table, and backed by per-collection tables in the PostgreSQL and SQLite engines)
• Entity: A discrete identity in a collection - Dynamic schema (similar to a schema-less database row)
• View: A read-only representation of an entity - Typed by the model
• Mut: A transaction-bound mutable handle for an entity - Typed by the model
• Event: An immutable change in an entity’s causal history, used for replay and synchronization

Design Philosophy

Ankurah follows an event-sourced architecture where:

• Every change is an immutable event whose id hashes the entity id, operation set, and parent clock
• Events form a per-entity DAG (like a git history); an entity’s current state points at the DAG’s head
• Entity ids are ULIDs, generated on any node without coordination
• Merge is deterministic per field: LWW fields use causal dominance plus a stable tiebreak for concurrent writes, while Yrs text fields combine CRDT updates – see Conflict Resolution & Guarantees

Quick Example

Create a live query on an initialized node:

// Using selection! macro with ctx.query()
let q: LiveQuery<AlbumView> = ctx.query(selection!("year > 1985"))?;

Then commit a model inside a transaction:

let trx = ctx.begin();

let album = trx.create(&Album {
    name: "Parade".into(),
    artist: "Prince".into(),
    year: 1986,
}).await?;

let album_id = album.id();
trx.commit().await?;

The local reactor updates matching queries after the commit. Connected peers receive matching changes through their subscriptions.

See Querying Data for the full query API, including the one-shot fetch() form and the fetch!/selection! macros.

Community

Join the conversation and contribute:

• GitHub Repository
• Discord Server

License

Ankurah is dual-licensed under MIT or Apache-2.0.

Quick Start (Template)

Prerequisites

Install a current stable Rust toolchain and cargo-generate before creating a project:

cargo install cargo-generate --locked

The web development scripts also require frontend-specific tools:

• React: wasm-pack and Bun
• Leptos: Trunk plus the Rust WASM target: rustup target add wasm32-unknown-unknown
• Postgres selection: Docker, because the React and Leptos scripts run a local database container

The generated scripts check their primary executables and print a missing-tool message. Install the Leptos WASM target explicitly; its script otherwise reports only that the Trunk preflight build failed.

Generate the project

Generate a new Ankurah app with cargo-generate, choosing the frontend that matches your application:

• Leptos — cargo generate https://github.com/ankurah/leptos-template
• React — cargo generate https://github.com/ankurah/react-template
• React Native — currently blocked at generation; see the note below

The React and Leptos templates generate working chat applications with shared Rust models, a durable server, a local client store, and live synchronization. Their client bindings and development tools differ:

React Native template status: as of July 2026, cargo-generate 0.23.12 against template main (f3c43b5) tries to substitute template text inside the binary iOS notification.caf asset and cargo generate https://github.com/ankurah/react-native-template exits with an invalid-syntax error. The asset is missing from the binary exclusions in the template’s cargo-generate.toml. The iOS steps below describe the intended generated project, but are not a working from-scratch quick start until that upstream template bug is fixed; follow react-native-template#10.

Choose the durable server’s storage engine at generation time: Sled (embedded, the default) or Postgres. Add --define storage=postgres to the generation command. The React and Leptos development runners manage a local Postgres container; a React Native server configured for Postgres reads DATABASE_URL and expects you to provide the database.

Run it

Leptos / React (web):

cd your-project-name
./dev.sh

dev.sh builds the server and frontend and starts them on randomized local ports — open the URL it prints. Stop with ./dev.sh --stop (--status, --logs, and --restart are also available).

React Native (iOS) — the current script requires macOS, Xcode 16.1+, Node 20+, Ruby/Bundler with CocoaPods, the Rust iOS targets, and an installed iPhone 16 simulator. Install the generated app’s dependencies first:

rustup target add aarch64-apple-ios-sim aarch64-apple-ios
cd react-app
npm install
bundle install
cd ios
bundle exec pod install
cd ../..

Then start the server and app in separate terminals:

cd your-project-name
cargo run -p your-project-name-server   # start the server (ws://localhost:9898)
./dev.sh                                # build the bindings + launch the iOS app

To see real-time synchronization in action, run the app as two independent nodes. On the web, open one regular browser tab and one incognito tab — the incognito tab gets its own IndexedDB store, so the two behave as separate nodes syncing through your server. The React Native dev.sh currently launches one fixed simulator target; a second client requires a separately configured simulator or device rather than another invocation of the documented script.

Need help? Join the Ankurah Discord!

Next Steps

• A Synchronized Feature, End to End – follow one model from Rust definition through replication and a reactive UI
• Defining Models – describe your data
• Querying Data – fetch and live-query it
• Run the Repository Examples – inspect the minimal examples in the Ankurah source tree

A Synchronized Feature, End to End

This guided trace connects the pieces you receive from an Ankurah template: a shared Rust model, a durable server node, an ephemeral client node, a live query, and a reactive component. The snippets are transcluded from this site’s Ankurah 0.9 example workspace so the Rust, WASM, and React builds validate the same code shown here.

This is an end-to-end tour of the site’s Album validation workspace, not a literal patch against the templates’ Message model. For a runnable generated application, complete Quick Start first; then use this tour to see where the equivalent model, server, binding, query, and UI pieces fit.

If you have not generated a project yet, start with the Quick Start. The template uses a chat model; this page uses a smaller Album model so the data flow is easy to see.

1. Define the shared model

Models live in the Rust model crate shared by the server and client bindings.

#[derive(Model, Debug, Serialize, Deserialize)]
pub struct Album {
    #[active_type(YrsString)]
    pub name: String,
    pub artist: String,
    pub year: i32,
}

Deriving Model implements the model contract for Album and generates the read-only AlbumView plus the transaction-bound AlbumMut handle. The user-defined Album struct is also the create input. See Defining Models for field types and Choosing a Merge Strategy for how each field resolves concurrent changes.

2. Start a durable node

The server owns a durable node and exposes it through the WebSocket connector:

let storage = SledStorageEngine::with_path(storage_dir)?;
let node = Node::new_durable(Arc::new(storage), PermissiveAgent::new());
node.system.wait_loaded().await;
if node.system.root().is_none() {
    node.system.create().await?;
}

let mut server = WebsocketServer::new(node);
println!("Running server...");
server.run("127.0.0.1:9797").await?;

wait_loaded() is the readiness barrier for checking the local system catalog before the code reads its root. The root is created only when the store has none, so restarting the process reopens the same system instead of trying to create a second one. PermissiveAgent is appropriate for local development only; use Authentication & Policy before exposing a server to untrusted clients. For the current readiness-wait edge case, see Deployment & Operations.

3. Create data in a transaction

Application model writes are transactional. Create the model, retain its generated id if you need it, and commit once:

let trx = ctx.begin();

let album = trx.create(&Album {
    name: "Parade".into(),
    artist: "Prince".into(),
    year: 1986,
}).await?;

let album_id = album.id();
trx.commit().await?;

The commit creates an immutable event, updates the local materialized state, and lets the node’s reactor update matching live queries. Connected durable peers receive the event; subscribed ephemeral peers receive matching changes through their live-query subscriptions.

4. Read once or stay subscribed

Use fetch() for a one-time snapshot:

// Fetch with a string query - one-time snapshot
let albums: Vec<AlbumView> = ctx.fetch("year > 1985").await?;

Use query() when the result set should continue updating:

// Using selection! macro with ctx.query()
let q: LiveQuery<AlbumView> = ctx.query(selection!("year > 1985"))?;

An entity can enter, change within, or leave a live query’s result set as commits apply. Querying Data explains that lifecycle; AnkQL Syntax covers predicates, ordering, limits, and safe value substitution.

5. Connect a browser node

The browser uses IndexedDB for its local working set and connects to a durable peer over WebSockets:

let storage = IndexedDBStorageEngine::open("myapp").await?;
let node = Node::new(Arc::new(storage), PermissiveAgent::new());
let client = WebsocketClient::new(node.clone(), server_url)?;
node.system.wait_system_ready().await;

let context = node.context(DEFAULT_CONTEXT)?;

CONTEXT.with(|slot| slot.replace(Some(context)));
CLIENT.with(|slot| slot.replace(Some(client)));

wait_system_ready() is the corresponding client-side readiness barrier for learning which system it joined. Keep the returned connector handle alive for as long as the node should remain connected.

Client initialization is asynchronous. Do not call ctx() or create queries during the first React render; wait for initialize_client() to resolve, then render the component that owns them. This hook makes that readiness state explicit:

let clientInitialization: Promise<void> | undefined;

function initializeClientOnce(): Promise<void> {
  clientInitialization ??= initialize_client("ws://localhost:9797");
  return clientInitialization;
}

function useClientReady() {
  const [ready, setReady] = useState(false);
  const [error, setError] = useState<string | null>(null);

  useEffect(() => {
    let cancelled = false;
    initializeClientOnce()
      .then(() => {
        if (!cancelled) setReady(true);
      })
      .catch((err) => {
        if (!cancelled) setError(String(err));
      });
    return () => {
      cancelled = true;
    };
  }, []);

  return { ready, error };
}

6. Observe the live query in React

The React templates create signalObserver with @ankurah/react-hooks; this small example defines the same wrapper directly from the generated useObserve binding. Create the live query only inside the subtree rendered after initialization:

export function signalObserver<T>(fc: React.FC<T>): React.FC<T> {
  return (props: T) => {
    const observer = useObserve();
    try {
      return fc(props);
    } finally {
      observer.finish();
    }
  };
}

let albumsQuery: AlbumLiveQuery | undefined;

function queryAlbums(): AlbumLiveQuery {
  albumsQuery ??= Album.query(ctx(), "year > 1985");
  return albumsQuery;
}

The factory caches the query for this app’s singleton client. That also makes the example safe when React development StrictMode probes a component more than once.

The ready-only component constructs that query once and passes it to the observed list:

const ReadyAlbums = signalObserver(() => {
  const albums = useMemo(queryAlbums, []);

  return (
    <div>
      <h2>{"Albums (year > 1985)"}</h2>
      <AlbumList albums={albums} />
    </div>
  );
});

Gate that subtree on the readiness state returned above:

const content = ready ? (
  <ReadyAlbums />
) : (
  <div className="status">
    Connecting to server at ws://localhost:9797...
  </div>
);

ReadyAlbums passes the query into an observed list component. Reading albums.items registers that render as a dependent of the live query:

interface Props {
  albums: AlbumLiveQuery;
}
/* Bind a React observer to the component. */
const AlbumList = signalObserver(({ albums }: Props) => {
  return (
    <ul>
      {/* Reading items registers this render as a live-query observer. */}
      {albums.items.map((album) => (
        <li key={album.id.to_base64()}>{album.name}</li>
      ))}
    </ul>
  );
});

The component rerenders when the query’s result set changes; application code does not need to maintain a second subscription or copy the rows into React state. See React Bindings for initialization and hook setup.

7. Write from React

The generated model namespace accepts the same transaction pattern from TypeScript. Create a row, then commit:

export async function createAlbum(
  name: string,
  artist: string,
  year: number,
): Promise<AlbumView> {
  const transaction = ctx().begin();
  const album = await Album.create(transaction, { name, artist, year });
  await transaction.commit();
  return album;
}

Views stay read-only. To update one, obtain its transaction-bound mutable handle and use the field backend’s mutation method before committing:

export async function renameAlbum(
  album: AlbumView,
  name: string,
): Promise<void> {
  const transaction = ctx().begin();
  album.edit(transaction).name.replace(name);
  await transaction.commit();
}

name is Yrs-backed text, so its wrapper exposes replace. An LWW field such as year exposes set instead.

8. Verify synchronization

The generated templates use Message rather than Album, but exercise the same sequence: a shared model, create/edit inside a transaction, a live query, and an observed component.

Run a generated React or Leptos project with ./dev.sh, then open the printed web URL in one regular browser window and one private/incognito window. The two windows use independent IndexedDB stores, so they behave as separate client nodes. Create or edit data in one and confirm that the other updates without a refresh. For an existing React Native checkout or previously generated project, start the Rust server as shown in the template and use a separately configured second simulator or device for the second client; the current generation blocker is noted in Quick Start, and dev.sh launches one fixed simulator target.

At that point you have exercised the complete path:

1. a model shared across server code and client code or generated bindings;
2. a transaction committed on one node;
3. replication through a connector;
4. storage on both durable and ephemeral nodes; and
5. a live query driving a reactive UI.

Where to go next

• Defining Models – add fields, references, and updates.
• Querying Data – choose between snapshots and live results.
• Authentication & Policy – replace the development policy agent.
• Deployment & Operations – choose durable storage and prepare a server.

Run the Repository Examples

Most projects should start from the template, then follow A Synchronized Feature, End to End. This page is for contributors and readers who want to run the smaller example applications maintained inside the Ankurah repository. It is not a from-scratch project setup guide.

Get the source

git clone https://github.com/ankurah/ankurah
cd ankurah

Prerequisites

• Install Rust:

  • rustup.rs

• Install Cargo Watch (useful for development workflow):

  cargo install cargo-watch
  

• Install wasm-pack:

  • wasm-pack installer

• Install Bun (used by the React example commands below):

  • Bun installation guide

Server Setup

Start the example server (keep this running):

cargo run -p ankurah-example-server

Or in development mode with auto-reload:

cargo watch -x 'run -p ankurah-example-server'

React Example App

1. Compile the WASM bindings in a second terminal:

   Navigate to the wasm-bindings example directory:

   cd examples/wasm-bindings
   wasm-pack build --target web --dev
   

   The one-off build exits when it finishes. For automatic rebuilds, keep this watcher running instead:

   cargo watch -s 'wasm-pack build --target web --dev'
   

2. Run the React example app in a third terminal, starting again from the repository root:

   cd examples/react-app
   bun install
   bun dev
   

3. Test the app:

   Load http://localhost:5173/ in one regular browser tab, and one incognito browser tab to see real-time synchronization in action!

   Note: You can also use two regular browser tabs, but they share one IndexedDB local storage backend, so incognito mode provides a better test of multi-node synchronization.

Leptos Example App

The maintained Leptos example is the standalone template; the older in-tree example is excluded from the Ankurah workspace. Clone ankurah/leptos-template and follow its README to run it. It uses Trunk to build and serve, and does not require building the Wasm bindings crate separately.

How It Works

In the example setup:

• The server is a durable node: it retains complete event history for the data it stores and answers other nodes’ retrieval requests.
• The browser is an ephemeral node. IndexedDB persists its synchronized working set across reloads, but the node is not expected to retain the system’s full history.
• The demo server currently uses the Sled backend; Postgres, SQLite, and (in the browser) IndexedDB are also supported.
• The WebSocket connector carries subscriptions, state, and events between the browser and server.

Next Steps

• Check out the Examples page for more code samples
• Learn how Ankurah works under the hood
• Read the Glossary to understand key terminology
• Join the Discord to ask questions and share your projects!

Defining Models

Models define the structure of your entities. Define them once in Rust, and they work everywhere—native servers, browser clients, and mobile apps.

Basic Model Definition

Use the #[derive(Model)] macro to define a model:

#[derive(Model, Debug, Serialize, Deserialize)]
pub struct Album {
    #[active_type(YrsString)]
    pub name: String,
    pub artist: String,
    pub year: i32,
}

This single definition generates:

Field Types

Basic Types

#[derive(Model, Debug, Serialize, Deserialize)]
pub struct Task {
    pub title: String,
    pub completed: bool,
    pub priority: i32,
}

Current built-in projected types include:

• String
• bool
• Integers: i16, i32, i64
• Floating point: f64
• Option<String>, Option<i32>, Option<i64>, Option<f64>
• Vec<u8>, Json, EntityId, and typed Ref<T> references

Plain String fields infer the Yrs text backend. Other built-ins infer LWW; use #[active_type(LWW)] when you explicitly want whole-value LWW semantics for a String.

CRDT Types

Use #[active_type(...)] to choose an active value backend explicitly. The shipped CRDT-backed type is YrsString for collaborative text:

#[derive(Model, Debug, Serialize, Deserialize)]
pub struct Document {
    #[active_type(YrsString)]
    pub content: String,
    pub title: String,
}

Entity References

Use Ref<T> to create typed references between entities:

#[derive(Model, Debug, Serialize, Deserialize, Clone)]
pub struct Artist {
    pub name: String,
}

#[derive(Model, Debug, Serialize, Deserialize, Clone)]
pub struct Song {
    pub title: String,
    pub artist: Ref<Artist>,
}

References enable graph-style navigation between related entities.

JSON Fields

Use Json for schemaless, dynamic data:

#[derive(Model, Debug, Serialize, Deserialize, Clone)]
pub struct Track {
    pub name: String,
    pub metadata: Json,
}

JSON fields support nested path queries like metadata.genre = 'rock'.

Creating Entities

Use a transaction to create new entities:

let trx = ctx.begin();

let album = trx.create(&Album {
    name: "Parade".into(),
    artist: "Prince".into(),
    year: 1986,
}).await?;

let album_id = album.id();
trx.commit().await?;

Reading Entities

Access data through the View type:

let view: AlbumView = ctx.get(album_id).await?;
println!("Album: {} by {} ({})", view.name()?, view.artist()?, view.year()?);

Updating Entities

Views remain read-only. To update an entity, edit the View inside a transaction, call the active field type’s mutation method, and commit:

let trx = ctx.begin();
let album = view.edit(&trx)?;
album.name().replace("Parade - Music from the Motion Picture")?;
album.year().set(&1987)?;
trx.commit().await?;

Here name() is a YrsString, so it offers text operations such as insert, delete, overwrite, and replace. year() is an LWW<i32>, so it uses set. Mutation handles stop accepting writes when their transaction closes.

Generated TypeScript

When you build your WASM bindings, TypeScript types are generated automatically:

Creation and mutation use the generated model and View APIs:

export async function createAlbum(
  name: string,
  artist: string,
  year: number,
): Promise<AlbumView> {
  const transaction = ctx().begin();
  const album = await Album.create(transaction, { name, artist, year });
  await transaction.commit();
  return album;
}

export async function renameAlbum(
  album: AlbumView,
  name: string,
): Promise<void> {
  const transaction = ctx().begin();
  album.edit(transaction).name.replace(name);
  await transaction.commit();
}

The generated surface includes the model’s creation/query namespace (Album), read-only AlbumView, transaction-bound AlbumMut, typed AlbumLiveQuery, result/change-set wrappers, and typed reference wrappers. View fields and live-query results are JavaScript getters (album.name, albums.items); mutations go through the active field wrapper returned by album.edit(transaction).

Next Steps

• Querying Data - How to query and filter entities
• Query Syntax - Full AnkQL syntax reference

Choosing a Merge Strategy (LWW vs Yrs)

Every replicated field in a model has a merge strategy: the rule that decides what happens when two nodes change that field concurrently. Picking one is a modeling decision you make per field, at definition time, and it is worth a moment of thought – “one writer wins” and “CRDT updates combine” are very different user experiences. Fields marked #[model(ephemeral)] are not replicated and do not use a property backend.

Under the hood each strategy is a property backend: the component that owns a property’s operation format and implements its merge policy. This page covers the two shipped backends and how to choose between them; the engine-facing contract and the full resolution algorithm live in the contributor Internals section.

The active backends

Model fields choose their backend at definition time. String fields default to Yrs text; a field can opt into LWW explicitly:

use ankurah::Model;
use serde::{Deserialize, Serialize};

#[derive(Model, Debug, Serialize, Deserialize)]
pub struct Record {
    #[active_type(LWW)]
    pub title: String,     // one winner under concurrency
    pub notes: String,     // Yrs text: concurrent edits interleave
}

How operations travel

When a transaction commits, each backend is asked for the operations performed since the last drain (to_operations). Whatever it returns is an opaque byte diff from everyone else’s perspective. The event stores these per backend name:

Event {
    collection, entity_id, parent,
    operations: {
        "lww": [ ...opaque diffs... ],
        "yrs": [ ...opaque diffs... ],
    },
}

collection travels with the event but is deliberately excluded from the EventId; the hash covers entity_id, operations, and parent.

On the receiving side, the backend name routes each diff back to the right backend. Backends also serialize a state buffer (their full current state) for snapshots; from_state_buffer reconstitutes a backend without replaying history.

The trait, in terms of responsibilities

PropertyBackend (in core/src/property/backend/mod.rs) asks each backend to be able to:

• Report itself: property_backend_name(), properties(), property_values().
• Round-trip state: to_state_buffer / from_state_buffer.
• Emit changes: to_operations, draining writes made through the model API since the last drain.
• Apply changes: apply_operations (no provenance) and apply_operations_with_event (with the writing event’s id). CRDTs ignore the event id; LWW records it as provenance per property.
• Resolve concurrency: apply_layer, the only method with no default. It receives an EventLayer and must implement the backend’s merge policy.

The iterator emits topological sweep generations, earliest first. Same-layer events in the divergent region are concurrent, but the sweep can also emit accumulated events below the meet as inert already_applied context. Such a context event can be causally related to a divergent event in the same layer. Backends therefore must use layer.compare, not layer membership or iteration order, to determine causality:

• already_applied events are context: their effects are in your state;
• to_apply events are new: fold them in according to your policy;
• the layer carries the accumulated DAG, so you can compare any two event ids (layer.compare) and test whether an id was part of the explored graph at all (layer.dag_contains).

Entities feed layers to every backend that appears in the merge, and if an event introduces a backend the entity has never seen, the new backend is created and replayed with all earlier layers first, so late-appearing backends do not miss context.

Yrs: concurrency handled by the data type

The Yrs backend wraps a yrs CRDT document. Its operations are encoded document updates; its state buffer is the whole document encoded as one update.

Its apply_layer is almost trivial, and that is the point:

for each to_apply event:
    apply its updates to the document

CRDT updates are commutative and idempotent, so order within the layer does not matter and already_applied context is unnecessary. Concurrent insert runs survive and whole-field replace calls may visibly interleave their insertions. The result converges, but convergence is not the same as preserving each author’s higher-level intent. Choose Yrs when collaborative text semantics fit the product, not as a promise of universally lossless editing.

LWW: one winner per property, chosen causally

The LWW backend keeps, per property, the current value plus the id of the event that wrote it. That provenance is what makes principled resolution possible later.

apply_layer runs a per-property tournament:

1. Seed with the stored value. The current value and its writing event enter as the incumbent candidate. If the stored provenance id is absent from the accumulated DAG, the implementation marks the incumbent older_than_meet; a candidate for that property in the current layer replaces it, after which normal causal/tiebreak comparisons apply. This is a conservative engine fallback, not proof that every event in the layer causally descends the incumbent.
2. Consider every event in the layer (both already_applied for context and to_apply), extracting each property write as a candidate.
3. Pairwise resolution between the current winner and each candidate:
   • if one causally descends the other, the newer one wins;
   • if they are truly concurrent, the higher event id wins. The id is a content hash, so this tiebreak is arbitrary but stable: every node picks the same winner with no coordination.
4. Mutate only for to_apply winners. If the tournament is won by something already reflected in state, there is nothing to write. Wins that do mutate also fire that property’s change signals.

Worth internalizing: resolution is per property. One event may win title while a concurrent event wins artist. LWW merges at property granularity, not event granularity.

The provenance invariant

The stored per-property event id is not decorative. The system maintains the invariant that a stored entry is already the LWW winner among all events in the head’s ancestry that touch that property, because heads only advance after an event’s operations are applied. Resolution therefore never needs to re-litigate history behind the stored entry; the incumbent faithfully represents everything below it. This invariant is pinned by tests, including an artificial construction demonstrating what would break without it.

Extending the built-in backend set

This is currently a contributor workflow inside the Ankurah repository, not a stable external plugin API. The event-DAG layer interfaces needed by a backend are crate-private, backend_from_string is hardcoded, and the derive registry loads only the built-in Yrs and LWW definitions. A future public registry would need to expose those seams first.

Within the core repository, a new backend is viable if it can honestly implement the contract:

1. Operations must round-trip: whatever to_operations emits, apply_operations must reproduce on another node.
2. State buffers must round-trip losslessly, including whatever provenance the backend needs for future resolution.
3. apply_layer must be deterministic given the same layer and prior state, and must not infer causality from layer membership or iteration order; use layer.compare.
4. Resolution may only depend on the graph (via layer.compare / layer.dag_contains) and event contents, never on wall clocks or arrival order. That is the property that makes every node converge.

Register the backend name in backend_from_string and extend the derive registry/configuration. The surrounding event pipeline is designed to remain unchanged, but external backend registration is unfinished in 0.9.

Querying Data

Ankurah provides a SQL-like query language called AnkQL for filtering and retrieving entities. The same public query API targets Postgres, SQLite, Sled, and browser IndexedDB, although execution strategy and backend maturity differ.

Two Ways to Query

There are two fundamental patterns for getting data:

fetch() - One-Time Snapshot

Use fetch() when you need data once and don’t need ongoing updates:

// Fetch with a string query - one-time snapshot
let albums: Vec<AlbumView> = ctx.fetch("year > 1985").await?;

The results are a Vec<AlbumView> containing all matching entities at that moment. If the data changes later, you won’t be notified.

query() - Live Subscription

Use query() when your UI should update automatically as data changes:

// Using selection! macro with ctx.query()
let q: LiveQuery<AlbumView> = ctx.query(selection!("year > 1985"))?;

A LiveQuery is reactive: local commits and remote changes delivered to the node can add an entity to the result set, update one already in it, or make one leave the predicate. The query updates automatically as those changes apply.

Query Methods

Using Macros (Recommended)

The recommended way to query is using the fetch! and selection! macros, which provide compile-time safety and variable interpolation:

Variable Interpolation with Macros

Use the fetch! and selection! macros for dynamic queries. They support multiple syntaxes:

Unquoted Form (Terse)

The unquoted form is the most concise. Variables expand to equality comparisons by default:

// Unquoted form: {variable} expands to variable = {variable}
let artist = "Prince";

let albums: Vec<AlbumView> = fetch!(ctx, {artist}).await?;

Add comparison operators as prefixes: {>year}, {<year}, {>=year}, {<=year}, {!=year}:

// Unquoted form with comparison operator: {>year} expands to year > {year}
let year = 1985;

let albums: Vec<AlbumView> = fetch!(ctx, {>year}).await?;

Combine multiple conditions with AND/OR:

// Combine multiple conditions with AND/OR
let artist = "Prince";
let year = 1985;

let albums: Vec<AlbumView> = fetch!(ctx, {artist} AND {>year}).await?;

Mix unquoted variables with explicit comparisons:

// Mix unquoted variables with explicit comparisons
let artist = "Prince";
let year = 1985;

let albums: Vec<AlbumView> = fetch!(ctx, {artist} AND year > {year}).await?;

Quoted Form (Flexible)

Use quoted form for string literals and positional arguments:

// Quoted form for pure string literals
let albums: Vec<AlbumView> = fetch!(ctx, "artist = 'Prince' AND year > 1985").await?;

Positional arguments with {}:

Placeholders are typed values. Do not wrap {} in AnkQL quotes, even when the argument is a string; quotes are only for string literals written directly in the query.

// Quoted form with positional arguments
let min_year = 1980;
let max_year = 1990;

let albums: Vec<AlbumView> = fetch!(ctx, "year >= {} AND year <= {}", min_year, max_year).await?;

Multiple positional arguments:

// String and numeric positional arguments are typed; do not quote `{}` placeholders.
let artist = "Prince";
let year = 1985;

let albums: Vec<AlbumView> = fetch!(ctx, "artist = {} AND year > {}", artist, year).await?;

With query() and selection!

The same syntaxes work with ctx.query(selection!(...)):

// Unquoted form with selection! macro
let artist = "Prince";

let live: LiveQuery<AlbumView> = ctx.query(selection!({artist}))?;

// Unquoted form with comparison operator
let year = 1985;

let live: LiveQuery<AlbumView> = ctx.query(selection!({>year}))?;

// Combine conditions with AND/OR
let artist = "Prince";
let year = 1985;

let live: LiveQuery<AlbumView> = ctx.query(selection!({artist} AND {>year}))?;

// Quoted form for string literals
let live: LiveQuery<AlbumView> = ctx.query(selection!("artist = 'Prince' AND year > 1985"))?;

Next Steps

• Query Syntax - Learn the full AnkQL query language
• React Bindings - Using queries in React components

AnkQL Syntax

AnkQL is Ankurah’s query language for filtering entities. It uses familiar SQL-like syntax that works consistently across all storage backends.

Basic Comparisons

field = value       # Equality
field != value      # Not equal
field > value       # Greater than
field >= value      # Greater than or equal
field < value       # Less than
field <= value      # Less than or equal

Examples

let albums: Vec<AlbumView> = ctx.fetch("name = 'Dark Side of the Moon'").await?;

let albums: Vec<AlbumView> = ctx.fetch("year > 1985").await?;

let albums: Vec<AlbumView> = ctx.fetch("artist != 'Unknown'").await?;

Logical Operators

Combine conditions with AND and OR:

condition1 AND condition2
condition1 OR condition2

Use parentheses for complex logic:

(condition1 OR condition2) AND condition3

Examples

let albums: Vec<AlbumView> = ctx.fetch("year > 1980 AND year < 1990").await?;

let albums: Vec<AlbumView> = ctx.fetch("artist = 'Prince' OR artist = 'Madonna'").await?;

let albums: Vec<AlbumView> = ctx.fetch("(artist = 'Prince' OR artist = 'Madonna') AND year > 1985").await?;

The IN Operator

Check if a value is in a list:

field IN (value1, value2, value3)

Example

let albums: Vec<AlbumView> = ctx.fetch("year IN (1984, 1985, 1986)").await?;

Ordering Results

Use ORDER BY to sort results:

... ORDER BY field ASC
... ORDER BY field DESC

Examples

let albums: Vec<AlbumView> = ctx.fetch("year > 1980 ORDER BY year DESC").await?;

let albums: Vec<AlbumView> = ctx.fetch("true ORDER BY name ASC").await?;

Selecting All Entities

Use true to match all entities:

let albums: Vec<AlbumView> = ctx.fetch("true ORDER BY name ASC").await?;

String Values

String literals use single quotes:

let albums: Vec<AlbumView> = ctx.fetch("name = 'Purple Rain'").await?;

For a value containing a single quote, use typed substitution. AnkQL does not implement SQL’s doubled-quote escape inside string literals:

let name = "Rock 'n' Roll";
let albums: Vec<AlbumView> = fetch!(ctx, "name = {}", name).await?;

Variable Interpolation

Use the fetch! and selection! macros for dynamic queries. They support multiple syntaxes:

Unquoted Form

The unquoted form is the most concise. Variables expand to equality by default:

let artist = "Prince";
fetch!(ctx, {artist}).await?;  // Equivalent to: artist = 'Prince'

Add comparison operators as prefixes:

// Unquoted form: {>year} expands to year > {year}
let year = 1985;

let albums: Vec<AlbumView> = fetch!(ctx, {>year}).await?;

All comparison operators work: {>var}, {<var}, {>=var}, {<=var}, {!=var}:

// All comparison operators work: >, <, >=, <=, !=
let year = 1985;

let _newer: Vec<AlbumView> = fetch!(ctx, {>year}).await?;
let _older: Vec<AlbumView> = fetch!(ctx, {<year}).await?;
let _gte: Vec<AlbumView> = fetch!(ctx, {>=year}).await?;
let _lte: Vec<AlbumView> = fetch!(ctx, {<=year}).await?;
let _not_eq: Vec<AlbumView> = fetch!(ctx, {!=year}).await?;

Combine conditions with AND/OR:

// Combine multiple conditions with AND/OR
let artist = "Prince";
let year = 1985;

let albums: Vec<AlbumView> = fetch!(ctx, {artist} AND {>year}).await?;

Mix unquoted variables with explicit comparisons:

// Mix unquoted variables with explicit comparisons
let artist = "Prince";
let year = 1985;

let albums: Vec<AlbumView> = fetch!(ctx, {artist} AND year > {year}).await?;

Quoted Form

Use quoted form for string literals and positional arguments:

Placeholders are typed values. Do not wrap {} in AnkQL quotes, even when the argument is a string; quotes are only for string literals written directly in the query.

// String positional arguments are typed; do not quote `{}` placeholders.
let artist = "Prince";

let albums: Vec<AlbumView> = fetch!(ctx, "artist = {}", artist).await?;

Multiple variables:

// Multiple variables with quoted form
let min_year = 1980;
let max_year = 1990;

let albums: Vec<AlbumView> = fetch!(ctx, "year >= {} AND year <= {}", min_year, max_year).await?;

Pure string literals (no variables):

// Quoted form for pure string literals
let albums: Vec<AlbumView> = fetch!(ctx, "artist = 'Prince' AND year > 1985").await?;

Common Patterns

Check if entity exists

// Check if any entities match the query
let album_name = "Purple Rain";
let matching_albums: Vec<AlbumView> = fetch!(ctx, "name = {}", album_name).await?;
let exists = matching_albums.len() > 0;

Get first match

let album = ctx.fetch::<AlbumView>("name = 'Purple Rain'").await?.into_iter().next();

Count matches

let count = ctx.fetch::<AlbumView>("year > 1985").await?.len();

Next Steps

• Querying Data - Overview of fetch vs query
• React Bindings - Using queries in React components

Reactivity & Signals

Ankurah’s headline behavior – UIs that update when local commits or delivered remote changes apply to their node – is built on a small set of signal-aware read surfaces. Reading a live query, View, or signal value inside an observed scope records a dependency. When a local or remote change applies, Ankurah notifies the dependents whose data changed; there is no application-level polling loop or event bus to wire up.

This page covers the model and the framework-agnostic API. For React specifics see React Bindings.

Three things you can observe

A live query’s results. ctx.query(...) returns a LiveQuery<View>. Subscribing yields a ChangeSet describing exactly what happened:

use ankurah::signals::Subscribe;
let live: LiveQuery<AlbumView> = ctx.query("year > 2000")?;
live.wait_initialized().await;
let _guard = live.subscribe(|changes| {
    println!("Received changes: {changes}");
});

Each change in the set is Initial, Add (entered the result set), Update (already in the set, fields changed), or Remove (left the set). A LiveQuery is also a signal itself: reading its items inside an observed scope subscribes that scope to future changes.

A single entity. Every View implements Subscribe; the listener fires on any field change to that entity:

let _guard = album.subscribe(|view: AlbumView| {
    // any field of this entity changed
});

A single field is not a public read-side signal yet. Transaction-bound mutable field wrappers expose backend signal machinery, but their broadcasts belong to the transaction fork and are not a reliable observer of committed state. To react to application data, observe the whole read-side View or a LiveQuery instead.

Subscribe vs. observe

There are two consumption styles, and framework integrations use the second:

• Subscribe – explicit: thing.subscribe(listener) returns a guard. You get the new value pushed to your callback.
• Observe (tracking) – implicit: run code inside an observer scope, and every signal it reads becomes a dependency. On any change, the scope re-runs (or re-renders). This is how the React and Leptos integrations work, and why component code contains no subscription bookkeeping at all – reading livequery.items inside an observed component is the subscription.

Rules that matter

1. Keep the guard. subscribe() returns a guard; dropping it unsubscribes. Assign it to a binding that lives as long as you want notifications (let _guard = ... – not let _ = ..., which drops immediately).
2. Account for live-query initialization. A listener attached before a LiveQuery initializes can receive its initial ChangeSet. The example above awaits wait_initialized() first, so its callback sees only later changes. Other already-initialized signals run listeners on changes, not merely because you subscribed. Observer-style consumers naturally render the current value first and then re-render on later changes.
3. One notification per commit. A transaction touching several fields of an entity produces a single notification per subscriber, after the commit applies – not one per field write.
4. Delivery is synchronous. Listeners run inline when the change applies, on whatever thread applied it. Keep listeners cheap; hand off heavy work.

Signals for your own state

The same primitives that power entity reactivity are exported for application state, so derived values can mix Ankurah data with local state in one dependency graph:

Framework wiring

React – the @ankurah/react-hooks package exports a factory:

import React from "react";
import { createAnkurahReactHooks } from "@ankurah/react-hooks";
import { ReactObserver } from "your-wasm-bindings";

const { signalObserver } = createAnkurahReactHooks({ React, ReactObserver });

Wrap components in signalObserver(...); it brackets the render’s tracking lifecycle, and any signal read during that render – a LiveQuery’s items, a View’s fields – subscribes the component via React’s useSyncExternalStore. There is deliberately no use_query hook: queries are plain LiveQuery objects you create and read; observation is the only React-specific part. Full usage: React Bindings.

The factory only wires observation. Your application must initialize its WASM module and Ankurah client before rendering code that calls ctx(); the React setup shows the maintained template’s startup order.

Leptos – support is built into ankurah-signals via the reactive-graph feature (enabled by default): install the bridge once at startup,

// Install the ReactiveGraphObserver at the base of the Ankurah observer stack
// so that Leptos components can observe Ankurah signals via reactive_graph.
use ankurah_signals::{CurrentObserver, ReactiveGraphObserver};

CurrentObserver::set(ReactiveGraphObserver::new());

then read Ankurah signals (livequery.get(), view fields) inside Leptos closures like any other reactive value – dependencies register with Leptos’s reactive graph automatically. The Leptos template shows the wiring end to end. The bridge works, but remains the younger, experimental frontend path; expect rough edges and use the maintained React path when you need the more established integration.

How it connects underneath

Per node, a reactor matches every applied entity change against active query subscriptions, updates each query’s result set, and emits the ChangeSets described above; entity and field broadcasts fire from the same applied-change notification path. The machinery – and the ordering guarantees behind “one notification per commit” – is contributor territory: see Entity Lifecycle and Conflict Resolution & Guarantees.

Queries in React

Ankurah’s Model derive generates TypeScript classes for queries, views, and mutation handles. React observation lives in the separate @ankurah/react-hooks package, which you bind to the ReactObserver exported by your WASM crate.

Setup

Create one app-local hooks module, src/ankurah-hooks.ts:

import React from "react";
import { createAnkurahReactHooks } from "@ankurah/react-hooks";
import { ReactObserver } from "your-wasm-bindings";

export const { useObserve, signalObserver } =
  createAnkurahReactHooks({ React, ReactObserver });

Initialize the WASM module and your app’s Ankurah client before rendering a component that calls ctx(). The maintained React template exposes that second readiness step as ready():

import { createRoot } from "react-dom/client";
import initBindings, { ready } from "your-wasm-bindings";
import App from "./App";

async function main() {
  await initBindings();
  await ready();
  createRoot(document.getElementById("root")!).render(<App />);
}

void main();

If your binding crate exposes a differently named initializer, await that instead; ready() is application wiring from the template, not a method generated by Model.

Creating Queries

The examples below use React’s lifecycle hooks plus types generated by the Ankurah model bindings:

import { useEffect, useMemo, useState } from "react";

import {
  Album,
  ctx,
  AlbumLiveQuery,
  AlbumView,
} from "ankurah-org-example-wasm-bindings";

Import the wrapper from the app-local hooks module created in Setup:

import { signalObserver } from "./ankurah-hooks";

Use the static .query() method on any model class:

let albumsQuery: AlbumLiveQuery | undefined;

function queryAlbums(): AlbumLiveQuery {
  albumsQuery ??= Album.query(ctx(), "year > 1985");
  return albumsQuery;
}

The query returns immediately with a LiveQuery object. Results stream in as they become available. This example caches the query because its Ankurah client is also a singleton; that avoids duplicate registration if React development StrictMode probes a component twice.

Signal Observation

Ankurah uses signals for reactivity. To make a React component reactive, wrap it with signalObserver:

interface Props {
  albums: AlbumLiveQuery;
}
/* Bind a React observer to the component. */
const AlbumList = signalObserver(({ albums }: Props) => {
  return (
    <ul>
      {/* Reading items registers this render as a live-query observer. */}
      {albums.items.map((album) => (
        <li key={album.id.to_base64()}>{album.name}</li>
      ))}
    </ul>
  );
});

The signalObserver wrapper:

1. Creates a reactive observer for the component render
2. Automatically tracks which signals are accessed during render
3. Re-renders the component when those signals change

How It Works

When you access albums.items inside a component wrapped with signalObserver, the observer tracks this access. When the live query’s results change—whether from local changes or remote sync—the component automatically re-renders.

Creating Entities

Use a transaction to create new entities:

export async function createAlbum(
  name: string,
  artist: string,
  year: number,
): Promise<AlbumView> {
  const transaction = ctx().begin();
  const album = await Album.create(transaction, { name, artist, year });
  await transaction.commit();
  return album;
}

Updating Entities

Views are read-only. Call .edit(transaction) to obtain the generated mutable handle, mutate its active field wrapper, then commit:

export async function renameAlbum(
  album: AlbumView,
  name: string,
): Promise<void> {
  const transaction = ctx().begin();
  album.edit(transaction).name.replace(name);
  await transaction.commit();
}

String fields use the Yrs text backend by default, hence .replace(...). LWW-backed fields expose .set(...) instead.

Querying All Entities

Use an empty selection string to match all entities. Give this query the same explicit lifetime as a filtered query:

const allAlbums = Album.query(ctx(), "");

Query Lifetime

Creating a LiveQuery registers a live resource; do not recreate one on every render. For a fixed, app-lifetime query, use a cached factory such as queryAlbums() above. The example then calls that factory from useMemo in the ready-only component.

For a component-owned dynamic query, construct it in an effect and explicitly free the previous WASM handle during cleanup:

export function useAlbumsByArtist(artist: string): AlbumLiveQuery | null {
  const [albums, setAlbums] = useState<AlbumLiveQuery | null>(null);

  useEffect(() => {
    const query = Album.query(ctx(), "artist = ?", artist);
    setAlbums(query);
    return () => query.free();
  }, [artist]);

  return albums;
}

Use ? placeholders and variadic substitution values for dynamic input. Do not interpolate user-controlled strings into AnkQL source.

Reactive State with JsValueMut

For local reactive state that integrates with the signal system:

import { useMemo } from "react";
import { AlbumView, JsValueMut } from "your-wasm-bindings";

function useSelectedAlbum() {
  const [selectedAlbum, selectedAlbumRead] = useMemo(
    () => JsValueMut.newPair<AlbumView | null>(null),
    [],
  );

  // Reading inside signalObserver tracks this value.
  const album = selectedAlbumRead.get();

  const selectAlbum = (next: AlbumView | null) => selectedAlbum.set(next);
  return { album, selectAlbum };
}

Next Steps

• Reactivity & Signals - The signals model behind these hooks
• Querying Data - Overview of fetch vs query
• Query Syntax - Full AnkQL syntax reference

Authentication & Policy

Every Ankurah node runs a policy agent: the component that identifies request contexts and supplies access-control hooks for local operations and peer requests. Each context carries the agent’s notion of an identity.

The examples elsewhere in this book use PermissiveAgent::new() – the development baseline that performs no authentication and no authorization: every gate is a no-op allow. Ship something real and you will want the JWT extension below, or your own PolicyAgent implementation.

What a policy agent decides

The PolicyAgent trait gates, in user terms:

Local denials surface as errors on the failing call (create, commit, fetch, get). Remote write and coarse request denials return request errors, not dropped connections. Remote row-level read denials are intentionally filtered out of get, fetch, and initial-subscription results, so an entity can look absent instead of returning ByPolicy.

The JWT extension

ankurah-jwt-auth provides JwtAgent: RS256-signed JWTs plus a JSON policy of roles, collection privileges, and row-level scope rules.

Enable the watcher feature in the durable server. Without it, new_durable reads the policy into memory but does not publish or refresh the replicated JwtPolicy entity used by ephemeral clients:

ankurah-jwt-auth = { version = "0.9", features = ["watcher"] }

Server – a durable node with signing keys and a policy file:

let keys = SigningKeys::from_pem(include_str!("path/to/private_key.pem"))?;
let agent = JwtAgent::new_durable(keys.clone(), "policy.json")?;

let node = Node::new_durable(Arc::new(storage), agent);
node.system.wait_loaded().await;
if node.system.root().is_none() {
    node.system.create().await?;
}

Authenticated context – verify the token before turning its claims into the context every operation runs under. JwtContext::from_claims is only a constructor; it does not verify the token itself:

let claims = keys.verify(&token)?;
let ctx = JwtContext::from_claims(claims, token);
let context = node.context(ctx)?;

let trx = context.begin();
trx.create(&Post { title: "Hello World".into(), body: "First post!".into() }).await?;
trx.commit().await?;

Browser-shaped clients start with no key material and never hold the private signing key or policy file. Construct the agent with JwtAgent::new_ephemeral(): with the durable watcher enabled, it syncs the policy config and server’s public key over the normal replication channel (they live in a JwtPolicy entity that only the server’s root context can write). Local application code must still verify a token before constructing a trusted JwtContext; from_claims does not do that automatically. Outgoing requests carry the raw JWT, and the receiving server independently verifies it in check_request.

The policy file

Two maps: roles grant named privileges, and collections require them.

{
  "roles": {
    "Admin":  ["*"],
    "Editor": ["view_posts", "write_posts", "manage_posts"],
    "Author": ["view_posts", "write_posts"]
  },
  "collections": {
    "post": {
      "read":  "view_posts",
      "write": "write_posts",
      "scope": [
        { "filter": "author = $jwt.sub", "unless_privilege": "manage_posts" }
      ]
    }
  }
}

• read / write name the intended operation privileges; "*" on a role is a full-access wildcard. The current coarse query-access caveat is called out under limitations below.
• scope rules add row-level restriction: the filter is an AnkQL predicate that is AND-ed onto every query the user runs. Here, Authors may read and write only their own posts (author = $jwt.sub). Editors also hold manage_posts, so they bypass the rule via unless_privilege while still satisfying the shared write_posts collection gate.

Scope details that matter in practice:

• Claims become literals, safely. $jwt.sub, $jwt.email, $jwt.name, and $jwt.custom.<field> substitute as values into the parsed predicate – claim content can never alter the filter’s structure, and a missing claim fails closed.
• Rules compose with AND, fail-closed: multiple rules all apply.
• applies_to scopes a rule to "read", "write", or both (default). A write-only rule gates mutations without hiding rows.
• Scope checks also cover point reads and transaction writes. Point reads re-evaluate scope against the entity’s actual state, and writes are checked against both the before and after state – so an update cannot move a row into or out of your scope to dodge the rule.

Current limitations (read before shipping)

Honest edges of the extension as it stands today:

• Policy enforcement is still being hardened. Ankurah is beta software, and the open soundness audit covers check/apply races, remote-commit atomicity, and passive replication paths. Treat the current JWT agent as a reference implementation rather than an audited security boundary; follow ankurah/ankurah#336.

• The coarse collection read gate currently accepts a write privilege. can_access_collection returns true when a role has either the configured read or write privilege, and fetch/query use that gate. Row scopes still apply, but do not treat a nominally write-only role as a confidentiality boundary. This belongs to the hardening work in #336.

• Token expiry has a built-in grace. Verification uses the JWT library’s default 15-minute clock-skew tolerance and does not tighten it: a token expired less than 15 minutes ago still verifies. Budget your token TTLs with that in mind.

• iss and aud are not validated. Signature and (grace-adjusted) expiry are checked; issuer and audience claims are ignored. Do not rely on audience separation between services sharing a key.

• RS256 only, PEM only. No JWKS endpoint support, no kid-based key selection; rotation means replacing the single active key (clients auto-adopt the new public key through the synced JwtPolicy entity).

• Transport handshake is unauthenticated. The WebSocket connection itself carries no credential; the token rides on each request inside the protocol, and that is where enforcement happens. Use wss:// for transport privacy.

• Scope filters use string claims. Non-string custom claims are rejected (fail-closed); there is no array/in-style claim matching yet.

• Token issuance is out of scope. The extension verifies tokens; your login flow (an IdP, or your own endpoint calling SigningKeys::sign) is up to you.

Failure modes at a glance

Writing your own agent

PolicyAgent is a normal trait: implement the hooks above over your own ContextData type if JWTs are not your model. PermissiveAgent (allow everything) and JwtAgent (claims-driven RBAC plus row scopes) are the two shipped reference points, and the API reference links to the full trait docs.

Deployment & Operations

A basic Ankurah deployment is small: one durable server process holding the system of record, any number of ephemeral clients (browsers, native apps) connecting over WebSockets, and a storage engine under the server. This page covers standing that up for real – storage choices, the one-time system bootstrap, ports and TLS, upgrades, and backups.

The durable server

The minimal durable-server shape, from the example workspace:

let storage_dir = dirs::home_dir().unwrap().join(".ankurah");
let storage = SledStorageEngine::with_path(storage_dir)?;
let node = Node::new_durable(Arc::new(storage), PermissiveAgent::new());
node.system.wait_loaded().await;
if node.system.root().is_none() {
    node.system.create().await?;
}

let mut server = WebsocketServer::new(node);
server.run("0.0.0.0:9797").await?;

Three decisions are hiding in those lines:

Durability. Node::new_durable marks this node as a system of record: it keeps full event history and answers other nodes’ fetches. Clients use Node::new (ephemeral) – they hold a synchronized working set and lean on durable peers for history.

The system bootstrap. wait_loaded() first lets the durable node inspect its store. node.system.create() is called only when that store has no system root; it initializes a brand-new system and errors if a root already exists. Every restart reuses the stored root. Ephemeral clients call node.system.wait_system_ready().await after connecting to wait until they have joined the server’s system; SystemManager::create() separately rejects non-durable nodes. A mismatched root identifies a different system, so do not replace or recreate it for stores you mean to keep.

Current 0.9 caveat: these readiness methods use a check followed by a notification wait, which has a rare lost-wakeup window; a catalog-load error is logged but does not release wait_loaded(). In a supervised deployment, put an external bound on startup and treat an unresolved readiness wait as a failure to investigate in the logs. The implementation fix is tracked in #347.

The policy agent. PermissiveAgent::new() performs no authentication and no authorization – it is the development baseline. Before exposing a server to anyone you do not fully trust, wire a real agent: see Authentication & Policy.

Choosing a storage engine

All engines expose the same StorageEngine and StorageCollection APIs and aim for equivalent query semantics. Their planning, platform constraints, feature maturity, and operational trade-offs differ. Details: Storage Engine Layer.

For Postgres, the URI is standard tokio-postgres form. Tables are created on demand – one state table per collection (named for the collection) plus {collection}_event – and property columns are added as models introduce them, so no schema migration step is needed for new fields.

Ports, TLS, and the browser

WebsocketServer::run("0.0.0.0:9797") binds plain TCP. The example server uses port 9797 and the React template defaults to 9898; pick your own and keep client URLs in sync. Before an internet-facing deployment:

• Terminate TLS in front of the server (reverse proxy or load balancer) and point clients at wss://your-host. Native clients can also bring their own TLS configuration on the connection builder.
• The WebSocket handshake itself is unauthenticated by design – requests are authenticated individually inside the protocol (see Authentication & Policy) – so wss:// provides transport confidentiality, integrity, and server identity, not application access control.
• Browser clients are WASM builds. React uses a wasm-pack --target web bindings crate; Leptos compiles the Rust application directly through Trunk. The templates wire each path into their development runners.

A Rust process can also be a client – useful for workers and services that participate in sync rather than owning it:

let node = Node::new(Arc::new(storage), PermissiveAgent::new());
let _client = WebsocketClient::new(node.clone(), "ws://localhost:9797").await?;
node.system.wait_system_ready().await;

Hold on to the returned WebsocketClient handle for the life of the connection, as the example does.

Upgrade note: 0.8 to 0.9

• Pre-0.9 LWW buffers load lazily. 0.9 reads pre-0.9 LWW state buffers through a legacy fallback and rewrites each entity in the current format on its next save – no migration step. Details in Property Backends.
• Upgrade all nodes together. There is no protocol version negotiation yet, and 0.8.x binaries cannot read 0.9 state buffers, so mixed-version fleets are not supported across the 0.8 -> 0.9 boundary.

Backups

Back up the storage engine’s data as a consistent whole – for Sled that is the storage directory (~/.ankurah by default), for SQLite the database file, for Postgres your normal database backup. Entity state and event history live side by side in the same store, and both matter: state is the materialized view, but events are the authoritative history that concurrent merges depend on. Snapshot them together; a state-only backup would leave future merges unable to walk history.

Use the storage engine’s supported consistent-backup procedure rather than copying files while writes are active. For an embedded store, the conservative path is to stop the Ankurah process before copying it. For a database server, use its transactional backup tooling. Test restoration into an isolated environment before relying on the backup.

On local commits and other event-bearing application paths, events are stored before the materialized state that references them. A crash can therefore leave those paths with stored events and stale state. A pure StateSnapshot payload is different: it may persist a snapshot without storing its head events, which is intentional for ephemeral working sets. Recovery of a stale event-backed state requires redelivery/reapplication of the unapplied event or a response carrying the required history; an unrelated descendant alone is not a documented recovery guarantee. Engine-level file or transaction recovery remains the responsibility of the selected engine.

What to monitor

The observability story is young: there is no built-in metrics endpoint or health check yet. Practical minimum today: process supervision on the server, disk growth of the storage directory (event history is append-only), and your reverse proxy’s WebSocket connection counts. Log output uses standard tracing, so a tracing-subscriber with an env-filter gives you leveled logs.

How Ankurah Works

Ankurah is a distributed, event-sourced state framework. Applications talk to a local Node; nodes persist entity state and immutable events through pluggable storage engines and synchronize with each other over connectors. The diagram shows the current, well-exercised topology: many ephemeral clients connected to one durable server.

flowchart LR
    subgraph client["Ephemeral Node (browser)"]
        direction TB
        app["Applications<br/>React / Leptos UI"]
        cnode["Node<br/>Context · Reactor · Policy"]
        ceng["Storage Engine Layer"]
        cidb[("IndexedDB<br/>state + events")]
        app --> cnode
        cnode --> ceng
        ceng --> cidb
    end

    subgraph server["Durable Node (server)"]
        direction TB
        snode["Node<br/>Context · Reactor · Policy"]
        seng["Storage Engine Layer"]
        sstore[("sled / Postgres / SQLite<br/>state + events")]
        snode --> seng
        seng --> sstore
    end

    client <-->|"WebSocket connector<br/>(replication)"| server

The client (ephemeral) node holds a working set in the browser, while the durable server retains complete history. Durable and ephemeral describe history-retention roles rather than fixed client/server identities, but 0.9’s supported deployment shape still uses one durable node per system; the multi-durable plumbing is not yet a tested deployment contract. Broader multi-durable deployment support is coming soon; schema-registration propagation is one concrete blocker tracked in ankurah/ankurah#309. Both run the same Node core – a Context for scoped access, a Reactor driving live queries, and a policy agent – and both use the same state/event storage interfaces. A snapshot payload may populate an ephemeral working set without storing all of the head events behind that state. Ephemeral and durable nodes synchronize over a WebSocket connector.

The pieces

Node – the fundamental unit. A participating application embeds one or more Nodes; each holds entity state, evaluates queries, runs policy, and replicates with peers. Durable nodes (commonly servers) keep the full event history; ephemeral nodes (commonly browsers) hold a synchronized working set and fetch history on demand. Details: Node Architecture and Replication.

Events and the DAG – every change is an immutable event whose id hashes the entity ID, operation set, and parent clock. Per entity, events form a DAG like a git history; the entity’s current state points at the DAG’s head, and concurrent branches merge deterministically, field by field. Narrative: How Ankurah Handles Concurrency; contract: Conflict Resolution & Guarantees.

Live queries and reactivity – applications read through one-shot fetch() or subscribe with query(), which returns a LiveQuery that updates as matching changes are delivered to the node. A reactor per node matches applied changes against active subscriptions and drives the signal graph your UI observes. Usage: Querying Data and React Bindings.

Storage engines – every node writes through a pluggable storage engine that can persist entity state snapshots (the materialized current view) and retained immutable event history per collection. Durable nodes keep the history; an ephemeral snapshot can exist without all of its head events. The same two traits back Sled, Postgres, SQLite, and browser IndexedDB. Sled and IndexedDB use the common planner; the SQL engines use backend-specific query builders and predicate pushdown while preserving the shared public API. Details: Storage Engine Layer.

Connectors – released nodes synchronize over WebSocket connectors today (server and native/WASM clients), carrying subscriptions, deltas, and event batches. An Iroh peer-to-peer connector is under active development (#341) and is not part of the released connector set yet. End-to-end encryption remains a separate roadmap concern – see Design Goals.

The write path, end to end

A local commit validates a transaction, generates and persists its events, relays them to required peers, then applies and persists canonical local state before notifying the reactor. Remote payloads enter through NodeApplier, which stages and orders their events before using the same causal comparison and property-backend merge semantics. The orchestration paths differ, but both converge on the same event-DAG rules. The full trace lives in The Compare-Apply Cycle.

Consistency Model

Given eventual delivery of the same event histories, nodes converge with strong per-entity guarantees: event-bearing updates are ordered parents before children, validated state snapshots may install cumulative state directly, and per-field conflict resolution is deterministic. Nodes can keep writing locally while disconnected, but 0.9’s WebSocket connector does not yet maintain an outbox or automatically replay disconnected writes on reconnect; the histories must be delivered by application or future connector machinery. Reliable failed/disconnected-write delivery is tracked in ankurah/ankurah#195, with broader anti-entropy work in ankurah/ankurah#115. The precise promises and non-promises are in Conflict Resolution & Guarantees.

Learn More

• See the Design Goals for the philosophy behind these choices
• Check out Examples for practical code demonstrating these concepts
• Join the Discord to discuss architecture and implementation details

How Ankurah Handles Concurrency

Ankurah’s event-DAG engine can merge changes that different nodes produce without a central lock. Once both causally complete histories are delivered, every node that successfully integrates them computes the same result. The 0.9 connectors do not yet provide an offline-write outbox or automatic replay on reconnect, so delivery of changes committed while disconnected remains a separate application or connector concern. Reliable replay is coming soon; track ankurah/ankurah#195 and the broader reconciliation design in ankurah/ankurah#115.

This chapter builds the mental model, and Conflict Resolution & Guarantees states the resulting contract. What “merging” means for each field is a modeling choice you make per field – see Choosing a Merge Strategy. The machinery itself lives in the contributor Internals section: the comparison algorithm that classifies incoming changes, the anatomy of the engine, and the engine-facing property backends.

Events form a DAG

Every change in Ankurah is an immutable event. An event records:

• which entity it modifies,
• a set of operations per property backend (more on those later),
• a parent clock: the ids of the event(s) it was built on top of.

An event’s id is a content hash of all of that. Ids are therefore self-verifying and collision-free in practice. Together, the parent references form a directed acyclic graph, much like a git commit graph. History is usually a straight line:

A <- B <- C          (C's parent is B, B's parent is A)

When two nodes write without seeing each other’s work, the graph forks:

      <- C           (C's parent is B)
A <- B
      <- D           (D's parent is also B)

C and D are concurrent: neither is an ancestor of the other. Nothing went wrong here. Concurrency is a normal, expected state of the world, and the system’s job is to integrate both sides deterministically.

The head: where an entity currently is

Each entity tracks a head: the set of tip events that represent its current state. Usually the head is a single event, [C]. After the fork above is merged locally, the head is [C, D]: two concurrent tips, both part of the present. A later event E written with parent = [C, D] collapses the head back to [E] and permanently records that someone observed both branches.

A set of event ids used this way is called a clock. Clocks are kept sorted and deduplicated internally, and no member of a well-formed head is an ancestor of another member.

Classifying an incoming change

When an event or a state snapshot arrives, the receiving node asks one question: how does this relate, causally, to what I already have? The comparison algorithm answers with one of six relations:

An optimized common case is StrictDescends with the incoming event sitting exactly one step above the current head. That case is detected with a cheap shortcut and never walks the graph at all.

Merging diverged branches

For DivergedSince, the node computes the meet (the most recent common ancestors) and sweeps the accumulated graph forward in layers. Divergent frontier events in a layer are concurrent, but a layer can also include inert already_applied context from below the meet; layer membership alone therefore does not prove concurrency. The sweep processes in-graph parents before their children.

Each entity property belongs to a property backend, and each backend decides what concurrency means for its data:

• The LWW backend picks a single winner per property, preferring causally newer writes and breaking true ties deterministically.
• The Yrs backend wraps a text CRDT, so concurrent updates apply deterministically; concurrent inserts survive, although their visible interleaving may not match either author’s intent.

The layer machinery keeps graph retrieval and traversal out of property backends. They receive topological generations and query the accumulated graph through layer.compare and layer.dag_contains when their policy needs causal facts.

The promises

The concurrency system is built to keep a small set of promises:

1. Convergence. Nodes that successfully integrate the same causally complete event set reach identical state across permitted delivery schedules. Comparison verdicts and merge results depend only on the graph, never on wall-clock timing. A randomized property test checks the comparison verdicts against a brute-force reachability oracle across hundreds of generated DAGs and many more clock comparisons.
2. Parents first within a delivered batch. Receivers topologically sort each event-bearing batch instead of trusting sender order. This protects a batch from child-before-parent delivery, but it does not fill ancestry the payload omitted; automatic gap replay is planned in #268.
3. No foreign history. A clock that smuggles in an unrelated lineage (a second genesis) is never adopted wholesale. It either merges through the layer machinery or is rejected as Disjoint.
4. Event-bearing durability ordering. Local commits and incoming paths that carry events store those events before persisting state that references them. Pure StateSnapshot payloads are an intentional exception for ephemeral working sets: their head events may remain available only from a durable peer.

Where to go next

• Conflict Resolution & Guarantees states the contract all of this machinery upholds, and what is deliberately not promised.
• Choosing a Merge Strategy covers the per-field LWW-vs-Yrs decision from the modeling side.
• In the contributor Internals section, Causal Comparison: Frontiers and Meets explains how the six relations are actually computed, and Anatomy of the Engine maps the code.

Conflict Resolution & Guarantees

This page is the contract: what Ankurah promises your application when nodes write concurrently, and – just as important – what it does not. The mental model chapter explains the machinery narratively; this one states the outcomes.

What happens to an incoming change

Every non-creation event and state snapshot arriving at an entity is classified against that entity’s current head before it mutates state. Creation events use dedicated genesis guards instead:

For a strictly newer state snapshot, adoption installs the cumulative state in that snapshot. For an event-only update, the direct path applies the received event’s operations; it does not currently replay omitted ancestor operations.

Once the required histories are available locally, merging does not coordinate with a lock server or another online peer. Divergent branches coexist in the entity’s head until a later write reunifies them, and every rule below is deterministic so that reunification looks identical everywhere.

The promises

1. Convergence. Nodes that successfully integrate the same causally complete event set reach identical state across permitted delivery schedules. Resolution depends only on the event graph, never on wall clocks. Missing ancestors or invalid creation order still produce errors.

2. Per-field merge, per your model. Conflicts resolve at property granularity using the merge strategy each field declared. A concurrent write to title never clobbers an unrelated concurrent write to artist on the same entity.

3. Deterministic winners. For last-writer-wins fields, a causally newer write always beats the write it saw; truly concurrent writes resolve by comparing the events’ content-hash ids – an arbitrary but stable order every node computes independently. For collaborative-text (Yrs) fields, concurrent CRDT updates apply deterministically instead of selecting one whole-field winner; the resulting interleaving is not a guarantee that either author’s higher-level intent is preserved.

4. Parents first within a delivered batch. Receivers re-sort every incoming event-bearing batch parents-first; sender order is not trusted. This does not synthesize missing payload history.

5. No wholesale adoption of foreign history. An update that includes an unrelated lineage (a second genesis, or a graft joining one) is never fast-forwarded wholesale – it either routes through merge machinery or is rejected.

6. Event-bearing durability ordering. Local commits and incoming paths that carry events store them before persisting state that references them. Pure StateSnapshot payloads intentionally allow an ephemeral node to persist state whose head events remain available only from a durable peer.

7. Delivered offline branches use the same merge rules. A change produced while disconnected forms a normal branch and merges normally once both histories are delivered. The 0.9 WebSocket connectors do not yet queue and replay disconnected writes automatically on reconnect. That work is coming soon; follow #195 and the broader anti-entropy design in #115.

What is not promised

• No global ordering of unrelated writes. Two writes to different entities have no defined order across nodes; ordering guarantees are per-entity, through each entity’s event DAG.
• Last-writer-wins is causal, not chronological. “Newer” means causally newer – the writer had seen the other value. Between truly concurrent writes, the winner is the stable tiebreak, not whichever had the later wall-clock time. Do not encode business rules in “who wrote last.”
• Winning values are not approvals. Resolution decides consistency, not intent. If a field needs human conflict handling, model it so both values survive (separate fields, a Yrs field, or an explicit review workflow).
• No automatic offline synchronization yet. Local writes can be created while disconnected, but applications cannot assume the 0.9 connectors will discover and upload them after reconnect without an explicit delivery path.
• No automatic event-gap replay yet. A strictly descending EventOnly update can advance the head after applying only that event’s operations; it does not replay ancestor operations omitted from the payload. Deliver a causally complete event batch or cumulative state snapshot. A planned ingest pipeline with gap replay is tracked in #268.

Where these promises come from

Current-main backend conformance tests compare independent backend instances under alternate valid layer schedules, and a randomized property test checks comparison verdicts against a brute-force reachability oracle. The suite does not yet include an end-to-end test that replays opposite event orders through two independent Node instances. The mechanics live in the contributor section: Causal Comparison computes the classification table above, The Compare-Apply Cycle traces one merge end to end, and LWW Merge Resolution derives the determinism argument formally.

Glossary

This glossary defines key terms and concepts used throughout Ankurah.

Core Concepts

Model

A struct that describes the fields and projected Rust types for entities in a collection. Deriving Model generates a read-only View and a transaction-bound mutable handle.

#[derive(Model, Debug, Serialize, Deserialize)]
pub struct Album {
    #[active_type(YrsString)]
    pub name: String,
    pub artist: String,
    pub year: i32,
}

This generates AlbumView and AlbumMut alongside the user-defined Album create input.

Collection

A named group of entities described by the same Model. It is similar to a table in a traditional database, although its physical representation depends on the storage engine. PostgreSQL and SQLite use per-collection state and event tables.

Entity

A discrete identity in a collection, similar to a row in a database. The entity stores dynamic property state; a Model provides a typed projection over that state. Its EntityId is an independently generated ULID, not a derivative of its creation event.

View

A struct that represents the read-only view of an entity which is typed by the Model. Views provide type-safe access to entity properties without allowing mutations.

let view: AlbumView = ctx.get(album_id).await?;
println!("Album: {} by {} ({})", view.name()?, view.artist()?, view.year()?);

Mutable

A generated handle such as AlbumMut that exposes an entity’s active field types while a transaction is open. Obtain it by editing a View; mutations become durable only when the transaction commits.

let trx = ctx.begin();
let album = view.edit(&trx)?;
album.name().replace("Parade - Music from the Motion Picture")?;
album.year().set(&1987)?;
trx.commit().await?;

Event

A committed, immutable change for one entity. Each event contains:

• The collection and EntityId
• Per-backend operation diffs
• A parent clock containing the immediate precursor event IDs

Its EventId is a SHA-256 content hash of the entity ID, operation set, and parent clock. Timestamps and node IDs are not implicit event metadata.

Infrastructure

Node

A participant in the Ankurah network. Nodes can be servers, clients, or peers. Each node has:

• A storage backend
• A policy agent (for permissions)
• Connection handlers
• A reactor for subscriptions

Storage Engine

A means of storing and retrieving state and events. The current repository ships four implementations:

• Sled: Embedded native key-value storage
• SQLite: Embedded relational storage
• Postgres: Client/server relational storage
• IndexedDB: Browser storage for WASM clients

Ankurah is beta software, so do not assume identical feature maturity across all four engines without checking the backend-specific tests.

Storage Collection

A collection of entities in a storage engine. The physical representation of a Collection in the storage layer.

Operations

Transaction

A local unit of work that groups creates and edits. An entity is snapshotted when it first enters the transaction; commit() validates the writes and generates an event for each changed entity. This API should not be read as a claim of globally serializable, cross-node ACID transactions.

let trx = ctx.begin();

let album = trx.create(&Album {
    name: "Parade".into(),
    artist: "Prince".into(),
    year: 1986,
}).await?;

let album_id = album.id();
trx.commit().await?;

Subscription

A live query that receives updates when matching entities change. Subscriptions use SQL-like predicates for filtering.

use ankurah::signals::Subscribe;
let live: LiveQuery<AlbumView> = ctx.query("year > 2000")?;
live.wait_initialized().await;
let _guard = live.subscribe(|changes| {
    println!("Received changes: {changes}");
});

Event Sourcing Terms

ULID

Universally Unique Lexicographically Sortable Identifier. Used for EntityId (and several internal request/query identifiers) to enable:

• Distributed ID generation without coordination
• Lexicographic ordering by the embedded creation timestamp (not causal order)
• Compact representation (128-bit)

EventId is different: it is a 256-bit content hash.

DAG (Directed Acyclic Graph)

The structure formed by events and their precursor relationships. The DAG enables:

• Per-entity causal comparison and merge ordering
• Conflict detection
• Efficient synchronization

Lineage

The causal event history that led to an entity’s current state, including branches and merges. Used for:

• Audit trails
• Conflict resolution
• Replication

Head

The most recent event or concurrent events in an entity’s DAG. Nodes track the head as a set because concurrent branches may produce more than one tip.

Reactivity

Signal

An observable value that notifies subscribers when it changes. Ankurah’s signal system is inspired by SolidJS and enables reactive UIs.

Reactor

A per-node component that matches applied entity changes against registered query predicates and updates live-query result sets. The signal/observer layer, rather than the reactor itself, tracks component dependencies and derived values.

Live Query

A query that automatically updates when the underlying data changes. Implemented using subscriptions and the reactor.

Policy & Security

Policy Agent

A component that authenticates peer requests and controls access to reads and writes. Agents decide:

• Can a node read an entity?
• Can a node modify an entity?
• Can a node subscribe to a collection?

Context

A wrapper around a Node that includes user/session information (ContextData). Operations performed through a Context are subject to policy checks.

let context = node.context(user_data)?;
let trx = context.begin();
let album = trx.create(&Album { /* ... */ }).await?;
trx.commit().await?;

Additional Resources

• See What is Ankurah? for a high-level overview
• Check Architecture for how these concepts fit together
• Visit Examples for practical usage

Design Goals

Ankurah is designed with specific goals in mind to create a powerful, flexible, and developer-friendly state management framework.

Schema / UX

Model-Based Schema Definition

• Define schema using “Model” structs, which define the data types for a collection of entities
• An ActiveRecord style interface with type-specific methods for each value
• The same Model definitions generate native Rust types and TypeScript classes for WASM browser clients
• Transaction, fetch/query, and selection helpers keep the typed API consistent across those targets

Example:

#[derive(Model, Debug, Serialize, Deserialize)]
pub struct Album {
    #[active_type(YrsString)]
    pub name: String,
    pub artist: String,
    pub year: i32,
}

Use it inside a transaction:

let trx = ctx.begin();

let album = trx.create(&Album {
    name: "Parade".into(),
    artist: "Prince".into(),
    year: 1986,
}).await?;

let album_id = album.id();
trx.commit().await?;

Observability

Signal-Style Reactive Pattern

• Utilize a “signal” style pattern to allow for observability of changes to entities, collections, and values
• Derived signals can filter and transform those changes
• React is the primary, maintained frontend path through @ankurah/react-hooks
• A reactive_graph bridge and template exist for Leptos, but that integration is still young and experimental

Benefits:

• Automatic UI updates when data changes
• Declarative data dependencies
• Efficient change propagation

Storage and State Management

Multiple Backing Stores

The repository currently implements:

• Sled for embedded native key-value storage
• SQLite for embedded relational storage
• Postgres for client/server relational storage
• IndexedDB for browser/WASM storage

Ankurah is beta software. These are real implementations, not a promise that every backend has identical production maturity or feature coverage.

Event Sourcing

Changes are committed as immutable events. An event carries per-backend operation diffs and a clock containing its parent event IDs:

• Content identity: An EventId is the SHA-256 hash of the entity ID, operation set, and parent clock
• Immutable History: Events are immutable (with future considerations for compaction and data-retention requirements)
• Current State: The present state of an entity is maintained per node, including the head of its event DAG
• Version Tracking: Nodes causally compare known entity versions and track the current local head

Entity and Event IDs

• EntityId is an independently generated ULID, allowing any node to create an identity without coordination
• EventId is a 256-bit content hash, so it commits to the change and its causal parents
• Entity identity is therefore stable across later events and is not derived from a genesis event

Future Considerations:

• Compact long event histories while preserving convergence and retention guarantees
• Harden peer validation and resource limits for adversarial environments
• Define explicit audit/export and deletion semantics above the immutable event layer

Development Milestones

Major Milestone 1 - Current beta foundation

Core functionality for early adopters:

• ✅ Working event-sourced model and query layer with off-the-shelf storage engines
• ✅ Rust structs for data modeling
• ✅ Signals pattern for notifications
• ✅ WASM Bindings for client-side use
• ✅ WebSocket server and client
• ✅ Maintained React bindings and React/WASM template
• ✅ Basic included data-types: CRDT text (yrs crate) and primitive types
• ✅ Sled, SQLite, Postgres, and browser IndexedDB storage implementations
• ✅ Basic single-field queries
• ✅ Multi-field queries
• ✅ Robust recursive query AST for declarative queries

Major Milestone 2 - Stuff we need, but can live without for a bit

Enhanced functionality and possible future work (not release commitments):

• Additional distributed storage engines
• Reliable replay of writes committed while disconnected (coming soon; #195)
• Broader stale-cache reconciliation research (#115)
• Unified ingest and missing-ancestor replay (#268)
• Validated multi-durable deployments (coming soon; schema-registration propagation blocker: #309)
• Iroh peer-to-peer connector (coming soon; #341)
• Graph Functionality
• User-definable data types
• Advanced indexing strategies
• Query optimization
• Performance profiling tools

Major Milestone 3 - Maybe someday…

Future aspirations:

• Portable cryptographic identities: User identities that work across nodes
• E2EE (End-to-End Encryption): Privacy-preserving data synchronization
• Hypergraph functionality: More complex relationship modeling
• CRDT compaction: Efficient storage of long operation histories
• Byzantine fault tolerance: Security against malicious nodes

Design Philosophy

Ankurah prioritizes:

1. Developer Experience: Easy to learn, hard to misuse
2. Type Safety: Compile-time guarantees where possible
3. Flexibility: Support various storage backends and use cases
4. Performance: Efficient synchronization and querying
5. Scalability: From embedded devices to large distributed systems

Inspirations

Ankurah draws inspiration from:

• Event Sourcing: CQRS, Event Store
• Reactive Programming: SolidJS signals, MobX
• ActiveRecord: Ruby on Rails, Ecto (Elixir)
• Distributed Systems: CRDTs, operational transformation
• Modern Databases: Postgres, TiKV, FaunaDB

Contributing

We welcome contributions! Join the discussion:

• GitHub Repository
• Discord Server

Help shape the future of Ankurah by:

• Reporting bugs and suggesting features
• Improving documentation
• Contributing code
• Building example applications
• Sharing your use cases

API Reference (docs.rs)

The book explains concepts and tasks; the complete published API surface lives on docs.rs. Start with the main crate and enable its derive feature, then add the storage engine and connector your node uses:

[dependencies]
ankurah = { version = "0.9", features = ["derive"] }
ankurah-storage-sled = "0.9"
ankurah-websocket-server = "0.9"

• ankurah – the main API crate. Re-exports Node, contexts, transactions, queries, and core types. With features = ["derive"], it also re-exports #[derive(Model)] and the derive/query macros used throughout this book.

The workspace splits into focused crates underneath; you will mostly meet them in error messages and Cargo trees rather than importing them directly:

Storage engines and connectors are separate crates so applications only compile what they use:

Ankurah is beta software and its extension crates may publish on a different cadence. Check compatible versions in the selected crate’s dependency list or use one of the maintained templates as a known-good set.

For how these layers fit together, see the overview; for the storage traits behind the engine crates, see the Storage Engine Layer chapter.

Examples

This page contains practical code examples demonstrating key Ankurah features.

Defining a Model

#[derive(Model, Debug, Serialize, Deserialize)]
pub struct Album {
    #[active_type(YrsString)]
    pub name: String,
    pub artist: String,
    pub year: i32,
}

This automatically generates:

• AlbumView (read-only)
• AlbumMut (transactional updates)

See Defining Models for full documentation.

Server Setup

let storage = SledStorageEngine::with_path(storage_dir)?;
let node = Node::new_durable(Arc::new(storage), PermissiveAgent::new());
node.system.wait_loaded().await;
if node.system.root().is_none() {
    node.system.create().await?;
}

let mut server = WebsocketServer::new(node);
println!("Running server...");
server.run("127.0.0.1:9797").await?;

Rust Client

let storage = SledStorageEngine::new_test()?;
let node = Node::new(Arc::new(storage), PermissiveAgent::new());
let _client = WebsocketClient::new(node.clone(), "ws://localhost:9797").await?;
node.system.wait_system_ready().await;

// Create album
let ctx = node.context(ankurah::policy::DEFAULT_CONTEXT)?;
let trx = ctx.begin();
trx.create(&Album { name: "Parade".into(), artist: "Prince".into(), year: 1986 }).await?;
trx.commit().await?;

React Component

interface Props {
  albums: AlbumLiveQuery;
}
/* Bind a React observer to the component. */
const AlbumList = signalObserver(({ albums }: Props) => {
  return (
    <ul>
      {/* Reading items registers this render as a live-query observer. */}
      {albums.items.map((album) => (
        <li key={album.id.to_base64()}>{album.name}</li>
      ))}
    </ul>
  );
});

See React Bindings for full documentation.

Live Query

// Using selection! macro with ctx.query()
let q: LiveQuery<AlbumView> = ctx.query(selection!("year > 1985"))?;

See Querying Data for full documentation.

Entity References

Create relationships between entities with Ref<T>:

// Create an artist
let trx = ctx.begin();
let artist = trx.create(&Artist { name: "Radiohead".into() }).await?;
let artist_id = artist.id();
trx.commit().await?;

// Create a song that references the artist
let trx = ctx.begin();
trx.create(&Song {
    title: "Paranoid Android".into(),
    artist: Ref::new(artist_id),
}).await?;
trx.commit().await?;

Traverse references to fetch related entities:

// Fetch the song and traverse to get the artist
let songs: Vec<SongView> = ctx.fetch("title = 'Paranoid Android'").await?;
let song = songs.first().unwrap();

// Get the referenced artist entity
let artist: ArtistView = song.artist()?.get(&ctx).await?;
println!("Artist: {}", artist.name()?);

JSON Queries

Create entities with dynamic JSON fields:

let trx = ctx.begin();

trx.create(&Track {
    name: "Test Track".into(),
    metadata: Json::new(serde_json::json!({
        "genre": "rock",
        "bpm": 120,
        "tags": ["guitar", "drums"]
    })),
}).await?;

trx.commit().await?;

Query by nested JSON paths:

// Query by nested JSON path
let tracks: Vec<TrackView> = ctx.fetch("metadata.genre = 'rock'").await?;

Numeric comparisons work too:

// Numeric comparison on JSON field
let fast_tracks: Vec<TrackView> = ctx.fetch("metadata.bpm > 100").await?;

Storage Backends

Postgres

let storage = Postgres::open(uri).await?;

SQLite

let storage = SqliteStorageEngine::open("ankurah.sqlite").await?;

Sled (Embedded)

let storage = SledStorageEngine::new()?;

IndexedDB (WASM)

let storage = IndexedDBStorageEngine::open("myapp").await?;

Next Steps

• Check out the Quick Start guide for step-by-step setup
• Review the Glossary to understand key terms
• Study the Architecture to see how it all fits together
• Join the Discord to discuss your use case!

Anatomy of the Engine

This chapter maps the concurrency system onto the codebase: which layer owns which decision, what each seam guarantees, and why the factorization looks the way it does. Read it when you need to change the engine, review a change to it, or figure out where a behavior actually lives.

The layers

From the wire up:

proto           wire-truth data types: EventId, Clock, Event, Attested, fragments
event_dag       pure graph logic: comparison, layers, ordering (no I/O policy)
retrieval       how events and state are found: traits + getter implementations
entity          one entity's state machine: apply_event / apply_state, TOCTOU
node_applier    wire payloads -> entity applications, batch semantics
node / context  peering, subscriptions, local commit, policy enforcement

Each layer only speaks to the one below through a deliberately narrow interface. The rest of this chapter walks them bottom-up.

proto: the data model is the contract

ankurah-proto defines what travels between nodes, and it enforces its own invariants at construction time rather than trusting callers:

• EventId is a SHA-256 content hash of the event’s entity, operations, and parent clock. Identity is therefore self-verifying, and parent cycles are structurally impossible, which downstream code (Kahn’s sort, ancestry walks) leans on.
• Clock is a sorted, deduplicated vector of event ids. Membership tests binary-search, so sortedness is load-bearing; every construction path, including deserialization of peer-supplied clocks, normalizes rather than trusting input order. Nothing downstream ever needs to wonder whether a clock is well-formed.
• Attested<T> pairs a payload with attestations. Policy decides what attestations mean; proto only carries them.
• Fragments (EventFragment, StateFragment) are events and states with the entity id and collection factored out, for wire compactness.

event_dag: pure logic, injected I/O

core/src/event_dag/ contains the algorithms from the previous chapter, factored so that none of them know where events come from:

Two design decisions shape this module:

The accumulator outlives the comparison. While the BFS walks, the accumulator records every parent edge it sees and caches fetched events. The comparison verdict is returned together with the accumulator as a ComparisonResult, and a diverged result converts into the layer iterator via into_layers(). The merge therefore replays exactly the graph the comparison saw, with no second discovery pass and no window for the two phases to disagree about the DAG’s shape. The accumulator is also what survives the internal budget-escalation retry.

Event access is a capability, not an ambient ability. Everything here is generic over a GetEvents implementation. The comparison can fetch and read; it cannot stage, commit, or write. That is enforced by the next layer.

retrieval: three traits instead of one

core/src/retrieval.rs splits event access into capabilities:

• GetEvents: read an event; ask whether it is durably stored (event_stored), and whether a negative answer is authoritative (storage_is_definitive). This is all the comparison ever gets.
• GetState: read entity state snapshots. Separate because state has different caching and is never needed mid-traversal.
• SuspenseEvents: extends GetEvents with stage_event and commit_event. Only the outermost applier holds this.

The split turns the staging discipline into a compile-time property: code that merely compares provably cannot commit. Commit capability remains confined to a small set of outer application paths (context, node remote transaction commit, system, and node_applier) rather than one universal call site.

Two getter implementations matter:

• LocalEventGetter: staging map, then local storage. Used for local commits on all node types.
• CachedEventGetter: staging, then local storage, then a remote peer fetch. Used when applying remote updates on ephemeral nodes, where history may live elsewhere.

The staging map is the mechanism behind a core invariant: an incoming event is staged (discoverable by BFS, held in memory) before anyone compares against it, and committed to durable storage only after it has been accepted and applied. get_event sees staging plus storage; event_stored sees storage only. That distinction is exactly what lets guards distinguish “I can see this event” from “this event is part of durable history”.

entity: one entity’s state machine

core/src/entity.rs owns the head and the backends for a single entity, and exposes two application paths:

apply_event integrates one event. Guards first: creation events on non-empty heads are re-deliveries or attacks (the event_stored fast path plus storage_is_definitive decide which); non-creation events on empty heads are rejected outright. Then the retry loop: compare the event’s clock against the head, act on the verdict (the table from the overview), and if the head moved between comparison and mutation, re-read and retry. That last part is the TOCTOU discipline: comparison is async and lock-free, so the head is re-checked under the write lock and the loop retries on interference, bounded at five attempts.

On the StrictDescends event-only path, that direct application covers the received event’s operations, not any omitted ancestor operations discovered during comparison. Callers currently need a causally complete event batch or a cumulative state snapshot; planned gap replay is tracked in #268.

apply_state integrates a whole state snapshot, using the same comparison but coarser actions: adopt (StrictDescends), skip (Equal / older), or report that a proper merge needs events (diverged).

The entity layer also owns the WeakEntitySet: the registry of resident entities. Application paths materialize entities speculatively when an update references one that is not resident; if the update then fails its guards, the speculative empty-head resident is evicted rather than left looking like a real entity with no state.

node_applier: wire payloads to entity applications

core/src/node_applier.rs translates each wire payload shape into the correct application sequence, and owns batch semantics:

• EventOnly: stage all events, topologically sort the batch, then apply and commit parents-first.
• StateAndEvent: stage and sort the events, try apply_state, and fall back to parents-first event application if the state cannot be adopted (which handles both divergence and stale-state cases).
• StateSnapshot: state only, for fetch responses.
• EventBridge: stage everything, topologically sort, then apply parents-first. Wire order is untrusted by design; the sender also sorts, but the receiver’s sort is the guarantee.
• StateAndRelation: declared in the payload enum but currently rejected as unimplemented.

Deliveries are failure-contained at the outer entity/update-item boundary: one bad item is recorded and included in the aggregate error while unrelated items continue, and the reactor is notified for the items that succeeded. A bad event can still stop the remaining events inside its own item.

context: local commits

Local writes go through a transaction and commit in phases (core/src/context.rs): generate events from changed entities, run every policy check (staging each event and applying it to a fork so the checker sees before and after states), and only then persist. A denial partway through a multi-entity transaction leaves nothing durable. After persistence, heads advance, required peers confirm, and state snapshots are written.

The invariants at the seams

These are the event-bearing path invariants and explicit API contracts; pure state snapshots are called out where they differ:

1. Stage before an event-applied head. On paths carrying events, an event must be BFS-discoverable before application advances a head to it. A validated StateSnapshot may install cumulative state whose head events live only on a durable peer.
2. Commit before event-backed state. Event-bearing paths make an event durable before persisting the state produced from it. A crash may leave an event without its updated materialized state. Pure snapshots are again the intentional exception.
3. get_event is staging plus storage; event_stored is storage only.
4. Comparison cannot write. GetEvents in, verdict out.
5. Budget handling is internal to compare. Callers see one call and at most one final BudgetExceeded; only the accumulator survives the retry.
6. Only a diverged verdict yields layers. into_layers returns Some for DivergedSince and None for every other relation; callers branch on that result.
7. Receivers sort batches. No application path trusts sender ordering.

Why chains are advisory

StrictDescends carries a chain of visited events. Its contents are deduplicated, but its order is traversal order, not topological, so nothing uses it as an application order today. Batch application derives ordering from parent edges instead (Kahn’s), which is self-verifying. Treat chain as a hint for future optimizations, not a contract.

Event DAG Subsystem

What is the Event DAG?

Every entity in Ankurah can be mutated concurrently by multiple nodes. Each mutation produces an event whose parent pointer records what the node believed the entity’s latest state to be. Over time these events form a directed acyclic graph – a history that branches when nodes mutate in parallel and reconverges when branches are merged.

        A          linear history
        |
        B          B's parent is A
       / \
      C   D        C and D were created concurrently (both parent B)
       \ /
        E          E merges the two branches

The event DAG subsystem answers two questions:

1. How do two points in history relate? Given the entity’s current head and an incoming event, are they linearly ordered, or have they diverged?
2. If they diverged, what happened since they last agreed? It produces a topologically sorted sequence of event layers that property backends use to merge concurrent operations.

The implementation lives in core/src/event_dag/ and is consumed primarily by Entity::apply_event.

Key Concepts

Event – A single mutation to an entity. Carries a parent clock (the entity’s head when the event was created) and a set of backend-specific operations. An event with an empty parent clock is a creation event (genesis).

Clock – An ordered set of event IDs representing a frontier in the DAG. An entity’s head is a clock: usually a single event ID (linear history), but multiple IDs when concurrent branches coexist.

Meet point – The greatest common ancestor(s) of two diverged clocks. The meet is itself a frontier: no member is an ancestor of another. Everything between the meet and the branch tips needs to be merged.

       A
       |
       B  <-- meet point
      / \
     C   D
      \ /
       E

Event layers – After finding the meet, the history is replayed in topological generations for merge. See Event Layers for the precise definition and its guarantees, and LWW merge for how property backends consume the layers.

Comparing Two Clocks

The comparison algorithm (core/src/event_dag/comparison.rs) determines the causal relationship between a subject clock and a comparison clock. There are six possible outcomes:

StrictDescends carries a chain of the events the subject traversal visited. Its contents are duplicate-free, but its order is traversal order, not guaranteed topological – consumers must not use it as an application order. Batch application paths sort independently (see batch ordering).

Quick-check: the linear-extension fast path

The overwhelmingly common case is that an incoming event extends the current head by exactly one step. Before launching a full traversal, the algorithm checks whether every member of the comparison clock appears directly in the subject event’s parent set. If so, it returns StrictDescends immediately without fetching the comparison events at all.

This matters for ephemeral nodes, where the comparison head events may not exist in local storage – the quick-check never needs them.

BFS traversal

When the quick-check does not apply, the algorithm walks backward through the DAG from both clocks simultaneously:

1. Initialize two frontiers, one from the subject clock and one from the comparison clock.
2. At each step, fetch every event on both frontiers, record its DAG structure, and extend the frontiers backward through parent pointers.
3. When an event is reached from both directions, it becomes a meet candidate.
4. After each step, check whether a conclusion can be drawn:
   • The subject traversal has seen every comparison head and its boundary is clean (see below): StrictDescends.
   • All subject heads seen by the comparison traversal: StrictAscends.
   • Both frontiers empty: compute the minimal meet (candidates with no common descendants in the traversal), return DivergedSince or Disjoint.

Why coverage alone is not enough

Seeing every comparison head proves the subject’s cover contains the comparison clock – but not that the subject introduces nothing else. A subject clock can smuggle in a foreign lineage two ways: as an extra head ([B, X] versus [A], where B descends A but X has an independent genesis), or through a single graft event whose parent clock joins a legitimate ancestor with an unrelated root. Fast-forwarding either shape would adopt the foreign line wholesale.

The guard is the clean-boundary check: before declaring StrictDescends, every genesis root the subject traversal has discovered, and every id still on its unexplored frontier, must lie within the comparison’s ancestry (computed over the accumulated DAG, including parent ids referenced by explored events but not yet fetched). Honest shapes pass exactly as before – a deep linear extension’s unexplored remainder sits below the comparison surface, and a sibling tip bottoms out at a shared ancestor. Smuggled shapes fail the check, the traversal runs to exhaustion, and the foreign line goes through the diverged merge or reject paths instead of being adopted.

Unfetchable events on both frontiers

On ephemeral nodes, historical events may live only on the durable peer. If an event ID appears on both frontiers but cannot be fetched, it is a common ancestor beyond local storage. The algorithm processes it with empty parents, correctly terminating traversal at that point. An unfetchable event on only one frontier is a genuine error – the DAG is incomplete.

Budget escalation

The initial budget (default 1000 events) caps each BFS attempt. If exhausted, the algorithm internally retries with 4x budget (up to initial * 4). The traversal itself restarts from the original clocks – only the accumulator (recorded DAG structure and LRU event cache) survives the retry, so re-walked steps avoid storage round-trips but still spend budget. The internal retry keeps the public API simple – callers do not need to manage retry logic.

Event Layers

When comparison returns DivergedSince, the merge machinery walks the accumulated DAG forward and hands property backends the history as a sequence of event layers. A layer is one generation of a topological sort:

A layer is the set of events whose parents have all been emitted by earlier layers – where the meet itself, and any parent outside the accumulated DAG, counts as already emitted.

Equivalently, an event’s layer number is its longest-path distance from the meet: an event whose parents sit at depths 1 and 3 lands at depth 4, waiting for its deepest parent.

        M              meet (never emitted)
       / \
     X1   Y1           layer 1: {X1, Y1}
      |
     X2                layer 2: {X2}
      |
     X3                layer 3: {X3}

Layers group by causal depth, not by branch. With head [X3] and incoming Y1, X1 and Y1 share layer 1 even though they sit on opposite sides of the divergence, and the local tip X3 flows through in layer 3. Two guarantees make per-layer merging sound:

1. Parents precede children. Every causal predecessor of an event above the meet appears in a strictly earlier layer, so applying layers in order respects causality.

2. Divergent-region layers are antichains. Above the meet, no event in a layer is an ancestor of another event in the same layer, even transitively. This holds because DivergedSince is only produced by an exhaustive traversal (see Invariants): every event between the meet and the tips is in the accumulated DAG, so a causal path cannot hide behind an unfetched event. Within a layer, concurrency is genuine.

Orthogonal to layering, each layer partitions its events by whether the local replica has already incorporated them: events in the current head’s ancestry are already-applied, the rest are to-apply. In the diagram, X1..X3 are already-applied and Y1 is to-apply. Already-applied events participate in merge resolution as context – they can defeat an incoming write – but only to-apply winners mutate state.

Order within a layer is deliberately meaningless: backends receive the layer as a set. Yrs ignores layer boundaries entirely (CRDT operations commute), while LWW treats each layer as an election round whose incumbent is re-seeded from stored state.

Two scope notes. First, the layer sweep covers the entire accumulated DAG, and an exhaustive traversal accumulates the common history below the meet as well – so when the meet sits above genesis, early layers also carry below-meet events (genesis itself surfaces in layer 1). These are always already-applied (they are ancestors of the current head by definition), so they never mutate state; they are inert electoral context. Second, the antichain guarantee is scoped to the divergent region: a below-meet straggler can share a layer with an event that descends it through the meet. This is harmless for the same reason – causal comparisons consult the full accumulated DAG, never layer membership.

The Staging Pattern

An incoming event must be discoverable by BFS before the entity’s head is updated to reference it. Otherwise, a concurrent traversal starting from the new head would encounter an unfetchable event. The staging pattern solves this with a four-phase lifecycle:

  stage_event -----> apply_event -----> commit_event -----> set_state
  (in-memory map)   (head update)      (durable storage)   (durable state)

1. Stage – Place the event in an in-memory map so that get_event can find it during BFS.
2. Apply – Compare the event against the entity head and update in-memory state on success.
3. Commit – Write the event to permanent storage and remove it from the staging map.
4. Persist state – Write the entity’s head clock and backend buffers to disk.

The ordering invariant

Stage before head update (in memory); commit before state persist (to disk).

The first half ensures BFS reachability. The second half ensures crash safety:

• Crash after commit but before state persist: recovery loads the old entity state; the event is in storage but unreferenced by the head, so the next apply_event integrates it normally via BFS.
• Crash before commit: neither event nor updated state are persisted – a clean rollback.

Trait separation enforces the protocol

The retrieval layer splits event access into distinct traits so that apply_event (which takes GetEvents) cannot accidentally stage or commit events. Only the outer caller holds SuspenseEvents, which adds stage_event and commit_event. This makes the staging protocol a compile-time guarantee rather than a convention.

The distinction between get_event (union of staging + storage) and event_stored (permanent storage only) enables the creation-event guard: on durable nodes where storage is definitive, event_stored() == false for a creation event proves it has never been seen, enabling a cheap rejection without BFS.

Invariants

BFS correctness

1. Both-frontiers = common ancestor. If an event appears on both frontiers, treat it as a meet point with empty parents. Do not require fetching it. This is essential for ephemeral nodes where the meet event is not in local storage.

2. Single-frontier unfetchable events are hard errors. The old “dead end” continue behavior left unfetchable IDs permanently on the frontier, causing infinite loops.

3. Never compute layers from an incomplete traversal. BudgetExceeded means the accumulated DAG is partial; layer computation would produce incorrect merge results.

4. Meet filter: common_child_count == 0. This ensures head tips always appear in the meet set (they have no descendants in the comparison, so they always pass the filter). Deep ancestors that also pass produce harmless no-op removals from the head.

Staging protocol

5. Stage before head update; commit before state persist. Violating the first causes BFS failures; violating the second breaks crash recovery.

6. get_event is the union view; event_stored is permanent-only. Mixing these up breaks creation-uniqueness semantics.

7. Creation guards execute before the retry loop in apply_event. They are properties of the event, not of the current head, and must not be re-evaluated on retry.

Design Decisions

Why compare uses the event’s own ID, not its parent clock

The original algorithm started BFS from the incoming event’s parent clock, then applied a transform to infer the event’s causal relationship. This assumed the event was genuinely novel. Re-delivery of a historical event violated the assumption, causing head corruption:

Chain: A -> B -> C   head=[C]
Re-deliver B:
  Old algorithm: BFS from parent(B)=[A] vs head=[C]
    -> StrictAscends, transform -> DivergedSince(meet=[A])
    -> Inserts B into head -> invalid head [C, B]

The fix: stage the event first, then pass the event’s own ID as the subject clock. BFS discovers the staged event, traverses its parents naturally, and produces the correct StrictAscends for re-delivery with no special-case logic.

Why the DuplicateCreation guard was refined

An early guard used event_stored() unconditionally to detect duplicate creation events. On durable nodes this works, but on ephemeral nodes that receive entities via StateSnapshot, the creation event is never individually committed – only the entity state referencing it is stored. When the creation event later arrives via subscription, event_stored() returns false, misclassifying a legitimate re-delivery as a different genesis.

The unconditional guard was replaced with conditional logic. Where storage is definitive (storage_is_definitive()), event_stored() == true short-circuits a re-delivery as a no-op and a missing event is rejected as a different genesis. Ephemeral nodes fall through to compare(), which handles both cases correctly: re-delivery yields StrictAscends; different genesis yields Disjoint.

Why the retrieval traits were split

The original monolithic Retrieve trait combined event access, state access, and staging. This made it impossible to express “read-only event access” at the type level. Since apply_event must not stage or commit events (the caller manages that), the split into GetEvents, GetState, and SuspenseEvents turns the staging protocol into a compile-time constraint. See the Event Retrieval and Staging documentation for details.

Causal Comparison: Frontiers and Meets

Conceptually, everything in the previous chapter reduces to one operation:

compare(event_getter, subject, comparison, budget)
    -> Result<ComparisonResult<E>, RetrievalError>

ComparisonResult contains the causal relation plus the accumulated graph needed by a later merge. Given two clocks, the operation answers: does subject strictly descend comparison, strictly precede it, equal it, diverge from it since some meet, or share nothing with it at all? This chapter walks through how that answer is computed, and why several of the details are load-bearing.

Throughout, the cover of a clock means the clock’s events plus all of their ancestors: everything that clock “knows about”.

Early exits

Two identical clocks are Equal. This includes two empty clocks: an entity with no history compared against another empty history is the same (non-)place. A one-sided empty clock shares nothing with a non-empty one and reports DivergedSince with an empty meet.

The quick check

Almost every comparison in a live system is a new event landing exactly one step above the current head. For that shape a full graph walk is wasted work, and on ephemeral nodes it can be worse than wasted: the head events themselves may not exist in local storage, only the state that references them.

So compare first fetches just the subject’s events and checks a shortcut:

Every subject event has a nonempty parent set contained in the comparison clock, and every comparison head appears as some subject event’s parent.

If that holds, the subject is the comparison advanced by one step: StrictDescends, no traversal, and crucially no fetch of the comparison events.

Both halves of the condition matter. Coverage alone (every comparison head is somebody’s parent) is not enough, because a subject event that contributes nothing to the parent union gets silently vouched for by its siblings. Consider subject = [B, X] versus comparison = [A], where B’s parent is A and X is an independent genesis root. The parent union is {A}, which covers the comparison, yet X carries an entire foreign lineage. Adopting [B, X] as a fast-forward would import unrelated history without a merge. The per-event guard sends any such shape to the full traversal, which classifies it correctly as diverged.

The traversal: two frontiers walking backward

When the shortcut does not apply, the algorithm walks the DAG backward from both clocks simultaneously. Each side owns a frontier: the set of event ids it is about to process. Each step:

1. Fetch every event on either frontier.
2. For each event: remove it from the frontier(s) it sits on, record which side(s) have now seen it, and extend that side’s frontier with the event’s parents.
3. Check whether a verdict can be declared.

An event reached by both sides is a common node: shared history. The first common nodes discovered are candidates for the meet.

Visit each node once per side

A node reachable by two paths of different lengths will be re-added to a frontier after it was already processed. Left unchecked, that causes three distinct problems: bookkeeping that counts the same head twice (and can declare a false verdict from it), traversal budget spent per path rather than per event (a chain of diamonds has linearly many events but exponentially many paths), and result chains polluted with duplicates.

The algorithm therefore keeps a per-side processed set. Expansion consults it twice: a node is only expanded the first time each side processes it, and parents already processed on a side are never re-added to that side’s frontier. Budget becomes proportional to events fetched, and every bookkeeping structure can trust that it sees each (node, side) pair once.

Declaring StrictDescends: coverage plus a clean boundary

The subject strictly descends the comparison when the subject’s traversal has seen every comparison head. That is cover containment: everything the comparison knows is inside what the subject knows.

But StrictDescends drives adoption: the receiver fast-forwards its head to the subject clock wholesale. Cover containment alone is not a safe basis for that, because a subject can smuggle a foreign lineage in two ways. The [B, X] clock above juxtaposes an extra head; a single graft event does it more subtly, with one event whose parent clock joins a legitimate ancestor line with an independent genesis root. Both shapes cover the comparison via their honest component while importing unrelated history.

So the completion requires, additionally, that the subject’s exploration boundary lies entirely within the comparison’s ancestry: every genesis root the subject traversal has discovered, and every id still on its frontier (the undiscovered remainder), must be part of the comparison’s lineage. A foreign root fails the first test the moment it is discovered; a deep foreign line keeps failing the second until its root surfaces.

Honest shapes fire exactly when they used to. A deep linear extension completes with its unexplored remainder sitting below the comparison surface, which is inside the comparison’s ancestry. And a subject head may be a sibling of a comparison head rather than its descendant: if the local head is [J] and an incoming state carries head [F, J], where F and J are concurrent children of the same creation event, F’s walk bottoms out at their shared ancestor, inside the comparison’s ancestry. That state genuinely contains everything the local node has, and adopting it is correct.

A subject with a dirty boundary never fast-forwards. The traversal simply continues to exhaustion and reports the honest diverged verdict, which routes the foreign line through the merge machinery instead of adopting it blind.

StrictAscends (the mirror direction) deliberately stays cover-only: it drives skip this incoming event, and for skipping, knowing that the local head already covers the incoming clock is sufficient.

Finding the meet

For diverged clocks, the verdict must name the meet: the most recent common ancestors, which become the floor for the layer merge.

Every common node is a meet candidate. When a candidate’s child is also common, the candidate is an ancestor of shared history, not its edge; the final meet is the set of common nodes with no common children. In other words, the maximal antichain of the common region.

One more check guards the verdict’s honesty. Each comparison head must be accounted for: some common node must be reachable backward from it. This is asked once, at exhaustion, by direct reachability over everything the traversal recorded, which is complete for both walks by then. A head that reaches no common node means part of the comparison clock shares nothing with the subject, and the verdict degrades to a diverged result with an empty meet rather than inventing a partial one. An earlier design propagated per-head origin markers through the walk and retired heads incrementally; that scheme is provably insufficient once re-expansion is deduplicated (a marker arriving at an already-expanded node is never carried forward), which is why the implementation asks the reachability question directly instead of maintaining running state.

Disjoint detection

If the frontiers exhaust with no common node at all, the traversal has seen both histories down to their genesis events. Different genesis events prove different lineages: Disjoint. This is the backstop for lineage integrity: an independently generated EntityId has one accepted creation event anchoring its history, and no amount of clock juggling can replace that genesis wholesale.

Budget

The current budget is a soft breadth-step guard, not a hard cap on storage fetches. A traversal step drains the entire current frontier before the next budget check, so a wide frontier may process more entries than the nominal remaining budget; the preliminary quick-check fetches are outside this budget as well. compare retries internally once with four times the budget before reporting BudgetExceeded. Only the accumulator (the recorded DAG structure and an LRU cache of fetched events) survives the retry; traversal state restarts from the original clocks. Re-walked entries can hit the cache but still count toward the soft guard.

Visits are deduplicated per side, so charged traversal work remains linear in the explored (event, side) region. Deep histories consume many sequential steps; wide histories can overshoot within one step. Common verdicts usually fire before full exhaustion.

Unfetchable events

On an ephemeral node, old events may exist only on a durable peer. If an id sits on both frontiers but cannot be fetched, it is provably a common ancestor; the algorithm processes it with empty parents, cleanly terminating both walks at that point. An unfetchable id on only one frontier is a real error: the local graph is missing something it needs.

The second frontier algorithm: layers

Comparison walks backward to classify. Merging then walks forward to apply. From the meet, the layer iterator repeatedly emits the set of events whose parents have all been emitted already: a generalized topological sort where each emission is a layer of mutually concurrent events.

Each layer is split into already_applied (events the local head already incorporates, provided as context) and to_apply (new work). Property backends receive whole layers and resolve concurrency within them; the property backends chapter covers exactly how.

The third: ordering event batches

When a peer needs to catch a node up across many events (an EventBridge), the batch travels as a set. Discovery order on the sender interleaves uneven branches, and receivers must not trust sender ordering anyway. The same rule applies to every multi-event wire shape. Applying a child before its staged parent would classify the child as a fast-forward (its ancestry resolves through staging), jump the head past the parent, and then discard the parent’s operations as “already history” while the event itself sits committed in storage: a silent, durable loss.

So both sides sort. The receiver runs Kahn’s algorithm over the batch using parent edges within it (parents outside the batch are the bridge floor, already known), applying parents before children unconditionally. Content addressing makes a cycle impossible in honest input, so cycle detection simply rejects the batch as malformed.

The specification, executable

The comparison semantics described here are pinned by a randomized property test: hundreds of generated DAGs, random antichain clock pairs, and a brute-force reachability oracle that recomputes every verdict from first principles (cover containment, root/boundary containment for safe adoption, and meet as the maximal common antichain). Divergence between the state machine and the oracle fails the build. When you need the precise semantics of an edge case, the oracle in core/src/event_dag/tests.rs is the most honest place to read them.

The Compare-Apply Cycle

Every mutation that reaches an entity – whether from a local commit or a remote peer – travels the same pipeline: the event is made discoverable, compared against the entity’s current head, and integrated according to the verdict. The reference chapters each describe one stage of that pipeline in isolation. This chapter traces one concrete divergence through the entire cycle, end to end, with every intermediate value shown.

arrive -> stage -> compare -> integrate -> commit -> persist -> notify

What integrate means is decided by the comparison verdict – six are possible (see the verdict table). The interesting one is DivergedSince, true concurrency, so that is the path we walk. The others appear at the end.

The Scenario

Two replicas, A and B, share an entity with a short linear history:

    G                genesis: creates the entity, title = "draft"
    |
    P                title = "v1"          (both replicas at head [P])

Working offline, each replica commits one event against head [P]:

• Replica A commits X: title = "alpha". A’s head is now [X].
• Replica B commits Y: title = "beta". B’s head is now [Y].

    G
    |
    P                <-- the meet (neither replica knows this yet)
   / \
  X   Y              concurrent: neither saw the other

EventIds are content hashes; in this telling id(X) > id(Y). Now Y arrives at replica A over a subscription. Everything below is replica A’s view.

Stage 0: Arrival

The node applier receives a batch of events. Three things happen before any comparison:

1. Topological sort. The batch is sorted parents-first (event_dag/ordering.rs). Neither discovery order nor the sender is trusted: applying a child before its staged parent would fast-forward the head past the parent, silently dropping the parent’s operations as StrictAscends.
2. Staging. Each event is placed in the in-memory staging map, making it visible to get_event – the union view of staging plus storage. The BFS can now discover Y.
3. Guards. Y is not a creation event and A’s head is non-empty, so the creation guards pass through to the retry loop.

Only after apply_event succeeds is the event committed to permanent storage, then entity state persisted – stage before head update, commit before state persist.

Stage 1: Compare

apply_event calls compare with the subject clock [Y] (just the incoming event’s id) against the comparison clock [X] (the current head).

Quick-check first. The linear-extension fast path asks: do Y’s parents all sit inside the comparison clock? Y’s parent is [P] and the comparison set is {X} – no. The fast path does not apply; fall through to the BFS.

Backward BFS, step by step. Both sides walk toward their ancestors simultaneously:

Neither side ever saw the other side’s head (X is not an ancestor of Y or vice versa), so neither StrictDescends nor StrictAscends fired. Both frontiers are empty: exhaustion. The verdict is computed from the shared bookkeeping:

• Meet candidates: {P, G} (reached from both directions).
• Meet filter – candidates with no common children: P qualifies (its children X and Y are one-sided); G does not (its child P is common).
• Meet = [P].

Result: DivergedSince { meet: [P], subject_chain: [Y], other_chain: [X] }. Four events were fetched; the budget spent 4 of 1000. Note that exhaustion walked below the meet all the way to G – the accumulated DAG now holds the parent pointers of all four events, and that completeness is what the next stage relies on.

Stage 2: Layers

The accumulator is consumed into an EventLayers iterator, seeded with the meet [P] marked as already emitted and the current head [X] for partitioning. One generation of the forward topological sweep:

Layer 1: { G, X, Y }
         G -> already-applied   (below the meet; inert context)
         X -> already-applied   (in head ancestry)
         Y -> to-apply          (the novel branch)

Everything lands in a single layer: G because its (empty) parent set is trivially satisfied, X and Y because their parent P is the meet. This tiny example exhibits both scope notes from Event Layers: a below-meet event rides along as inert already-applied context, and the divergent region’s members (X, Y) are genuinely concurrent – depth groups by causal distance, not by which replica produced the event. P itself is never emitted: the meet’s own writes are already baked into stored state on both replicas and take no part in what follows.

Stage 3: Apply

apply_event collects all layers (the async fetches happen before locking), takes the entity write lock, and re-checks that the head is still [X]. It then feeds the layer to every backend.

LWW runs a per-property election (resolution rules). For title:

The winner is X – an already-applied event. Winners only mutate state when they come from to-apply events, so nothing is written and no field signal fires. This is the already-applied side doing its real job: providing the context that stops a losing remote write from clobbering the local winner.

Yrs (contrast) would simply apply Y’s operations and ignore the rest – CRDT commutativity needs no election.

Head update. The meet ids are removed from the head (P is not in it) and Y is inserted:

head: [X]  ->  [X, Y]

The punchline of this trace: the merge changed no property value, yet it still changed the entity – the head grew a second tip, recording that Y’s lineage is now integrated. apply_event returns true and the entity-level signal fires (field-level signals stay quiet). The caller then commits Y to permanent storage and persists the state with its two-tip head.

Stage 4: Convergence on the Other Side

Replica B runs the mirror image when X arrives: same BFS, same meet [P], same single layer – but partitioned as already = {G, Y}, to-apply = {X}. The election reaches the same winner X, and this time the winner is to-apply: B writes title = ("alpha", X), the field signal fires, and B’s head becomes [Y] -> [X, Y].

Both replicas now hold identical state and identical heads, having applied the branches in opposite orders. The tiebreak (max over EventIds) is commutative and associative, which is the heart of the determinism argument.

Stage 5: Healing the Fork

The two-tip head is not a degenerate state, but it does not persist forever. The next mutation on either replica – say A commits Z, title = "final" – uses the full head as its parent clock: parent(Z) = [X, Y].

When Z arrives at B (head [X, Y]), the quick-check fires: Z’s parents are non-empty and all lie inside the comparison clock, and together they cover it. Verdict: StrictDescends, no BFS, no fetches of X or Y at all. Z’s operations are applied directly and the head collapses:

head: [X, Y]  ->  [Z]

The fork is closed. One ordinary event, created with no knowledge that it was “merging” anything, reunifies the lineage simply by naming both tips as its parents.

The Paths Not Taken

For completeness, the same machinery handles every other arrival shape:

• Re-delivery of Y (head [X, Y]): the BFS’s comparison side reaches Y – a subject head – in one step. Verdict StrictAscends, no-op. Idempotency falls out of the comparison; there is no separate dedup table.
• Equal – the incoming clock is the head: no-op before any traversal.
• Disjoint – traversal bottoms out at two different genesis events: rejected. See the creation guards for the cheap durable-node shortcut.
• BudgetExceeded – the traversal ran out of budget even after escalation: surfaced as an error carrying both frontiers.
• Unfetchable meet – on ephemeral nodes the traversal may hit an event that exists only on the durable peer; if it sits on both frontiers it is treated as the meet and the cycle proceeds normally.

Where to Go Deeper

Each stage of this trace has a reference chapter:

Event Retrieval and Staging

The retrieval layer is the bridge between the event DAG and persistent storage. Its job is to answer two questions during event processing: “give me this event’s data” and “has this event already been durably stored?” These sound similar but have deliberately different semantics, and the layer’s design follows from keeping them separate. The durable backends it sits on top of – the traits, the engine matrix, and the predicate fetch path – are documented separately in the Storage Engine Layer.

Why Three Traits Instead of One

The system originally had a monolithic Retrieve trait that bundled event reading, state reading, and staging/committing into a single interface. The problem was that apply_event – the core integration function – only needs read access to events, but receiving a Retrieve reference gave it the ability to stage or commit events. That is the caller’s responsibility, not apply_event’s.

The split enforces staging discipline at the type level:

• GetEvents – Read-only event access. This is what the BFS comparison algorithm and apply_event accept. It exposes get_event (the union of staging and permanent storage), event_stored (permanent storage only), and storage_is_definitive (whether a negative event_stored result is authoritative).

• GetState – Entity state snapshot retrieval. Separated because state has different caching characteristics and is not needed by BFS.

• SuspenseEvents – Extends GetEvents with stage_event and commit_event. Only the outermost caller (e.g., NodeApplier) holds this; it is deliberately not passed into apply_event.

Staging vs Permanent Storage

The retrieval layer maintains two tiers of event storage:

The staging map is an in-memory HashMap behind an Arc<RwLock>. When an event is staged, it becomes visible to get_event and therefore to BFS traversal, but event_stored will still return false. Staging is what makes the comparison algorithm work on incoming events: the event is staged first, then compare uses the event’s own ID as the subject clock, and BFS discovers the event body through the staging map.

Permanent storage is the durable backend. After apply_event succeeds, the caller commits the event – writing it to permanent storage and removing it from the staging map. From this point, event_stored returns true.

The distinction between get_event (union view) and event_stored (permanent-only) matters for the creation-event guard: on durable nodes where storage_is_definitive() is true, a false from event_stored for a creation event proves it has never been seen, enabling a cheap rejection without BFS. On ephemeral nodes, this shortcut is unsafe because entities can arrive via StateSnapshot without their individual events being stored.

Durable vs Ephemeral Lookup Strategies

The two concrete implementations of the retrieval traits differ in how far they search for a missing event:

LocalEventGetter – Used for local commits on all node types. Checks the staging map, then local storage. If the event is not found locally, that is a hard error. Sets storage_is_definitive based on the durable flag passed at construction.

CachedEventGetter – Used by ephemeral nodes. Adds a third lookup tier: if the event is not in staging or local storage, it requests the event from a random durable peer, caches the response locally, and returns it. This transparent remote fallback is what allows BFS to succeed on ephemeral nodes that lack historical events. storage_is_definitive is always false.

LocalStateGetter – Shared by both paths. Wraps storage to retrieve entity state snapshots, translating “not found” into Ok(None).

The Event Lifecycle: Stage, Apply, Commit, Persist

A typical flow in NodeApplier::apply_update for EventOnly content:

for each event:
    validate(event)
    event_getter.stage_event(event)            // (1) stage

entity = get_or_create(...)

for each attested_event:
    if entity.apply_event(event_getter, &event) // (2) compare + apply in memory
        event_getter.commit_event(&attested_event) // (3) commit to disk

if any event applied:
    save_state(entity)                         // (4) persist entity state

For StateAndEvent content, the flow first tries apply_state. If the state is strictly newer, all staged events are committed and state is saved. If the state diverges, it falls back to per-event apply_event followed by commit_event, then saves state.

The same parent-first rule applies to every multi-event wire shape – EventOnly, StateAndEvent, and EventBridge (in apply_delta) alike: the receiver stages the whole batch upfront, topologically sorts it by in-batch parent edges (event_dag/ordering.rs), and only then applies, commits, and finally saves entity state. The producer also sorts what it sends, but wire order is untrusted: applying a child before its staged parent would jump the head past the parent and silently drop the parent’s operations.

Crash Safety

The ordering invariant – commit events before persisting state – provides clean recovery in every failure scenario:

• Crash after commit, before state save: The event is in storage but the entity state still references the old head. On recovery, the next delivery of the same event (or any descendant) integrates it via BFS. No data loss.

• Crash after stage, before commit: The staging map is in-memory only and lost on crash. Neither the event nor the updated state are persisted. Clean rollback.

• Crash after state save: Fully consistent. Normal operation.

• Concurrent apply_event on the same entity: The try_mutate helper checks that the head has not changed since comparison. If it has, the caller retries (up to 5 attempts), preventing TOCTOU races.

Storage Engine Layer

The storage engine layer is the bottom of the Ankurah stack – the only part that actually touches a disk, a browser database, or a SQL server. Everything above it (the event retrieval and staging layer, entity persistence, the replication protocol) is written against two traits and never names a concrete backend. Swapping sled for Postgres changes where bytes land; it does not change a line of the node, the applier, or the compare-apply cycle.

        Node / Context / Reactor
                  |
     Event retrieval & staging  (retrieval.md)
        LocalEventGetter / CachedEventGetter
                  |
     StorageCollectionWrapper  (Arc<dyn StorageCollection>)
                  |
   +--------+-----+------+-----------------+
   |        |            |                 |
  sled   postgres   indexeddb-wasm      sqlite      <- engines
   |        |            |                 |
  disk    SQL server   browser IDB      single file

The traits live in core/src/storage.rs; the engines are separate crates under storage/. A large body of shared query machinery lives in storage/common, so engines implement placement and I/O, not query planning from scratch.

The Two Traits

StorageEngine is the collection factory and lifecycle handle. It is small:

• collection(&CollectionId) -> Arc<dyn StorageCollection> – open or create the storage for one collection. This is where an engine does per-collection setup: sled opens a tree, Postgres and SQLite run CREATE TABLE IF NOT EXISTS for the state and event tables under a DDL lock, IndexedDB hands back a bucket bound to the shared object stores.
• delete_all_collections() – drop everything (used by tests and resets).
• an associated Value type – the engine’s native value representation (Vec<u8> for sled, PGValue for Postgres, SqliteValue, JsValue).

StorageCollection is the real contract – the interface the rest of the system depends on. Grouped by responsibility rather than method-by-method:

StorageCollectionWrapper (core/src/storage.rs) is a thin Deref newtype around Arc<dyn StorageCollection> that the retrieval layer holds; it adds no behavior, only a stable handle to clone.

One Collection, Two Stores

Every collection persists two kinds of data, and the split mirrors the event-sourcing model directly:

• Entity state snapshots – the materialized current view of each entity. A snapshot carries the serialized property state plus the entity’s head clock (the set of event ids that produced it). This is what get_state and fetch_states return, and what queries run against.
• Events – the immutable history. Each event names its parent clock and the operations it applied. This is what add_event/get_events persist and the event DAG walks during comparison.

State is derived; events are authoritative. State exists so that reads and predicate queries do not have to replay history, and the head clock on each snapshot is the join point back to the DAG.

Physically, engines keep the two stores in separate namespaces, but the naming is engine-specific – there is no single scheme enforced across the layer:

• Postgres / SQLite give each collection two tables: the state table is named for the collection itself (bare {collection}) and the event table is {collection}_event.
• Sled / IndexedDB use two shared stores for all collections – an entities tree/object-store and an events tree/object-store – plus a per-collection collection_{id} tree (sled) or index object stores (IndexedDB) holding the materialized, indexable projection used for queries.

core/src/storage.rs defines helper functions state_name() and event_name() that produce {collection}_state / {collection}_event, but note that the current engines do not route through them (Postgres/SQLite build their own names, and the KV engines use fixed shared-store names); treat those helpers as a naming convention rather than the authoritative source of table names.

Write ordering. The state snapshot and its events are written by different calls, and the order matters. The invariant enforced one layer up is: commit the events to permanent storage before persisting the entity state that references them (see event retrieval and staging and entity lifecycle -> persistence ordering). A crash between the two leaves events on disk with a stale head, which the next delivery heals via BFS – never state pointing at events that were never stored. The storage layer itself does not span the two writes in a transaction; the ordering discipline lives in the caller.

The Predicate Fetch Path

fetch_states is the one method that takes a query rather than an id. Its argument is an ankql::ast::Selection – a predicate plus optional ORDER BY and LIMIT. Turning that into an engine operation is where most of the per-engine complexity would be, so it is deliberately factored into storage/common and shared:

• Planner (storage/common/src/planner.rs) takes the Selection and the primary-key field name and enumerates candidate Plans: Index { .. } scans, a TableScan fallback, or EmptyScan when the predicate can never match. It splits the predicate into conjuncts (predicate.rs), separates equalities from inequalities, chooses index key parts, and computes how much of the ORDER BY a scan direction can satisfy versus what must be sorted in memory (the OrderByComponents presort/spill split in types.rs). It is capability-aware via PlannerConfig: supports_desc_indexes is true for engines with real descending indexes and false for IndexedDB, which only has ascending index parts.
• bounds.rs normalizes per-column index bounds into a single canonical lexicographic range that each KV engine lowers to its own cursor range.
• filtering.rs / sorting.rs provide streaming combinators (filter_predicate, sort_by, top_k, limit) over any stream of Filterable items. Residual predicates the index could not satisfy are evaluated here in Rust via core’s evaluate_predicate, so no engine reimplements predicate evaluation.

What differs between engines is how much of the query is pushed down to the backend versus evaluated with the shared Rust combinators:

• Postgres and SQLite push the predicate into SQL. Each has a split_predicate_for_* pass (sql_builder.rs) that partitions the predicate into a sql_predicate (translated into a WHERE clause) and a remaining_predicate that SQL cannot express. The pushable part becomes a real query; the residual is post-filtered in Rust. When a residual exists, LIMIT is dropped from the SQL and re-applied after post-filtering, so the database is never allowed to truncate rows that the residual might have kept. Debug builds record the spilled predicate so tests can assert full pushdown.
• Sled and IndexedDB run the Planner and then scan. They pick the first viable plan, open an index cursor (or a full collection scan for a TableScan), materialize candidate rows, and run the residual predicate, sort, and limit through the shared filtering/sorting streams. Sled reads ids from an index tree and does a secondary lookup into the shared entities tree to hydrate each state; IndexedDB drives IDB index cursors.

Engine Matrix

Only claims verified against the code in each crate.

Two notes the code makes explicit. SQLite positions itself in its crate docs as sitting “between Sled (pure KV) and Postgres (full SQL server)” and requires SQLite 3.45+ for JSONB; its implementation is a full pushdown engine, not a stub. On the KV side, dump_entity_events is a full scan of the shared events tree in sled (flagged as acceptable only because it is test-facing), whereas SQLite and IndexedDB index events by entity_id for that lookup.

Index Maintenance

For the KV engines, secondary indexes are a real subsystem, not a free byproduct of the store. Sled’s IndexManager (storage/sled/src/index.rs) maintains per-collection index trees: set_state calls update_indexes_for_entity with the old and new materialized property tuples so index entries stay consistent with state, and fetch_states calls assure_index_exists to create an index on demand when a plan needs one. The key encoding these indexes share – ordered, typed, multi-column keys – lives in core/src/indexing (KeySpec, IndexKeyPart, and the tuple encoder), which is also what the storage/common planner reasons about when it decides which index a query can use. IndexedDB follows the same shape using native IDB indexes. The SQL engines lean on the database’s own indexing and add columns lazily as properties appear.

How Event Retrieval Layers On Top

The event retrieval and staging layer is the immediate consumer of StorageCollection. Its concrete getters call straight into these methods:

• LocalEventGetter (durable path) checks an in-memory staging map, then falls back to collection.get_events(..); commit_event calls add_event. Its storage_is_definitive() returns the durable flag it was constructed with.
• CachedEventGetter (ephemeral path) adds a third tier: staging, then get_events, then a request to a durable peer whose response it writes back via add_event. Its storage_is_definitive() is always false.

That storage_is_definitive bit is exactly the durable/ephemeral distinction surfacing at the storage boundary. On a durable node the local store holds every event, so event_stored() == false is conclusive and enables cheap guards without a DAG walk; on an ephemeral node the same store is a cache, a miss means “not here yet,” and the getter must go to a peer. The two lookup strategies are covered in retrieval -> durable vs ephemeral lookup.

Writing a New Engine

The contract is small and the shared code carries the hard parts:

1. Implement StorageEngine – a collection() factory that opens/creates the state and event stores for a collection, and delete_all_collections().
2. Implement StorageCollection – the state (set_state/get_state/ fetch_states), and event (add_event/get_events/dump_entity_events) methods. set_states/get_states come for free from the defaults.
3. Use storage/common – run the Planner (with the right PlannerConfig for your index capabilities), lower bounds to your native ranges, and evaluate residual predicates/sorts/limits through the filtering/sorting streams. Do not hand-roll predicate evaluation. If the backend speaks SQL, follow the Postgres/SQLite pattern: split the predicate, push what you can, post-filter the rest, and defer LIMIT when a residual exists.
4. Preserve the write-ordering contract – add_event and set_state may be separate writes, but the state you persist must reference a head whose events are already durable (the caller guarantees the ordering; your engine just must not reorder or lose the event write).

Conformance is checked by exercising each engine through the same model/query API rather than a single generic trait-test macro. The crate-independent behavioral tests live in the workspace tests/ crate (which runs against sled), and each SQL/IDB engine carries a parallel suite under its own tests/ directory (for example storage/postgres/tests, storage/sqlite/tests, storage/indexeddb-wasm/tests) covering predicate checks, ordering, JSON semantics, and undefined-column handling. A new engine is expected to pass the equivalent behavioral tests for its platform. See the Testing Strategy chapter for how these fit together.

Entity Lifecycle

Mental Model

An entity in ankurah is a replicated, convergent data object. Its lifecycle follows four phases:

flowchart LR
    creation["Creation"]
    mutation["Local Mutation<br/>(transaction)"]
    commit["Commit<br/>(validate, relay, persist)"]
    persisted["Persisted<br/>(stored state)"]
    remote["Remote Events<br/>(apply or merge)"]
    others["other nodes"]

    creation --> mutation --> commit --> persisted
    persisted --> remote
    others --> remote

At every stage, two things determine what happens next:

• The head clock – a set of event IDs recording which events have been integrated into the entity’s current state.
• The event DAG – which determines whether an incoming update extends, duplicates, or conflicts with that state.

Head and backend state are bundled under a single lock so they are always updated atomically.

Creation

An entity comes into existence through Transaction::create(). This does two things:

1. Mints a primary entity with an empty head and empty backends, registered in a node-wide weak set (which guarantees at most one live instance per entity ID).
2. Forks a transactional snapshot by cloning every backend and the current head. The snapshot is Transacted – it holds a back-pointer to its primary but is the only copy the user mutates.

This snapshot isolation means the primary entity stays read-only until commit. User mutations (setting properties) go through the snapshot’s backends, which accumulate pending operations.

System root entities follow a different path: they are created outside a transaction, have their properties set directly, and produce a creation event that is immediately applied and persisted. This is the only code path where a creation event is applied to the same entity that generated it.

Local Transaction Commit

When a transaction commits, five phases execute in order:

1. Generate events. Each entity’s backends are asked for pending operations (via to_operations()). These become an Event whose parent is the snapshot’s current head. Entities with no pending operations are skipped. A validation check ensures creation events can only come from entities that were actually created through the transaction – preventing “phantom entities.”

2. Fork-based validation. For each entity/event pair, a second fork is created as a validation sandbox. The event is staged, applied to the sandbox, and the resulting before/after state is passed to the policy agent for attestation. Attested events are committed to storage.

3. Update heads. Heads on the transacted entities are updated to include the new event ID. This happens before relaying to peers – so if a peer echoes the event back, the local entity already recognizes it as already-integrated.

4. Relay to peers. Attested events are sent to durable peers. The commit waits for peer confirmation.

5. Persist state. The event is applied to the upstream primary entity (via apply_event), bringing it up to date. The entity’s state is serialized and persisted to storage. Change notifications are emitted to the reactor.

Remote Event Application

Remote events arrive via NodeApplier through two delivery mechanisms (see Node Architecture and Replication for the full protocol):

Subscription updates come in two forms:

• EventOnly – the common incremental case.
• StateAndEvent – used for initial subscription delivery and fetch responses. The system first tries the fast path: apply the state snapshot directly. If that succeeds, done. If the state diverges (concurrent edits exist), it falls back to the accompanying events. This two-phase approach ensures events are never silently dropped on divergence.

Delta application (fetch/query responses) similarly comes as either a StateSnapshot (applied directly) or an EventBridge (events connecting the requester’s known head to the responder’s).

For every multi-event payload – EventOnly, StateAndEvent, and EventBridge alike – the receiver validates and stages the whole batch, then topologically sorts it by in-batch parent edges (event_dag/ordering.rs) and applies parents before children. Sender order is not trusted: applying a child before its staged parent would fast-forward the head past the parent, whose operations would then be silently dropped as StrictAscends.

How Events Are Applied

apply_event is the central integration point, used by both local commit and remote delivery. It works in two stages: guard checks, then a retry loop.

Guard Ordering

Three guards execute before the main logic, handling edge cases around creation events and empty heads:

1. Creation event on a non-empty head. On durable nodes where storage is definitive, event_stored() == true identifies a re-delivery – no-op, while a not-yet-stored event proves different genesis – reject as Disjoint. On ephemeral nodes, fall through to BFS which distinguishes re-delivery from different genesis.

2. Creation event on an empty head. Acquire the write lock, re-check that the head is still empty (TOCTOU protection), apply operations, set the head.

3. Non-creation event on an empty head. The entity was never created properly. Reject with InvalidEvent rather than letting BFS produce a spurious DivergedSince(meet=[]).

The Retry Loop

After guards pass, apply_event enters a bounded retry loop (up to 5 attempts). Each attempt reads the current head, runs compare() against the event DAG, and acts on the causal relation:

Retries happen when the head moves between comparison and mutation (see TOCTOU protection below).

How State Snapshots Are Applied

apply_state handles full state snapshots rather than individual events. It follows the same compare-then-mutate pattern but cannot merge divergent state – merging requires the per-operation detail that only events carry (see LWW Merge Resolution).

When a new state arrives for an entity that may not exist locally yet, WeakEntitySet::with_state handles the lookup: check the in-memory weak set, then local storage, then create from the incoming state if neither has it.

TOCTOU Protection

Because DAG comparison is async (and lock-free), the head can move between comparison and mutation. The try_mutate helper serializes this:

#![allow(unused)]
fn main() {
fn try_mutate(&self, expected_head: &mut Clock, body: F) -> Result<bool, E> {
    let mut state = self.state.write().unwrap();
    if &state.head != expected_head {
        *expected_head = state.head.clone();
        return Ok(false);  // head moved -- caller should retry
    }
    body(&mut state)?;
    Ok(true)
}
}

If the head moved, the caller’s expected_head is updated in place and the retry loop re-runs comparison against the fresh value. Both apply_event and apply_state use this pattern. Retries are bounded to 5 attempts.

Head Clock Evolution

The head clock evolves through three patterns:

Linear extension – the common case. Head is [A], event B arrives with parent=[A], comparison yields StrictDescends, head becomes [B].

Divergence – two events B and C are created concurrently from A. After applying B (head=[B]), C arrives and comparison yields DivergedSince{meet=[A]}. After layer-based merge, head becomes [B, C] – a multi-element clock indicating concurrent tips.

Merge – event D arrives with parent=[B, C], matching the current head exactly. Head collapses back to [D].

Persistence Ordering

State persistence follows a strict ordering invariant: commit events to storage before persisting state (see The Staging Pattern and Crash Safety).

This gives clean crash recovery semantics:

• Crash after commit_event but before set_state: recovery loads the old state and the event is re-applied on next delivery.
• Crash before commit_event: neither event nor updated state is persisted – a clean rollback.

Key Invariants

1. Atomic head + backend updates. Both live under a single RwLock and are always updated together.

2. TOCTOU protection on every mutation path. Compare-then-mutate is serialized with bounded retries (5 attempts).

3. Creation event idempotency. Re-delivery is detected by the durable fast path or by BFS (StrictAscends). Neither corrupts state.

4. Transaction snapshot isolation. The primary entity is not modified until commit phase 5.

5. Staging before comparison; commit before persistence. Events must be staged (discoverable by BFS) before apply_event is called. Events must be committed to storage before entity state referencing them is persisted.

6. StateAndEvent divergence fallback. When apply_state does not apply the incoming state (divergence, or the state is older than what the receiver has), the applier falls back to event-by-event application. Events are never silently dropped on divergence.

Node Architecture and Replication Protocol

What Is a Node?

A node is the unit of participation in an Ankurah deployment. It owns a storage engine, holds live entities in memory, and communicates with peer nodes to keep entity state converged. Every node can create, mutate, and query entities – the difference between node types is how much history they keep and where they go when history is missing.

Durable vs Ephemeral Nodes

Think of durable nodes as the archival backbone. A durable node persists every event it accepts and every entity state snapshot. Because it has the complete event history for every entity it knows about, it never needs to ask anyone else for data. When it says “I have not seen this event,” that statement is authoritative.

Ephemeral nodes are lightweight participants – typically client-side processes. They also write to a local storage engine, but they often receive entity state without the underlying events (via a state snapshot). Their local storage is therefore incomplete: a missing event might simply mean “I was told the answer without being shown the work.” When an ephemeral node needs an event it does not have, it transparently fetches it from a durable peer and caches it locally.

This distinction surfaces in one critical API: storage_is_definitive(). On a durable node this returns true – a negative lookup is conclusive. On an ephemeral node it returns false, which forces the system to do a deeper DAG traversal instead of taking the shortcut. See the creation-event guard for the main place this matters.

For details on the event-getter implementations that back this behavior, see the Event Retrieval and Staging document.

The Replication Protocol

Data moves between nodes through two mechanisms: streaming updates (pushed) and request/response deltas (pulled). Both carry entity state and/or events, but they serve different moments in the lifecycle.

Streaming updates (UpdateContent)

Once an ephemeral node has an active subscription with a durable peer, the durable node pushes changes as they happen. An update item carries either the events alone (EventOnly, when the sender expects the receiver to already have the state) or the new entity state plus the events that produced it (StateAndEvent). EventOnly is a valid wire format handled by the receiver, though current senders always include state. On the receiving side the node validates the state, integrates the events via apply_event, and persists the result. If the incoming state diverges from what the receiver already has, the receiver falls back to event-by-event apply_event with BFS comparison.

Request/response deltas (DeltaContent)

When a subscription is first established – or when an ephemeral node runs a fetch – the durable node assembles a delta for each entity. The choice of delta depends on what the requester already knows:

• State snapshot – the requester has never seen the entity, or the gap is too complex to bridge. The full state is sent.
• Event bridge – the requester has a known (older) head and the durable node can trace a clean forward path through the DAG. Only the missing events are sent, avoiding a full state transfer.

The event bridge is especially valuable after brief disconnections: a handful of events is far cheaper than retransmitting the full state. The bridge is built by walking backward from the current head through parent pointers until every frontier member is in the requester’s known head, then reversing the collected events into causal (oldest-first) order.

Known limitation: The backward walk currently has no traversal budget. A stale or malicious known_head could trigger unbounded event collection. Chunked bridge framing, size limits, and resource governance are tracked in the phase-2 spec (specs/concurrency/phase-2.md).

Subscription Propagation

Subscriptions are how ephemeral nodes stay synchronized with durable peers. The flow has three phases: establishment, streaming, and recovery.

Establishment. When application code creates a live query on an ephemeral node, the node registers it as a pending subscription. When a durable peer is (or becomes) available, the node sends a subscribe request that includes a map of entities it already knows and their head clocks. The durable peer responds with deltas – state snapshots for unknown entities, event bridges for known ones – and begins monitoring its local reactor for future changes.

Streaming. After establishment, the durable node pushes updates whenever entities matching the subscribed query change. The ephemeral node applies these incrementally, as described above.

Recovery. If the durable peer disconnects, all subscriptions associated with it revert to pending and are automatically re-established when a new durable peer connects. A background retry loop (every 5 seconds) also picks up any subscriptions that failed with transient errors. Permanent failures (access denied, server error) are marked as failed and not retried.

Each subscription carries context data used for policy validation of incoming events and state.

Commit Paths

Ephemeral node commits locally

An ephemeral node creates events referencing the entity’s current head and sends them to durable peers via a CommitTransaction request. The durable peer validates, applies, and persists the events, then its reactor distributes the changes to all other connected peers. The originating ephemeral node may see its own events echo back as a streaming update; it recognizes them as re-delivery and no-ops.

Durable node commits locally

A durable node applies and persists events directly. Its reactor distributes the resulting updates to connected ephemeral peers. There is no upstream relay step because the durable node is itself the authority.

Durable node receives remote commit

When a durable node receives events from a peer, it validates each event against the policy agent, applies it to the entity (forking first for safe validation), persists the result, and notifies the reactor to propagate the change to other peers. See also the entity lifecycle document for the full commit flow.

Integration Test Patterns

See the Testing Strategy document for the full test matrix.

The durable_ephemeral tests exercise four core scenarios: ephemeral writes propagated to a durable node (including DAG forks), durable writes observed by an ephemeral node, cross-node concurrent writes that must converge, and late-arriving branches from deep history.

The multi_ephemeral tests extend this to topologies with multiple ephemeral nodes connected to a single durable node, verifying that independent writes, same-property conflicts, and three-way races all converge deterministically across all participants.

Property Backends

What is a Property Backend?

Every entity in ankurah stores its mutable state in one or more property backends. A backend is the layer responsible for three things: holding named property values, applying mutations to those values, and deciding what happens when two replicas change the same property concurrently.

The system ships two backends, each with a different conflict-resolution strategy:

• LWW (Last-Writer-Wins) – for scalar values (strings, integers, JSON, entity references). When two replicas write to the same property at the same time, a single winner is chosen deterministically so that every replica converges on the same result.
• Yrs – for collaboratively-editable text, powered by the Yrs CRDT library. Concurrent edits are merged automatically; no winner needs to be chosen because CRDT operations are commutative and idempotent.

When to use which: Use LWW for any property where the value is a discrete whole – a name, a count, a JSON blob, a reference to another entity. Use Yrs when the property is text that multiple users may edit simultaneously and you want their keystrokes to merge rather than overwrite each other.

The PropertyBackend Trait

Both backends implement a shared PropertyBackend trait (defined in core/src/property/backend/mod.rs). Rather than listing every method, the trait covers four responsibilities:

1. Mutation – collect local changes into serializable operations, and apply incoming operations from other replicas.
2. Serialization – snapshot the backend to a byte buffer and restore it later. Entity state is persisted as a State containing a Clock (the head) and a StateBuffers map keyed by backend name ("lww", "yrs"). See Entity State Persistence.
3. Query – read the current value of any property.
4. Lifecycle – fork a copy for transaction isolation and subscribe to per-field change notifications.

LWW Backend

Value lifecycle

An LWW property value moves through three states:

Only Committed entries carry an EventId. The backend refuses to serialize its state if any entry is still Uncommitted or Pending – this prevents persisting incomplete state.

Serialized LWW state buffers begin with a one-byte version header. Versions are offset high (0xA1 = version 1) because unversioned pre-0.9 buffers were raw bincode maps whose first byte is a small property count – one byte therefore classifies any buffer with no parse-probing. Unversioned buffers load through a legacy fallback: their values are stamped with an all-zeros sentinel event id, which merge resolution treats as older-than-meet, so any later write to the property wins – the same outcome true provenance would produce for pre-0.9’s linear histories, and one every replica computes identically. The next state save rewrites the buffer in the current versioned format, so old stores upgrade lazily. Buffers with an unknown future version are refused outright rather than guessed at.

Conflict resolution

When concurrent branches merge, the entity feeds each backend a sequence of EventLayers in topological order; each layer contains concurrent events. For each layer the LWW backend must choose a single winning value per property. The full algorithm is described in LWW Resolution Rules; the conceptual steps are:

1. Seed winners from stored state. Every existing Committed value becomes a candidate. If its EventId is absent from the accumulated DAG, the value is flagged older than meet.

2. Process layer events. Each event in the layer may write to the same property. The new candidate is compared against the current winner:

   • If the current winner is older than meet, the new candidate wins unconditionally.
   • Otherwise, causal ordering decides: a causally newer event replaces an older one.
   • For concurrent events (no causal relationship), the tie is broken by lexicographic EventId comparison.

3. Apply only new winners. Winners from events not yet in the entity’s state are written as Committed entries; winners from already-applied events need no mutation. Changed properties notify signal subscribers.

The “older than meet” rule

When the stored value’s EventId is not in the accumulated DAG, it was written by an event that predates the meet point. Any layer candidate is guaranteed to be at least as recent as the meet, so it wins unconditionally. This avoids expensive ancestry traversals to events outside the DAG.

Why lexicographic EventId comparison is safe

EventId is a SHA-256 hash of (entity_id, operations, parent_clock). The ordering of hashes has no relationship to wall-clock time, but it provides a deterministic total order: every replica comparing the same pair of concurrent events will pick the same winner. That is all convergence requires. See Determinism for the formal argument.

Yrs Backend

The Yrs backend stores a yrs::Doc document internally and tracks a StateVector for computing diffs.

Conflict resolution is far simpler than LWW: the backend applies every operation from new events and ignores already-applied events entirely. This works because:

• Commutativity – Yrs operations produce the same result regardless of application order. No winner selection is needed.
• Idempotency – Yrs deduplicates operations internally via its state vector. Re-applying an update is a no-op.

Known limitation: empty-string/null ambiguity

Yrs cannot distinguish between a text field that has never been written and one set to the empty string. An entity created with an empty Yrs property produces no CRDT operations, which can prevent persistence. This is tracked as issue #236 (originally reported as #175); see also Known Gaps.

Backend Registration

Backend instances are created on demand via backend_from_string in core/src/property/backend/mod.rs, which maps a name ("lww" or "yrs") to a constructor.

During layer application, if an event references a backend that does not yet exist on the entity, a new empty backend is created and all earlier layers are replayed on it before the current layer is applied. This ensures a backend first encountered mid-merge receives the full causal history from the meet point forward.

Value Types

User code interacts with backends through value-type wrappers generated by the derive macro system (defined in core/src/property/value/):

Both LWW<T> and YrsString<P> enforce write guards: calling a mutating method outside an active transaction returns PropertyError::TransactionClosed.

End-to-End Merge Flow

1. A remote event arrives and is staged. Entity::apply_event calls compare(), which returns DivergedSince.
2. The EventAccumulator produces EventLayers – topologically ordered batches of events from the meet point forward.
3. Each layer is applied to every backend: LWW resolves per-property winners; Yrs applies CRDT updates.
4. Late-created backends receive replayed earlier layers first.
5. The entity head is updated and signal subscribers are notified.

LWW Property Resolution During Concurrent Updates

The Problem

Two users concurrently edit the same entity. User A sets title = "Alpha" on one replica; User B sets title = "Beta" on another. Both writes happen independently – neither knows about the other. When the replicas sync, the system must pick one winner for title and guarantee that every replica picks the same one, no matter which write it saw first.

LWW (Last-Writer-Wins) resolution answers this question with two rules applied in order:

1. Causal dominance. If one write causally follows another (the writer had already seen the earlier value), the later write wins.
2. Deterministic tiebreak. If neither write has seen the other (they are truly concurrent), the write with the lexicographically greater EventId wins. Since EventIds are SHA-256 hashes, this is an arbitrary but consistent total order that every replica can compute independently.

The Three-Stage Pipeline

When the event DAG detects that branches have diverged, the merge proceeds in three conceptual stages:

1. Layer computation

All events between the meet point (the last common ancestor of the two branches) and the branch tips are partitioned into concurrency layers – groups of events at the same causal depth. Within a layer, events are either already applied (the replica has already incorporated them) or to-apply (new to this replica). Layers are produced in topological order so that earlier causal history is resolved before later history. See Event Layers for the precise definition of a layer and the guarantees it provides.

2. Per-layer resolution

For each layer, every LWW property is resolved independently. The algorithm starts with the property’s current stored value as the incumbent, then considers every event in the layer – both already-applied and to-apply – as challengers. The resolution rules below determine whether a challenger displaces the incumbent.

3. State mutation

Only winners that originate from to-apply events actually mutate the backend. If the winning write was already in the replica’s state (from an already-applied event or from the stored seed), nothing changes. After all mutations, subscribers are notified for each changed property.

Resolution Rules

When a challenger event competes against the current incumbent for a property, three rules are applied in priority order:

1. Older-than-meet rule. If the incumbent value was written by an event that predates the meet point, the challenger wins unconditionally. Rationale: every event in the layer descends from the meet, which itself descends from (or equals) the old event. The challenger is strictly newer.

2. Causal dominance. If one event is an ancestor of the other in the accumulated DAG, the descendant wins. This respects user intent: a write made after seeing a prior value should supersede it.

3. Lexicographic tiebreak. If the two events are truly concurrent (neither descends from the other), the one with the greater EventId wins. This is an arbitrary but deterministic rule that ensures every replica reaches the same conclusion.

If no incumbent exists for a property, the first event to write it wins by default.

Per-Property Independence

Each property is resolved independently. In a diamond DAG:

      A (genesis, title="Init", artist="Init")
     / \
    B    C

Event B writes title = "B-title". Event C writes artist = "C-artist".

• title: Only B touched it. B’s value wins.
• artist: Only C touched it. C’s value wins.

Final state: title = "B-title", artist = "C-artist". When both branches write the same property, the resolution rules above determine the winner. Different properties may have different winning events from the same layer.

Determinism

The system guarantees that the same set of concurrent events always resolves to the same property values, regardless of event delivery order.

Worked example

Replicas R1 and R2 both hold an entity at head [G] (genesis). Three concurrent events A, B, C – all parented on [G] – arrive in different orders. Assume EventId(A) < EventId(B) < EventId(C).

R1 receives A, then B, then C:

R2 receives C, then A, then B:

Both replicas converge to the same head [A, B, C] and the same winner (C) for every property that all three events wrote. The key insight: pairwise max() over EventIds is commutative and associative, so the evaluation order does not matter.

The ingredients of determinism

1. Lexicographic tiebreak is a total order. EventIds are SHA-256 hashes of (entity_id, operations, parent_clock). The byte-level ordering is consistent across all replicas.

2. max() is commutative and associative. Pairwise competition produces the same result regardless of evaluation order.

3. Causal dominance is objective. The DAG structure is the same on every replica, so ancestor/descendant queries return the same answers everywhere.

4. The older-than-meet rule is deterministic. Whether a stored event falls inside or outside the accumulated DAG depends only on the DAG contents, which converge across replicas.

5. Stored state tracks winners. Each layer updates the stored event_id for mutated properties, so subsequent layers seed from the correct incumbent.

Prerequisite: causal application order

Determinism relies on events being applied parents-first: if event D has parent [C], a replica must integrate C before D. This is an application-order invariant, not a transport guarantee – the receiver enforces it itself by staging each multi-event batch and topologically sorting it by in-batch parent edges (event_dag/ordering.rs) before applying. Sender order is never trusted.

Integration With Entity::apply_event

The DivergedSince handler in Entity::apply_event ties the pipeline together:

1. Decompose the comparison result into the causal relation and the accumulated DAG.
2. Build the layer iterator from the meet point and current head.
3. Collect all layers (async; may hit storage).
4. Acquire the entity write lock; re-check the head for TOCTOU safety.
5. For each layer in topological order, call the resolution algorithm on every property backend. If a to-apply event introduces a new backend type, create it and replay earlier layers first.
6. Update the head, release the lock, and broadcast change notifications.

Test Coverage

See Testing Strategy for the full test matrix. Key test files:

• tests/tests/lww_resolution.rs – deeper-branch-wins, sequential last write, lexicographic tiebreak, per-property independence, order independence.
• tests/tests/determinism.rs – two-event same-property determinism (the most critical test: two nodes apply events in opposite order and assert identical final state), deep diamond, multi-property convergence, three-way fork.

Testing Strategy: Event DAG and Concurrent Updates

Philosophy

Tests are structured in three levels, each proving something different:

1. Unit tests (core/src/event_dag/tests.rs) exercise DAG comparison and conflict resolution algorithms in isolation. No node, storage engine, or transaction is involved – just hand-built topologies and direct function calls. These prove the algorithms are correct.

2. Integration tests (tests/tests/) spin up real durable and ephemeral Node instances, create entities via the transaction API, commit events, and assert on persisted state and cross-node convergence. These prove the system behaves correctly end-to-end.

3. Staging lifecycle tests (core/src/retrieval.rs) verify the stage-compare-commit protocol – that staged events are visible to comparison but do not appear committed until explicitly promoted, and that the durable/ephemeral flag is reported correctly.

What the Tests Verify

Idempotency

Re-delivery of an already-applied event must be a no-op. This was the motivating bug for the Phase 5 staging rewrite: the old compare_unstored_event would corrupt the head when a historical event was re-delivered.

At the unit level, re-delivering the current head returns Equal and re-delivering an ancestor returns StrictAscends – both no-ops. Integration tests build a linear chain, re-deliver a middle event via commit_remote_transaction, and confirm the head, entity state, and event count are unchanged.

Determinism

The same set of events applied in any order must produce identical state. LWW resolution prefers causally newer writes and uses the lexicographic EventId tiebreak for truly concurrent ones. (Branch depth is never the rule: a longer branch wins only when its writing event causally descends the incumbent.)

Unit tests cover two-event, three-event (all six permutations), multi-property, and sequential-layer determinism at the LWWBackend level. Integration tests create events on one node, replay them in reversed order on a second node, and assert identical values. Three-way concurrent forks verify the winner matches the highest EventId.

Concurrent merge correctness

Several topologies are tested at both levels:

• Diamond merge – Two concurrent branches from a common ancestor, verifying DAG structure, multi-head state, and that both property changes are applied.
• Deep diamond – Symmetric and asymmetric branches (up to 8 events) asserting DivergedSince with correct meet point and chain lengths.
• Three-way concurrency – Three branches from the same head, verifying the lexicographic winner and assert_dag! structure.
• Multiple merge cycles – Repeated fork-merge sequences (e.g., A -> B||C -> D -> E||F -> G) verifying the full DAG with final single head.
• Missing events / BFS meet – Diamonds where the common ancestor is absent from the retriever. BFS discovers it on both frontiers and returns DivergedSince without erroring. A missing event on only one frontier correctly returns EventNotFound.

Durable/ephemeral interaction

See Node Architecture for the distinction and Replication Protocol for message formats.

Scenarios covered: ephemeral node writes while durable node receives via propagation; both nodes fork from the same head and commit concurrently, then converge; a late-arriving branch from deep (20-event) history merges without BudgetExceeded; multiple ephemeral nodes racing on the same property.

Creation event handling

See Guard Ordering for how creation events are handled.

A second, different genesis event applied to an existing entity produces Disjoint (mapped to MutationError::LineageError). Re-delivery of the exact same creation event is a no-op. Unit tests additionally verify that two independent DAGs sharing no common ancestor return Disjoint with correct root identification.

Budget escalation

Tests verify that budget escalation works: an initial budget of 2 internally escalates 4x to 8, which is enough for a 6-event chain. An initial budget of 1 (max 4 after escalation) correctly returns BudgetExceeded when the chain is longer.

Test Infrastructure

MockRetriever (unit tests) – Implements GetEvents with an in-memory HashMap. Tests build topologies imperatively with make_test_event(seed, parent_ids), which produces deterministic content-hashed IDs. No storage, node, or transaction needed.

TestDag (integration tests) – Defined in tests/tests/common.rs. Assigns single-character labels (A, B, C, …) to events in commit order. Two macros provide declarative verification:

#![allow(unused)]
fn main() {
assert_dag!(dag, events, {
    A => [],       // genesis
    B => [A],
    C => [A],
    D => [B, C],   // merge
});
clock_eq!(dag, state.payload.state.head, [D]);
}

Test models – Album uses Yrs-backed fields for multi-node and edge-case tests. Record uses LWW-backed fields for determinism and resolution tests.

Known Gaps

TOCTOU retry exhaustion (#[ignore]) – Testing retry exhaustion requires a mock that modifies the entity head between comparison and the CAS attempt inside try_mutate, which needs interior mutability and precise timing over the entity’s RwLock. Expected behavior: after MAX_RETRIES (5) attempts, apply_event returns Err(MutationError::TOCTOUAttemptsExhausted).

Yrs empty-string as null (#[ignore]) – Blocked on issue #236 (empty-string treated as null). Creating an entity with empty-string content produces no CRDT operations and no creation event.

Yrs multi-node concurrency – Order-independent convergence is tested at the unit level, but multi-node Yrs-specific concurrency (e.g., concurrent text inserts at the same position across nodes) is not yet covered.

Per-field notification path – Entity-level notification correctness is covered by tests/tests/notifications.rs (exactly one notification per commit, causal ordering across sequential commits, Add-vs-Update membership semantics, multi-subscriber consistency). What remains untested is the per-field path end to end: per-field subscription from a View is not yet supported, so no test subscribes to a single field and asserts a field-scoped notification.

apply_state divergence path – The integration test creates a diverged topology but verifies that both concurrent changes are applied, rather than testing the apply_state rejection path (Ok(false)) directly.

Multi-column ORDER BY (#[ignore]) – Three tests in tests/tests/sled/multi_column_order_by.rs are blocked on issue #210 (i64 sorted lexicographically). Unrelated to the event DAG.