clockworklabs_spacetimedb/crates/bindings-cpp/ARCHITECTURE.md

SpacetimeDB C++ Bindings Architecture

Overview

The SpacetimeDB C++ bindings provides a sophisticated compile-time/runtime hybrid system for building database modules in C++ that compile to WebAssembly (WASM) and run inside the SpacetimeDB database. This document describes the architectural components, type registration flow, and key differences from other language SDKs.

Core Architecture Principles

1. Hybrid Compile-Time/Runtime System

• Compile-time validation: C++20 concepts and static assertions catch constraint violations before compilation
• Runtime registration: preinit functions execute during WASM module load to register types and metadata
• Nominal type system: Types identified by their declared names, not structural analysis
• Error detection: Multi-layer validation system from compile-time through module publishing

2. Outcome - Rust-Like Error Handling

The SDK provides Outcome<T>, a type-safe error handling mechanism matching Rust's Result<T, E> pattern:

• Outcome (type alias ReducerResult): Used by reducers, can return success (Ok()) or error (Err(message))
• Outcome: Useful for methods used by Reducers to return a value or an error message
• No exceptions: Errors are handled via return values, not C++ exceptions
• Graceful error handling: Reducer errors are caught by the runtime, rolled back, and reported to the caller without crashing
• Serializable: Error messages are automatically serialized and sent to clients

2. Priority-Ordered Initialization System

The SDK uses a numbered preinit function system to ensure correct initialization order:

__preinit__01_ - Clear global state (first)
__preinit__10_ - Field registration
__preinit__19_ - Auto-increment integration and scheduled reducers
__preinit__20_ - Table and lifecycle reducer registration
__preinit__21_ - Field constraints
__preinit__25_ - Row level security filters
__preinit__30_ - User reducers
__preinit__40_ - Views
__preinit__50_ - Procedures
__preinit__99_ - Type validation and error detection (last)

Error Handling: Outcome System

Overview

The C++ bindings uses Outcome<T> for type-safe, exception-free error handling that matches Rust's Result<T, E> pattern (where E is always std::string).

Type Aliases and Core Types

// For reducers - cannot fail with a value, only with an error message
using ReducerResult = Outcome<void>;

Reducer Error Handling (ReducerResult / Outcome)

Creating Results:

#include <spacetimedb.h>

using namespace SpacetimeDB;

struct User {
    Identity identity;
    std::optional<std::string> name;
    bool online;
};
SPACETIMEDB_STRUCT(User, identity, name, online);
SPACETIMEDB_TABLE(User, user, Public);
FIELD_PrimaryKey(user, identity);


SPACETIMEDB_REDUCER(create_user, ReducerContext ctx, std::string name) {
    // Validation with early error return
    if (name.empty()) {
        return Err("Name cannot be empty");
    }
    if (name.length() > 255) {
        return Err("Name is too long");
    }
    
    // Success path
    ctx.db[user].insert(User{ctx.sender(), name, false});
    return Ok();  // No value needed - just success
}

Checking Results:

SPACETIMEDB_REDUCER(call_other_logic, ReducerContext ctx) {
    // Note: In practice, reducers don't call other reducers directly
    // But if implementing error-handling helper functions:
    auto result = validate_something();
    
    if (result.is_err()) {
        return Err(result.error());  // Propagate error
    }
    
    // Continue on success
    return Ok();
}

Error Semantics:

• When Err() is returned:
  • The reducer transaction is rolled back (not committed to the log)
  • The error message is captured and returned to the caller
  • No database changes are persisted
  • No WASM crash or panic occurs
• When Ok() is returned:
  • All database mutations are committed
  • The transaction is logged
  • Success is reported to the caller

Procedure Error Handling

Key Difference from Reducers:

• Procedures return raw T (not Outcome<T>)
• On error, procedures can use LOG_PANIC() or LOG_FATAL() to end the host call which uses std:abort() behind the scenes
• Return values are sent directly to the caller

Outcome API Reference

// Creating success outcomes
Outcome<T>::Ok(value)      // Outcome<T> - with a value
Ok()                       // Outcome<void> - without a value  
Ok(value)                  // Helper - type deduced from value

// Creating error outcomes
Outcome<T>::Err(message)   // Outcome<T> - with error message
Err(message)               // Outcome<void> - with error message
Err<T>(message)            // Helper - explicit type specification

// Checking results
outcome.is_ok()            // bool - true if success
outcome.is_err()           // bool - true if error

// Accessing values/errors
outcome.value()            // T& or T&& - get success value (UB if error)
outcome.error()            // const std::string& - get error message (UB if success)

Design Rationale

Why not exceptions?

• WASM modules have limited error handling facilities, the latest WASM allows for them but requires GC
• Exceptions add code size and complexity
• Explicit error returns fit better with BSATN serialization
• Matches Rust SDK's error handling pattern

Why separate ReducerResult and Outcome?

• Reducers need rollback semantics (transactions)
• ReducerResult provides clearer intent for reducer code
• Outcome is more flexible for general operations

---

Detailed Type Registration Flow

Phase 1: Compile-Time Validation

Location: Template instantiation during compilation

Components:

• C++20 Concepts (table_with_constraints.h):

  template<typename T>
  concept FilterableValue = 
      std::integral<T> ||
      std::same_as<T, std::string> ||
      std::same_as<T, Identity> ||
      // ... other filterable types
  
  template<typename T>
  concept AutoIncrementable = 
      std::same_as<T, int8_t> ||
      std::same_as<T, uint32_t> ||
      // ... integer types only
  

• Static Assertions in FIELD_ macros:

  #define FIELD_Unique(table_name, field_name) \
      static_assert([]() constexpr { \
          using FieldType = decltype(std::declval<TableType>().field_name); \
          static_assert(FilterableValue<FieldType>, \
              "Field cannot have Unique constraint - type is not filterable."); \
          return true; \
      }(), "Constraint validation for " #table_name "." #field_name);
  

Validation Coverage:

• ✅ AutoIncrement constraints (only integer types)
• ✅ Index/Unique/PrimaryKey constraints (only filterable types)
• ✅ Type compatibility with BSATN serialization
• ✅ Template parameter validation

Error Output: Clear compile-time error messages with specific guidance

Phase 2: Runtime Registration (preinit functions)

Location: WASM module load, before any user code executes

2.1 Global State Initialization (_preinit__01)

extern "C" __attribute__((export_name("__preinit__01_clear_global_state")))
void __preinit__01_clear_global_state() {
    ClearV9Module();  // Reset module definition and handler registries
    getModuleTypeRegistration().clear();  // Reset type registry and error state
}

2.2 Component Registration (_preinit__10-30)

Generated by macros during preprocessing:

Table Registration (_preinit__20):

SPACETIMEDB_TABLE(User, users, Public)
// Generates:
extern "C" __attribute__((export_name("__preinit__20_register_table_User_line_42")))
void __preinit__20_register_table_User_line_42() {
    SpacetimeDB::Module::RegisterTable<User>("users", true);
}

Field Constraints (_preinit__21):

FIELD_PrimaryKey(users, id);
// Generates:
extern "C" __attribute__((export_name("__preinit__21_field_constraint_users_id_line_43")))
void __preinit__21_field_constraint_users_id_line_43() {
    getV9Builder().AddFieldConstraint<User>("users", "id", FieldConstraint::PrimaryKey);
}

Auto-Increment Integration Registration (_preinit__19): Auto-increment fields require special handling during insert() operations. When SpacetimeDB processes an auto-increment insert, it returns only the generated column values (not the full row) in BSATN format. The C++ bindings uses a registry-based integration system to properly handle these generated values and update the user's row object.

FIELD_PrimaryKeyAutoInc(users, id);
// Generates both constraint registration AND auto-increment integration:

// 1. Auto-increment integration function (unique per field via __LINE__)
namespace SpacetimeDB { namespace detail {
    static void autoinc_integrate_47(User& row, SpacetimeDB::bsatn::Reader& reader) {
        using FieldType = decltype(std::declval<User>().id);
        FieldType generated_value = SpacetimeDB::bsatn::deserialize<FieldType>(reader);
        row.id = generated_value;  // Update field with generated ID
    }
}}

// 2. Registration function to register the integrator
extern "C" __attribute__((export_name("__preinit__19_autoinc_register_47")))
void __preinit__19_autoinc_register_47() {
    SpacetimeDB::detail::get_autoinc_integrator<User>() = 
        &SpacetimeDB::detail::autoinc_integrate_47;
}

Runtime Integration Process: When insert() is called on a table with auto-increment fields:

1. The logic in the bindings serializes and sends the row to SpacetimeDB
2. SpacetimeDB processes the insert and generates the auto-increment value(s)
3. SpacetimeDB returns a buffer containing only the generated column values in BSATN format
4. SDK calls the registered integrator function to update the original row with generated values
5. insert() returns the updated row with the correct generated ID

This system enables users to immediately access generated IDs:

struct User {
    uint64_t id;
    std::optional<std::string> name;
};
SPACETIMEDB_STRUCT(User, id, name);
SPACETIMEDB_TABLE(User, user, Public);
FIELD_PrimaryKeyAutoInc(user, id);

SPACETIMEDB_REDUCER(create_user2, ReducerContext ctx, std::string name) {
    User new_user{0, name};  // id=0 will be auto-generated
    User inserted_user = ctx.db[user].insert(new_user);  // Returns user with generated ID
    LOG_INFO("Created user with ID: " + std::to_string(inserted_user.id));
    return Ok();  // Must return ReducerResult
}

Reducer Registration (_preinit__30):

SPACETIMEDB_REDUCER(add_user, ReducerContext ctx, std::string name) {
    if (name.empty()) {
        return Err("Name cannot be empty");  // Return error - rolled back
    }
    ctx.db[user].insert(User{0, name});
    return Ok();  // Success - transaction committed
}
// Generates registration function that captures parameter types, creates dispatch handler,
// and wraps return value in ReducerResult (Outcome<void>)

2.3 Multiple Primary Key Detection

During constraint registration, track primary keys per table:

// In V9Builder::AddFieldConstraint
if (constraint == FieldConstraint::PrimaryKey) {
    if (table_has_primary_key[table_name]) {
        SetMultiplePrimaryKeyError(table_name);  // Set global error flag
    }
    table_has_primary_key[table_name] = true;
}

Phase 3: Type System Registration

Component: ModuleTypeRegistration system (module_type_registration.h)

Core Principle: Only user-defined structs and enums get registered in the typespace. Primitives, arrays, Options, and special types are always inlined.

Architecture Note: V9Builder serves as the registration coordinator but delegates all type processing to the ModuleTypeRegistration system. This separation ensures a single, unified type registration pathway.

Registration Flow:

class ModuleTypeRegistration {
    AlgebraicType registerType(const bsatn::AlgebraicType& bsatn_type,
                              const std::string& explicit_name = "",
                              const std::type_info* cpp_type = nullptr) {
        // 1. Check if primitive → return inline
        if (isPrimitive(bsatn_type)) return convertPrimitive(bsatn_type);
        
        // 2. Check if array → return inline Array with recursive element processing
        if (bsatn_type.tag() == bsatn::AlgebraicTypeTag::Array) 
            return convertArray(bsatn_type);
        
        // 3. Check if Option → return inline Sum structure
        if (isOptionType(bsatn_type)) return convertOption(bsatn_type);
        
        // 4. Check if special type → return inline Product structure
        if (isSpecialType(bsatn_type)) return convertSpecialType(bsatn_type);
        
        // 5. User-defined type → register in typespace, return Ref
        return registerUserDefinedType(bsatn_type, explicit_name, cpp_type);
    }
};

Circular Reference Detection:

// Track types currently being registered
std::unordered_set<std::string> types_being_registered_;

AlgebraicType registerUserDefinedType(...) {
    if (types_being_registered_.contains(type_name)) {
        setError("Circular reference detected in type: " + type_name);
        return createErrorType();
    }
    types_being_registered_.insert(type_name);
    // ... process type ...
    types_being_registered_.erase(type_name);
}

Phase 4: Validation and Error Detection (_preinit__99)

Location: Final preinit function - runs after all registration is complete

Error Detection:

extern "C" __attribute__((export_name("__preinit__99_validate_types")))
void __preinit__99_validate_types() {
    // 1. Check for circular reference errors
    if (g_circular_ref_error) {
        createErrorModule("ERROR_CIRCULAR_REFERENCE_" + g_circular_ref_type_name);
        return;
    }
    
    // 2. Check for multiple primary key errors
    if (g_multiple_primary_key_error) {
        createErrorModule("ERROR_MULTIPLE_PRIMARY_KEYS_" + g_multiple_primary_key_table_name);
        return;
    }
    
    // 3. Check for type registration errors
    if (getModuleTypeRegistration().hasError()) {
        createErrorModule("ERROR_TYPE_REGISTRATION_" + sanitize(error_message));
        return;
    }
}

Error Module Creation: When errors are detected, the normal module is replaced with a special error module containing an invalid type reference. When SpacetimeDB tries to resolve the type, it fails with an error message that includes the descriptive error type name.

Phase 5: Module Description Export

Function: __describe_module__() - Called by SpacetimeDB after preinit functions complete

Process:

1. Serialize the completed V9 module definition
2. Include typespace (all registered types)
3. Include tables with constraints
4. Include reducers with parameter types
5. Include named type exports
6. Return binary module description

Namespace Qualification System

Overview

The C++ bindings provides a unique compile-time namespace qualification system for enum types, allowing better organization in generated client code without affecting server-side C++ usage.

Architecture Components

1. Compile-Time Namespace Storage

Location: enum_macro.h - namespace_info template specialization

namespace SpacetimeDB::detail {
    // Primary template - no namespace by default
    template<typename T>
    struct namespace_info {
        static constexpr const char* value = nullptr;
    };
}

// SPACETIMEDB_NAMESPACE macro creates specialization
#define SPACETIMEDB_NAMESPACE(EnumType, NamespacePrefix) \
    namespace SpacetimeDB::detail { \
        template<> \
        struct namespace_info<EnumType> { \
            static constexpr const char* value = NamespacePrefix; \
        }; \
    }

2. LazyTypeRegistrar Integration

Location: module_type_registration.h - Compile-time namespace detection

template<typename T>
class LazyTypeRegistrar {
    static bsatn::AlgebraicType getOrRegister(...) {
        std::string qualified_name = type_name;
        
        // Compile-time check for namespace information
        if constexpr (requires { SpacetimeDB::detail::namespace_info<T>::value; }) {
            constexpr const char* namespace_prefix = 
                SpacetimeDB::detail::namespace_info<T>::value;
            if (namespace_prefix != nullptr) {
                qualified_name = std::string(namespace_prefix) + "." + type_name;
            }
        }
        
        // Register with qualified name
        type_index_ = getModuleTypeRegistration().registerAndGetIndex(
            algebraic_type, qualified_name, &typeid(T));
    }
};

3. Type Registration with Namespaces

When an enum with namespace qualification is registered:

1. SPACETIMEDB_ENUM defines the enum and its BSATN traits
2. SPACETIMEDB_NAMESPACE adds compile-time metadata
3. LazyTypeRegistrar detects namespace at compile-time
4. Type is registered with qualified name (e.g., "Auth.UserRole")
5. Client generators recognize the namespace structure

Design Rationale

Why Separate Macros?

• Clean separation of concerns: enum definition vs. namespace qualification
• Optional feature - enums work without namespaces
• Non-intrusive - doesn't modify the enum type itself
• Compile-time only - zero runtime overhead

Why Template Specialization?

• Type-safe association between enum and namespace
• Compile-time resolution - no runtime lookups
• Works with C++20 concepts and if constexpr
• No memory overhead - constexpr strings

Comparison with Other Approaches

Alternative 1: Preinit Runtime Modification (Rejected)

• Would require modifying types after registration
• Complex synchronization with type registry
• Runtime overhead for namespace lookup

Alternative 2: Embedded in SPACETIMEDB_ENUM (Rejected)

• Would complicate the macro syntax
• Makes namespace mandatory rather than optional
• Harder to add namespaces to existing code

Current Approach Benefits:

• Clean, modular design
• Zero runtime cost
• Optional and backwards-compatible
• Easy to understand and maintain

Key Differences from Rust and C# SDKs

1. Type Registration Approach

Rust bindings:

• Derive macros automatically generate type registration code
• Compile-time code generation using procedural macros
• Direct integration with Rust's type system
• Option types automatically inlined by macro system

C# bindings:

• Reflection-based runtime type discovery
• Attribute-based configuration
• Dynamic type registration during module initialization
• .NET type system integration

C++ bindings:

• Template-based compile-time validation with runtime registration
• Macro-generated preinit functions for ordered initialization
• Manual type registration via SPACETIMEDB_STRUCT macros
• Hybrid approach combining compile-time safety with runtime flexibility

2. Constraint Validation

Rust bindings:

• Procedural macros generate compile-time validation
• Type system automatically enforces valid constraints
• No runtime constraint checking needed

C# bindings:

• Runtime validation using reflection
• Attributes specify constraints, validated during registration
• Dynamic error reporting

C++ bindings:

• Three-layer validation system:
  1. Compile-time: C++20 concepts and static assertions
  2. Registration-time: Multiple primary key detection
  3. Module load: preinit_99_ comprehensive validation
• Most sophisticated error detection of all SDKs

3. Error Handling Strategy

Rust bindings:

• Result<T, E> for operation errors with rich error types
• Compile-time errors prevent building invalid modules
• Type system prevents most runtime errors
• Standard Rust error messages

C# bindings:

• Runtime exceptions with detailed error messages
• Graceful error handling with exception propagation
• .NET debugging tools integration

C++ bindings - Two-Tier System:

1. Reducer errors (ReducerResult / Outcome):

   • Return Ok() on success (transaction committed)
   • Return Err(message) on failure (transaction rolled back)
   • Exceptions not used for normal error cases
   • Matches Rust's Result<(), E> pattern

2. Type registration errors:

   • Invalid modules replaced with special error modules
   • Error type names embed descriptive information
   • SpacetimeDB server provides clear error messages
   • Comprehensive error categorization and reporting

Outcome Type:

• Type-safe, exception-free error handling
• Serializable to binary format for client transmission
• Works in WASM environment without exception infrastructure
• API matches Rust Result pattern: is_ok(), is_err(), value(), error()

4. Type System Philosophy

Rust bindings:

• "If it compiles, it works" - maximum compile-time validation
• Leverages Rust's ownership and type system
• Minimal runtime overhead

C# bindings:

• "Flexibility with safety" - runtime validation with rich error messages
• Leverages .NET reflection and attributes
• Dynamic type discovery

C++ bindings:

• "Validate early, validate often" - multi-layer validation system
• Combines C++20 compile-time features with runtime checks
• Nominal type system with explicit registration
• Optimized for catching errors at the earliest possible phase

Memory Management and Performance

Compile-Time Optimizations

• Template specialization eliminates runtime overhead
• Constexpr evaluations reduce WASM binary size
• Zero-cost abstractions for type-safe database access

Runtime Efficiency

• Minimal allocation during type registration
• Efficient binary serialization with BSATN
• Optimized field accessors with index caching

WASM Constraints

• 16MB initial memory limit (configurable)
• No dynamic memory growth during module registration
• Careful memory management in preinit functions

Development Workflow Integration

Error Detection Timeline

Developer writes code
    ↓
C++ compilation → Compile-time validation (concepts, static_assert)
    ↓
Emscripten WASM build → Template instantiation validation
    ↓
Module publishing → Runtime validation (__preinit__99_)
    ↓
SpacetimeDB loading → Server-side validation and error reporting

Debugging Support

• Compile-time: Clear error messages with field/constraint guidance
• Build-time: Template instantiation error reporting
• Runtime: Comprehensive logging with error categorization
• Server-side: Descriptive error module names for easy diagnosis

Future Architecture Considerations

Potential Improvements

1. Unified Validation: Move more validation to compile-time using concepts
2. Better Error Recovery: Partial module loading with isolated error handling
3. Performance Optimization: Reduce template instantiation overhead
4. Enhanced Debugging: Source location tracking for runtime errors

Scalability

• Type registration system scales linearly with module complexity
• Preinit function count grows with table/reducer count but remains manageable
• Memory usage is predictable and bounded

Related Documentation

• Type System Details
• Constraint Validation Tests
• API Reference
• Quick Start Guide