Storage

Tenzro Network uses a high-performance embedded database as its persistent storage layer with multiple stores for different data types. The storage system includes Merkle Patricia Trie state roots for cryptographic verification, snapshot creation with compression, and optimized caching for high-performance blockchain operations.

Architecture Overview

The storage module provides persistent state management for the Tenzro Ledger L1. All blockchain data is stored in RocksDB, an embedded key-value store optimized for SSD storage with high write throughput.

Storage Engine: Embedded key-value store optimized for SSDs

Cache Size: 1 GB (configurable)

Snapshot Retention: 100 snapshots

State Root: Merkle Patricia Trie

Durability: Synchronous writes for finalized blocks

Data Stores

Separate data stores organize different data types for optimized access patterns and compaction strategies. Tenzro uses 9 distinct stores to organize blockchain data.

CF_BLOCKS

Stores finalized blocks indexed by block height. Each block contains transactions, state root, previous block hash, timestamp, and consensus metadata. Blocks are written with fsync for crash consistency.

// Block storage schema
Key:   block_height (u64, big-endian)
Value: Block {
    height: u64,
    hash: Hash,
    previous_hash: Hash,
    state_root: Hash,
    timestamp: u64,
    transactions: Vec<Hash>,
    metadata: BlockMetadata,
}

// Example operations
let block = storage.get_block(1234)?;
storage.store_block(&block)?;  // Calls write_batch_sync()
let latest_height = storage.latest_height()?;

CF_STATE

Stores current world state organized as a Merkle Patricia Trie. Keys are account addresses or contract storage slots; values are account state (balance, nonce, code hash, storage root).

// State storage schema
Key:   keccak256(address) or storage_key
Value: Account {
    balance: u128,
    nonce: u64,
    code_hash: Hash,
    storage_root: Hash,
}

// Merkle Patricia Trie node structure
TrieNode {
    Branch(children: [Option<Hash>; 16], value: Option<Vec<u8>>),
    Extension(prefix: Vec<u8>, child: Hash),
    Leaf(key_suffix: Vec<u8>, value: Vec<u8>),
}

// Example operations
let account = storage.get_account(&address)?;
storage.update_account(&address, &account)?;
let state_root = storage.compute_state_root()?;

CF_ACCOUNTS

Optimized index for quick account lookups without traversing the Merkle trie. Stores balances, nonces, and metadata for all accounts.

// Account index schema
Key:   address (20-32 bytes)
Value: AccountInfo {
    balance: u128,
    nonce: u64,
    created_at: BlockHeight,
    last_modified: BlockHeight,
}

// Fast balance queries for RPC
let balance = storage.get_balance(&address)?;
let nonce = storage.get_nonce(&address)?;

CF_TRANSACTIONS

Stores all transactions indexed by transaction hash. Includes transaction data, block height, execution result, and gas used.

// Transaction storage schema
Key:   transaction_hash (Hash)
Value: TransactionRecord {
    tx: Transaction,
    block_height: u64,
    block_index: u32,
    receipt: ExecutionReceipt {
        status: Success | Revert | OutOfGas,
        gas_used: u64,
        logs: Vec<Log>,
        created_contract: Option<Address>,
    },
}

// Retrieve transaction by hash
let tx_record = storage.get_transaction(&tx_hash)?;
let receipt = storage.get_receipt(&tx_hash)?;

CF_METADATA

Stores chain metadata: genesis configuration, current epoch, validator set, network parameters, and protocol version.

// Metadata storage schema
Key:   "genesis_hash" | "current_epoch" | "validator_set" | "chain_id"
Value: Serialized metadata

// Example keys
"genesis_hash" → Hash
"current_epoch" → u64
"validator_set" → Vec<ValidatorInfo>
"chain_id" → u64
"protocol_version" → String
"total_supply" → u128

CF_SNAPSHOTS

Stores compressed state snapshots for fast sync and recovery. Snapshots are created at configurable intervals (default: every 1000 blocks). Retention limit is 100 snapshots.

// Snapshot storage schema
Key:   block_height (u64)
Value: CompressedSnapshot {
    height: u64,
    state_root: Hash,
    compression: Zstd,
    data: Vec<u8>,
    size_uncompressed: u64,
    created_at: Timestamp,
}

// Snapshot operations
let snapshot = storage.create_snapshot(block_height)?;
storage.restore_snapshot(snapshot_height)?;
storage.prune_old_snapshots(retention: 100)?;

CF_SETTLEMENTS

Stores payment settlement records for AI inference, TEE services, and bridge transfers. Includes settlement proofs and receipts.

// Settlement storage schema
Key:   settlement_id (Hash)
Value: SettlementRecord {
    from: Address,
    to: Address,
    amount: u128,
    asset: Asset,
    proof: ServiceProof,
    receipt: SettlementReceipt,
    status: Pending | Completed | Failed,
    settled_at: BlockHeight,
}

// Query settlements
let settlement = storage.get_settlement(&id)?;
let receipts = storage.get_settlements_by_address(&address)?;

CF_CHANNELS

Stores micropayment channel state for off-chain per-token billing in AI inference. Channels enable low-latency payments without on-chain transactions for every token generated.

// Micropayment channel schema
Key:   channel_id (Hash)
Value: ChannelState {
    payer: Address,
    payee: Address,
    asset: Asset,
    initial_balance: u128,
    current_balance: u128,
    total_paid: u128,
    nonce: u64,
    opened_at: BlockHeight,
    expires_at: BlockHeight,
    status: Open | Closed | Challenged,
}

// Channel operations
let channel = storage.get_channel(&id)?;
storage.update_channel_balance(&id, new_balance)?;
storage.close_channel(&id)?;

CF_CHALLENGES

Stores HTTP 402 payment challenges for MPP and x402 protocols. Challenges are created by resource servers and verified against credentials from payers.

// Payment challenge schema
Key:   challenge_id (String)
Value: PaymentChallenge {
    id: String,
    resource: String,
    amount: u128,
    asset: Asset,
    protocol: Mpp | X402,
    expires_at: Timestamp,
    created_at: Timestamp,
}

// Challenge lookup for payment verification
let challenge = storage.get_challenge(&challenge_id)?;
storage.remove_challenge(&challenge_id)?;  // After settlement

Merkle Patricia Trie

The Merkle Patricia Trie provides cryptographic commitments to the world state. State roots are included in blocks for verification. The trie enables efficient state proofs for light clients.

Trie Structure

// Merkle Patricia Trie implementation
pub enum TrieNode {
    // Branch node with 16 children (hex nibbles)
    Branch {
        children: [Option<Hash>; 16],
        value: Option<Vec<u8>>,
    },
    // Extension node for path compression
    Extension {
        prefix: Vec<u8>,
        child: Hash,
    },
    // Leaf node with key suffix and value
    Leaf {
        key_suffix: Vec<u8>,
        value: Vec<u8>,
    },
}

// Trie operations
impl MerklePatriciaTrie {
    pub fn insert(&mut self, key: &[u8], value: &[u8]) -> Result<()>;
    pub fn get(&self, key: &[u8]) -> Result<Option<Vec<u8>>>;
    pub fn delete(&mut self, key: &[u8]) -> Result<()>;
    pub fn root_hash(&self) -> Hash;
    pub fn prove(&self, key: &[u8]) -> Result<Vec<TrieNode>>;
    pub fn verify_proof(root: Hash, key: &[u8], proof: &[TrieNode]) -> bool;
}

State Root Computation

State roots are computed after each block execution. The root hash commits to all account balances, nonces, contract code, and storage.

// Compute state root from all accounts
pub fn compute_state_root(storage: &Storage) -> Result<Hash> {
    let mut trie = MerklePatriciaTrie::new();

    // Insert all accounts into trie
    for (address, account) in storage.iter_accounts() {
        let key = keccak256(&address);
        let value = encode_account(&account);
        trie.insert(&key, &value)?;
    }

    Ok(trie.root_hash())
}

// Include state root in block
let state_root = compute_state_root(&storage)?;
let block = Block {
    height,
    previous_hash,
    state_root,  // Cryptographic commitment
    transactions,
    ..
};

Snapshot Creation and Restoration

Snapshots capture complete blockchain state at specific block heights. They enable fast sync for new nodes and disaster recovery. Snapshots are compressed with Zstd for efficient storage.

Creating Snapshots

// Create snapshot at current block height
pub fn create_snapshot(&self, height: u64) -> Result<Snapshot> {
    // Serialize all state data
    let mut data = Vec::new();

    // Export all column families
    data.extend(self.export_cf("CF_STATE")?);
    data.extend(self.export_cf("CF_ACCOUNTS")?);
    data.extend(self.export_cf("CF_BLOCKS")?);
    data.extend(self.export_cf("CF_TRANSACTIONS")?);

    // Compress with Zstd (level 3)
    let compressed = zstd::encode_all(&data[..], 3)?;

    let snapshot = Snapshot {
        height,
        state_root: self.compute_state_root()?,
        compression: Compression::Zstd,
        data: compressed,
        size_uncompressed: data.len() as u64,
        created_at: timestamp_now(),
    };

    // Store snapshot
    self.store_snapshot(&snapshot)?;

    // Prune old snapshots (keep 100)
    self.prune_old_snapshots(100)?;

    Ok(snapshot)
}

// Automatic snapshot creation every 1000 blocks
if block_height % 1000 == 0 {
    storage.create_snapshot(block_height)?;
}

Restoring from Snapshots

// Restore state from snapshot
pub fn restore_snapshot(&mut self, height: u64) -> Result<()> {
    // Load snapshot from storage
    let snapshot = self.get_snapshot(height)?;

    // Decompress data
    let data = zstd::decode_all(&snapshot.data[..])?;

    // Clear existing state
    self.clear_cf("CF_STATE")?;
    self.clear_cf("CF_ACCOUNTS")?;

    // Import snapshot data
    self.import_data(&data)?;

    // Verify state root matches
    let computed_root = self.compute_state_root()?;
    if computed_root != snapshot.state_root {
        return Err(StorageError::CorruptSnapshot);
    }

    info!("Restored snapshot at height {}", height);
    Ok(())
}

// Fast sync for new nodes
let latest_snapshot = storage.get_latest_snapshot()?;
storage.restore_snapshot(latest_snapshot.height)?;
// Then sync remaining blocks from peers

Performance Optimization

Cache Configuration

RocksDB uses a 1 GB block cache to reduce disk I/O. Cache configuration is optimized for SSD storage with high write throughput.

// RocksDB configuration
let mut opts = rocksdb::Options::default();
opts.create_if_missing(true);
opts.create_missing_column_families(true);
opts.set_max_open_files(10000);
opts.set_use_fsync(false);  // Use fdatasync for performance
opts.set_bytes_per_sync(1048576);  // 1 MB
opts.set_keep_log_file_num(10);
opts.set_max_background_jobs(4);

// Block cache (1 GB shared across all column families)
let cache = rocksdb::Cache::new_lru_cache(1024 * 1024 * 1024);
let mut block_opts = rocksdb::BlockBasedOptions::default();
block_opts.set_block_cache(&cache);
block_opts.set_block_size(16 * 1024);  // 16 KB
block_opts.set_cache_index_and_filter_blocks(true);
opts.set_block_based_table_factory(&block_opts);

// Write options for finalized blocks
let mut write_opts = rocksdb::WriteOptions::default();
write_opts.set_sync(true);  // fsync for durability

Compaction Strategy

// Level compaction for CF_STATE (write-heavy)
let mut cf_opts = rocksdb::Options::default();
cf_opts.set_compaction_style(rocksdb::DBCompactionStyle::Level);
cf_opts.set_level_compaction_dynamic_level_bytes(true);
cf_opts.set_max_bytes_for_level_base(512 * 1024 * 1024);  // 512 MB
cf_opts.set_target_file_size_base(64 * 1024 * 1024);  // 64 MB

// Universal compaction for CF_BLOCKS (append-only)
let mut block_cf_opts = rocksdb::Options::default();
block_cf_opts.set_compaction_style(rocksdb::DBCompactionStyle::Universal);
block_cf_opts.optimize_for_point_lookup(256);  // 256 MB hash index

Durability and Crash Recovery

Finalized blocks are written with fsync to ensure durability. The storage layer survives power loss and system crashes without data corruption.

// Write finalized block with fsync
pub fn store_block(&self, block: &Block) -> Result<()> {
    let key = block.height.to_be_bytes();
    let value = bincode::serialize(block)?;

    // Use sync write for finalized blocks
    let mut write_opts = rocksdb::WriteOptions::default();
    write_opts.set_sync(true);  // Forces fsync

    self.db.put_cf_opt(&self.cf_blocks, key, value, &write_opts)?;

    info!("Stored finalized block {} with fsync", block.height);
    Ok(())
}

// WAL (Write-Ahead Log) for crash recovery
// RocksDB automatically maintains WAL
// On restart, replays uncommitted writes from WAL

Storage Operations

Batch Writes

// Atomic batch write for block execution
pub fn commit_block_execution(&self, block: &Block, updates: StateUpdates)
    -> Result<()>
{
    let mut batch = rocksdb::WriteBatch::default();

    // Write block
    batch.put_cf(&self.cf_blocks,
        block.height.to_be_bytes(),
        bincode::serialize(block)?);

    // Write all state updates
    for (address, account) in updates.accounts {
        let key = keccak256(&address);
        let value = bincode::serialize(&account)?;
        batch.put_cf(&self.cf_state, key, value);
        batch.put_cf(&self.cf_accounts, address.as_bytes(), value);
    }

    // Write all transactions
    for tx in &block.transactions {
        let key = tx.hash.as_bytes();
        let value = bincode::serialize(tx)?;
        batch.put_cf(&self.cf_transactions, key, value);
    }

    // Atomic commit with sync
    let mut write_opts = rocksdb::WriteOptions::default();
    write_opts.set_sync(true);
    self.db.write_opt(batch, &write_opts)?;

    Ok(())
}

Iterators

// Iterate over all accounts
pub fn iter_accounts(&self) -> impl Iterator<Item = (Address, Account)> {
    self.db.iterator_cf(&self.cf_accounts, rocksdb::IteratorMode::Start)
        .map(|(key, value)| {
            let address = Address::from_slice(&key);
            let account = bincode::deserialize(&value).unwrap();
            (address, account)
        })
}

// Iterate blocks in range
pub fn iter_blocks(&self, start: u64, end: u64)
    -> impl Iterator<Item = Block>
{
    let start_key = start.to_be_bytes();
    self.db.iterator_cf(&self.cf_blocks,
        rocksdb::IteratorMode::From(&start_key, rocksdb::Direction::Forward))
        .take_while(move |(key, _)| {
            u64::from_be_bytes(key.as_ref().try_into().unwrap()) < end
        })
        .map(|(_, value)| bincode::deserialize(&value).unwrap())
}

Storage Constants

Storage Cache: 1 GB

Snapshot Retention: 100 snapshots

Snapshot Interval: Every 1000 blocks

Compression: Zstd level 3

Block Cache Size: 16 KB blocks

Max Open Files: 10,000

Background Jobs: 4 threads

WAL Size Limit: 1 GB

Best Practices

# Use SSD storage for production deployments
# NVMe is recommended for validators

# Mount with noatime for performance
/dev/nvme0n1 /data ext4 noatime,nodiratime 0 2

# Monitor disk usage
df -h /data
du -sh /data/rocksdb/*

# Backup snapshots to object storage
aws s3 sync /data/snapshots/ s3://tenzro-backups/snapshots/

# Compact database manually (if needed)
tenzro-node --compact-db --data-dir /data

← Networking Next: Consensus →