Why split the full node
At production scale, scaling query capacity by “deploying a few more Geth instances” runs into several hard problems:
Leafage’s answer: let only one node execute blocks; every other node consumes the execution results.
Component responsibilities
Write node
The execution client plus the pipeline integration; exports state changes, call traces, and events while executing blocks.
pipeline
Distribution library. Serializes execution data, writes to two S3 buckets, and on the Leader node publishes block change notifications to Kafka.
leafage-evm
Query node. Consumes Kafka + S3 to rebuild state and runs
eth_call with revm; no P2P, no transaction storage.consistency-checker
Verification layer. Waits for replicas to converge, marks fork blocks, maintains node health state in etcd, and publishes confirmed notifications to the external Kafka.
nodex-proxy
Gateway. Discovers nodes through etcd and routes by block context to the State / Archive / Native node pools.
Runtime topology
Data plane and control plane
The key to understanding Leafage is to look at the two paths separately. They use different middleware, and a failure in each affects a different scope.Data plane: Kafka + S3
Block data flows in one direction only, from the write node to consumers.- Kafka (internal topic) carries notifications only, not data payloads. A message holds the block hash, parent hash, height, timestamp, and the list of blocks dropped during a reorg.
- S3 (internal bucket) stores Header and StateDiff; it is the data source leafage-evm rebuilds state from.
- S3 (external bucket) stores BlockFile (transactions, traces, events) and BlockValidation for external consumers and analytics platforms.
- Kafka (external topic) is published by consistency-checker; it is the entry point external consumers should subscribe to, and it comes with consistency guarantees.
Control plane: etcd
etcd is the source of truth for all runtime state; three components coordinate through it:
See Interface contracts for the full key space.
Node types
nodex-proxy routes between the three node pools based on the block parameter carried by the request; see the nodex-proxy component documentation for the rules.
Deployment unit
One chain is one independent deployment unit; chains do not share Kafka topics, S3 prefixes, or etcd prefixes (all of which start withchainID). A typical single-chain deployment includes:
chainId in the path, so usually the whole cluster deploys only one set of proxies.
Multi-chain support
The write side and the read side are adapted separately:- Write side: pipeline must be embedded in the target chain’s execution client. Client forks for 40+ chains have already been adapted; see the leafage-evm README for the list. See Adding a new chain for how to adapt one.
- Read side: leafage-evm selects the executor through
--evm-type; it currently supportsmainnet,arbitrum,op,base,bsc,cosmos,mantlev2,tempo,citrea,iotex,moonbeam,moonriver,polygon,hemi.
Design trade-offs
leafage-evm does not store transaction data
leafage-evm does not store transaction data
eth_call only needs account state (balance, nonce, code, storage), so transaction bodies, receipts, and logs can all be discarded. The cost is that the block query APIs (eth_getBlockByNumber and so on) return only the header, and transactions and uncles are always empty. Consumers that need transaction data should read the S3 external bucket or subscribe to the external Kafka topic.Only 64 blocks of diffs are kept in memory
Only 64 blocks of diffs are kept in memory
The state of the most recent 64 blocks stays in memory as a linked list of diff layers; layers beyond that depth are flushed to RocksDB. This window is also the upper bound on reorg handling: deeper reorgs must go through the S3 backfill path. The window size is controlled by
--diff-depth-limit.Consistency is not guaranteed by leafage-evm itself
Consistency is not guaranteed by leafage-evm itself
All query nodes consume the same Kafka partition and apply the same blocks in the same order, so there is no P2P non-determinism. But “have all replicas caught up” is decided by consistency-checker, which notifies external consumers only after
ready_ratio (default 0.8) is reached.Kafka carries notifications only; payloads go through S3
Kafka carries notifications only; payloads go through S3
Notification messages are small, so Kafka is responsible only for ordering and low latency; the large StateDiff and BlockFile payloads go through S3, where consumers can fetch in parallel and backfill history on demand during a cold start.
Next steps
Data flow
Follow one block through the five stages: execution, distribution, ingestion, finalization, and query.
Interface contracts
Exact format definitions for Kafka, S3, etcd, and RPC.