nodex-proxy

nodex-proxy converges a leafage-evm cluster into a single RPC endpoint. It decides by the request’s block context whether to send it to a State, Archive, or Native node, and takes on traffic governance duties such as rate limiting, mirroring, and observability.

Item	Value
Repository	Chaintable/nodex-proxy
Language	Go 1.22+
License	Apache-2.0
Core dependencies	Hertz, etcd client v3, OpenTelemetry, zap, sonic

Ports and paths

Port	Purpose
`8663`	JSON-RPC (`POST /:chainId`) and the admin API, sharing the same listener
`8664`	Prometheus metrics

One process serves multiple chains at the same time; chains are distinguished by the chainId in the path. Hexadecimal IDs are normalized to decimal strings.

The admin API shares the port with JSON-RPC and can change routing state. This port should not be exposed outside the internal network, or it needs to be placed behind an external authentication layer.

Request lifecycle

Pre-processing runs in a fixed order:

Record request → method deny list → method name validation → metrics → per-method rate limiting → method-specific rewrites → request mirroring

Post-processing:

Parse response → record response → method-specific post-processing → slow request / error log → metrics

Node selection

First decide the node pool by block context, then pick a specific node within the pool by strategy.

Request context	Node pool
`latest` / `pending`	State
Explicit height, within 64 blocks of the known chain head	State
Explicit height, more than 64 blocks behind the chain head	Archive
`Contains` type context	State
Explicit archive flag (request header)	Archive
Native retry	Native

The chain head height comes from {chainId}/lastBlockNumber in etcd, maintained by consistency-checker. When the chain height is unknown, it conservatively selects an Archive node. The two pools fall back to each other: when the State pool is empty, the Archive pool is used, and vice versa. Within each pool, Available nodes are used first.

Load balancing strategies

random (default)
round_robin

Selects by weighted probability. Non-batch requests first filter candidate nodes by method routing rules, then select by weight. Nodes without an explicit weight default to 100.

Automatic retries

Error code	Condition	Retry target
`-39006` `StateBlockNotFound`	The first request was not an archive request	Archive node
`-39008` `CosmosPrecompile`	—	Native node, upstream path rewritten to `/`

Service discovery

Watches an etcd prefix (etcd_prefix is configurable); the following key suffixes are interpreted as runtime data:

Key suffix	Purpose
`{chainId}/nodes/{nodeKey}`	State / Archive nodes
`{chainId}/nativeNodes/{nodeKey}`	Native fallback nodes
`{chainId}/lastBlockNumber`	Current chain head height
`{chainId}/gateway`	Node weights and method routing
`{chainId}/mirror/{addrKey}`	Mirror targets and optional rate limits
`{chainId}/version`	Overrides base chain routing to a versioned chain ID
`{chainId}/{version}/nodes/{nodeKey}`	Versioned nodes, internal ID normalized to `{chainId}-{version}`

The proxy watches PUT / DELETE events to add and remove nodes and update configuration in real time, with no restart needed.

Health check

A node that newly appears in etcd does not join the node pool immediately:

Probe

Send an RPC call to the node to verify reachability. State / Archive nodes use getLatestBlock, Native nodes use eth_blockNumber.

Retry

On failure, retry every 5 seconds until node_health_check_max_wait (default 300 seconds).

Join

Only after passing the check does the node enter the load balancing pool.

Version routing

When a request uses the base chain ID with no explicit version suffix, the proxy reads {chainId}/version from etcd and rewrites the request to the {chainId}-{version} node pool. This makes version switching transparent to clients.

Traffic governance

Capability	Config option	Description
Method deny list	`method_denied`	List of methods rejected outright; includes `txpool_*` and others by default
Method name validation	`method_name_checker`	Validates the method name format with a regex
Per-method rate limiting	`rate_limiter.rpc_methods`	Token bucket, RPS configured per method
Block range limit	`block_range_query_limit`	Limits block range queries; can rewrite to `latest`
Request mirroring	`request_mirror`	Asynchronously copies to a shadow backend without affecting the main request
Slow request log	`observability_log.slow_threshold`	Threshold configured per method

Usage reporting

Once usage is configured, RPC time is aggregated locally by the client-id request header and written to Kafka when 10000 aggregation keys or report_interval (default 5s) is reached.

{
  "id": "3c9d1b7e-52aa-4f0e-8d21-77b4e0c9a1f2",
  "client_id": "instance:019f45e26c307c86bd45ab350bb52ca8",
  "service": "leafage",
  "resource_type": "read",
  "usage": 123,
  "timestamp": 1783568373000
}

usage is the aggregated time in milliseconds, with a minimum of 1.
A missing client-id is recorded as unknown.
Sending is best-effort: the last batch is sent on graceful shutdown; loss is acceptable when the process crashes or Kafka fails.
The current number of aggregation keys is reflected by jrpcx_usage_aggregation_keys.

Admin API

Endpoint group	Purpose
`/getChains`, `/:chainId/getAllNodes`, `/:chainId/debug_chooseOneNode`	View chains, nodes, and selection behavior
`/:chainId/addNode`, `/updateNode/:nodeKey`, `/deleteNode/:nodeKey`	Node changes persisted to etcd
`/:chainId/addLocalNode`, `/deleteLocalNode/:nodeKey`	In-memory only, not written to etcd
`/:chainId/setWeight`, `/getWeight`, `/deleteWeight`	Weight management
`/:chainId/addMethodRoute`, `/removeMethodRoute`, `/deleteMethodRoute/:method`	Per-method include / exclude routing
`/:chainId/addMirror`, `/deleteMirror`, `/deleteAllMirrors`	Mirror targets (persisted to etcd)
`/:chainId/writers`, `/writers/leader`, `/writers/switchLeader`	View write nodes, view and switch the pipeline Leader

debug_chooseOneNode is useful when troubleshooting routing problems: it returns the node that would be selected under the current request conditions, without actually forwarding.

Metrics

Exposed at :8664/metrics, with common labels host, target, chain_id, chain_version.

Metric	Type
`jrpcx_rpc_calls_started` / `finished` / `failed`	Counter
`jrpcx_rpc_calls_time`	Histogram
`jrpcx_rpc_batch_calls_finished` / `_time`	Counter / Histogram
`jrpcx_rpc_request_payload_sizes` / `response_payload_sizes`	Histogram
`jrpcx_rpc_http_status_code`	Counter
`jrpcx_rpc_calls_cache_hits`	Counter

Some metrics carry extra labels:

Metric	Extra labels
Per-method metrics	`method`
`jrpcx_rpc_calls_started`	`sourcedapp`
Failure metrics	`status_code`, `upstream_related`, `reason`

Running

go build -o node-proxy cmd/proxy/main.go
./node-proxy -config config/config.example.yaml -listen 8663

Multiple instances can connect to the same etcd cluster to scale horizontally, each maintaining its own local selector state.

Development

make build
make test
make lint
make ci

docs/architecture_cn.md (Chinese) — Component boundaries and runtime architecture
docs/deployment_cn.md (Chinese) — Docker Compose, Kubernetes, systemd, and production tuning

Getting started

Core concepts

Architecture

Components

Guides

Reference

Ports and paths

Request lifecycle

Node selection

Load balancing strategies

Automatic retries

Service discovery

Health check

Version routing

Traffic governance

Usage reporting

Admin API

Metrics

Running

Development

​Ports and paths

​Request lifecycle

​Node selection

​Load balancing strategies

​Automatic retries

​Service discovery

​Health check

​Version routing

​Traffic governance

​Usage reporting

​Admin API

​Metrics

​Running

​Development

​Related documents

Ports and paths

Request lifecycle

Node selection

Load balancing strategies

Automatic retries

Service discovery

Health check

Version routing

Traffic governance

Usage reporting

Admin API

Metrics

Running

Development

Related documents