Reth is an all-in-one toolkit for building high performance and customizable nodes. We recently published our performance roadmap for improving Reth’s performance >100x, and Reth AlphaNet, our testnet rollup for pushing Reth’s modularity and extensibility to the limits.

Today, we are excited to announce Reth Execution Extensions (ExEx). ExEx is a framework for building performant and complex off-chain infrastructure as post-execution hooks. Reth ExExes can be used to implement rollups, indexers, MEV bots and more with >10x less code than existing methods. With this release, we demonstrate from scratch a prod-ready reorg tracker in <20 LoC, an indexer in <250 LoC and a rollup in <1000 LoC.

ExEx was co-architected with init4, a research collective building next-generation Ethereum infrastructure. We look forward to continuing to collaborate with the init4 team as we make Reth the #1 platform for building crypto infrastructure!

How do we build off-chain infrastructure today?

A blockchain is a clock that confirms blocks with transaction data on a regular interval. Off-chain infrastructure subscribes to these regular block updates and updates its own internal state as a response.

For example, consider how an Ethereum indexer works:

1. It subscribes to Ethereum events such as blocks and logs, usually over eth_subscribe or by polling with eth_getFilterChanges.
2. On each event, it proceeds to fetch any additional data needed over JSON-RPC such as the receipts alongside the block and its transactions.
3. For each payload it ABI decodes the logs it needs based on a configuration such as the address or the topics it cares about.
4. For all decoded data, it writes them to a database such as Postgres or Sqlite.

This is the standard Extract Transform Load (ETL) pattern that you see in large scale data pipelines, with companies like Fivetran owning the data extraction, Snowflake handling the loading into a data warehouse, and customers focusing on writing the transformation’s business logic.

We observe that this same pattern also applies to other pieces of crypto infrastructure such as rollups, MEV searchers, or more complex data infrastructure like Shadow Logs.

Using that as motivation, we identify key challenges when building ETL pipelines for Ethereum nodes:

1. Data Freshness: Chain reorganizations mean that most infrastructure is usually trailing behind the tip of the chain to avoid operating over state that might no longer be part of the canonical chain. This in practice means that building real-time crypto data products is challenging, evidenced by the proliferation of products with high latencies (on the order of multiple blocks, tens of seconds) relative to what they could be providing to their customers. We believe that this happens because nodes do not have a great developer experience for reorg-aware notification streams.
2. Performance: Moving data, transforming it and stitching it together across different systems means there are non negligible performance overheads. For example, a Reth-based indexer that directly plugs on Reth’s database showed 1-2 orders of magnitude improvement vs other indexers that plug on JSON-RPC, pointing at serious improvements by colocating workloads and removing intermediate layers of communication.
3. Operational Complexity: Running Ethereum nodes with high uptime is already a big challenge. Running additional infrastructure on top of them further exacerbates the problem and requires developers to think about job orchestration APIs, or running multiple services for relatively simple tasks.

There is a need for a better API for building off-chain infrastructure that depends on a node’s state changes. That API must be performant, ‘batteries-included’ and reorg-aware. We need an Airflow moment for building Ethereum ETL infrastructure and job orchestration.

Introducing Reth Execution Extensions (ExEx)

Execution Extensions (ExExes) are post-execution hooks for building real-time, high performance and zero-operations off-chain infrastructure on top of Reth.

An Execution Extension is a task that derives its state from Reth's state. Some examples of such state derives are rollups, indexers, MEV extractors, and more. We expect that developers will build reusable ExExes that compose with each other in a standardized way, similar to how Cosmos SDK modules or Substrate Pallets work.

We co-architected Execution Extensions with the init4 team (follow them!), a new research collective building next-generation Ethereum infrastructure. We are excited to continue collaborating with their team as we productionize ExExes and make Reth the #1 platform for building crypto infrastructure!

We are still early in the best practices of building ExExes, and we’d like to invite developers to join us in exploring this new frontier of building off-chain crypto infrastructure. Please reach out with ideas to collaborate.

How do ExExes work?

In Rust terms, an ExEx is a Future that is run indefinitely alongside Reth. ExExes are initialized using an async closure that resolves to the ExEx. Here is the expected end to end flow:

1. Reth exposes a reorg-aware stream called ExExNotification which includes a list of blocks committed to the chain, and all associated transactions & receipts, state changes and trie updates with them.
2. Developers are expected to consume that stream by writing ExExes as async functions that derive state such as a rollup block. The stream exposes a ChainCommitted variant for appending to ExEx state and a ChainReverted/Reorged-variant for undoing any changes. This is what allows ExExes to be operating at native block time, while also exposing a sane API for handling reorgs safely, instead of not handling reorgs and introducing latency.
3. ExExes get orchestrated by Reth’s ExExManager that is responsible for routing notifications from Reth to ExExes and ExEx events back to Reth, while Reth’s task executor drives ExExes to completion.
4. Each ExEx gets installed on the node via the install_exex API of the Node Builder.

Here is how this roughly looks like from the node developer’s perspective:

use futures::Future;
use reth_exex::{ExExContext, ExExEvent, ExExNotification};
use reth_node_api::FullNodeComponents;
use reth_node_ethereum::EthereumNode;

// The `ExExContext` is available to every ExEx to interface with the rest of the node.
// 
// pub struct ExExContext<Node: FullNodeComponents> {
//     /// The configured provider to interact with the blockchain.
//     pub provider: Node::Provider,
//     /// The task executor of the node.
//     pub task_executor: TaskExecutor,
//     /// The transaction pool of the node.
//     pub pool: Node::Pool,
//     /// Channel to receive [`ExExNotification`]s.
//     pub notifications: Receiver<ExExNotification>,
//     // .. other useful context fields
// }
async fn exex<Node: FullNodeComponents>(mut ctx: ExExContext<Node>) -> eyre::Result<()> {
    while let Some(notification) = ctx.notifications.recv().await {
        match &notification {
            ExExNotification::ChainCommitted { new } => {
                // do something
            }
            ExExNotification::ChainReorged { old, new } => {
                // do something
            }
            ExExNotification::ChainReverted { old } => {
                // do something
            }
        };
    }
    Ok(())
}

fn main() -> eyre::Result<()> {
    reth::cli::Cli::parse_args().run(|builder, _| async move {
        let handle = builder
            .node(EthereumNode::default())
            .install_exex("Minimal", |ctx| async move { exex(ctx) } )
            .launch()
            .await?;

        handle.wait_for_node_exit().await
    })
}

The above <50 LoC snippet encapsulates defining and installing an ExEx. It is extremely powerful and allows extending your Ethereum node’s functionality with zero additional pieces of infrastructure.

Let’s walk through some examples now.

Hello ExEx!

The “Hello World” of Execution Extensions is a reorg tracker. The ExEx shown in the screenshot below illustrates logging whether there was a new chain or a reorganization. One could build a reorg tracker on top of their Reth node easily just by parsing the info logs emitted by the below ExEx.

In this example, the old and new chains have full access to every state change in that range of blocks, along with the trie updates and other useful information in the Chain struct.

async fn exex<Node: FullNodeComponents>(mut ctx: ExExContext<Node>) -> eyre::Result<()> {
    while let Some(notification) = ctx.notifications.recv().await {
        match &notification {
            ExExNotification::ChainCommitted { new } => {
                info!(committed_chain = ?new.range(), "Received commit");
            }
            ExExNotification::ChainReorged { old, new } => {
                info!(from_chain = ?old.range(), to_chain = ?new.range(), "Received reorg");
            }
            ExExNotification::ChainReverted { old } => {
                info!(reverted_chain = ?old.range(), "Received revert");
            }
        };

        if let Some(committed_chain) = notification.committed_chain() {
            ctx.events.send(ExExEvent::FinishedHeight(committed_chain.tip().number))?;
        }
    }
    Ok(())
}

Building an indexer for the OP Stack using ExEx

Now that we have the basics of hooking on node events, let's build a more elaborate example, such as an indexer for deposits and withdrawals in common OP Stack chains, using SQlite as the backend.

In this case:

1. We have loaded the OP Stack's bridge contract using Alloy's sol! macro to generate type-safe ABI decoders (this is an extremely powerful macro that we encourage developers to dive deeper in).
2. We initialize the SQLite connection and set up the database tables.
3. On each ExExNotification we proceed to read the logs for every committed block, decode it, and then insert it into SQLite.
4. If the ExExNotification is for a chain reorganization, then we remove the corresponding entries from the SQLite tables.

That's it! Super simple, and probably the highest performance locally hosted real-time indexer you can build in 30 minutes. See the code below, and go through the full example.

use alloy_sol_types::{sol, SolEventInterface};
use futures::Future;
use reth_exex::{ExExContext, ExExEvent};
use reth_node_api::FullNodeComponents;
use reth_node_ethereum::EthereumNode;
use reth_primitives::{Log, SealedBlockWithSenders, TransactionSigned};
use reth_provider::Chain;
use reth_tracing::tracing::info;
use rusqlite::Connection;

sol!(L1StandardBridge, "l1_standard_bridge_abi.json");
use crate::L1StandardBridge::{ETHBridgeFinalized, ETHBridgeInitiated, L1StandardBridgeEvents};

fn create_tables(connection: &mut Connection) -> rusqlite::Result<()> {
    connection.execute(
        r#"
            CREATE TABLE IF NOT EXISTS deposits (
                id               INTEGER PRIMARY KEY,
                block_number     INTEGER NOT NULL,
                tx_hash          TEXT NOT NULL UNIQUE,
                contract_address TEXT NOT NULL,
                "from"           TEXT NOT NULL,
                "to"             TEXT NOT NULL,
                amount           TEXT NOT NULL
            );
            "#,
        (),
    )?;
    // .. rest of db initialization

    Ok(())
}

/// An example of ExEx that listens to ETH bridging events from OP Stack chains
/// and stores deposits and withdrawals in a SQLite database.
async fn op_bridge_exex<Node: FullNodeComponents>(
    mut ctx: ExExContext<Node>,
    connection: Connection,
) -> eyre::Result<()> {
    // Process all new chain state notifications
    while let Some(notification) = ctx.notifications.recv().await {
        // Revert all deposits and withdrawals
        if let Some(reverted_chain) = notification.reverted_chain() {
            // ..
        }

        // Insert all new deposits and withdrawals
        if let Some(committed_chain) = notification.committed_chain() {
            // ..
        }
    }

    Ok(())
}

/// Decode chain of blocks into a flattened list of receipt logs, and filter only
/// [L1StandardBridgeEvents].
fn decode_chain_into_events(
    chain: &Chain,
) -> impl Iterator<Item = (&SealedBlockWithSenders, &TransactionSigned, &Log, L1StandardBridgeEvents)>
{
    chain
        // Get all blocks and receipts
        .blocks_and_receipts()
        // .. proceed with decoding them
}

fn main() -> eyre::Result<()> {
    reth::cli::Cli::parse_args().run(|builder, _| async move {
        let handle = builder
            .node(EthereumNode::default())
            .install_exex("OPBridge", |ctx| async move {
                let connection = Connection::open("op_bridge.db")?;
              	create_tables(&mut connection)?;
                Ok(op_bridge_exex(ctx, connection))
            })
            .launch()
            .await?;

        handle.wait_for_node_exit().await
    })
}

Building a Rollup using ExEx

Let's do something more interesting now, and build a minimal Rollup as an ExEx, with an EVM runtime and SQLite as the backend!

If you zoom out, even Rollups are ETL-ish pipelines:

1. Extract the data posted on L1 and convert to an L2 payload (e.g. OP Stack derivation function).
2. Run the state transition function (e.g. EVM).
3. Write the updated state to a persistent storage.

In this example, we demonstrate a simplified rollup that derives its state from RLP encoded EVM transactions posted to Zenith (a Holesky smart contract for posting our rollup's block commitments) driven by a simple block builder, both built by the init4 team.

The example specifically:

1. Configures an EVM & instantiates an SQLite database and implements the required revm Database traits to use SQLite as an EVM backend.
2. Filters transactions sent to the deployed rollup contract, ABI decodes the calldata, then RLP decodes that into a Rollup block which gets executed by the configured EVM.
3. Inserts the results of the EVM execution to SQLite.

Again, super simple. It also works with blobs!

ExEx Rollups are extremely powerful because we can now run any number of rollups on Reth without additional infrastructure, by installing them as ExExes.

We are working on extending the example with blobs, and providing a built-in sequencer, for a more complete end to end demo. Reach out if this is something you'd like to build, as we think this has potential for introducing L2 PBS, decentralized / shared sequencers or even SGX-based sequencers and more.

Example snippets below.

use alloy_rlp::Decodable;
use alloy_sol_types::{sol, SolEventInterface, SolInterface};
use db::Database;
use eyre::OptionExt;
use once_cell::sync::Lazy;
use reth_exex::{ExExContext, ExExEvent};
use reth_interfaces::executor::BlockValidationError;
use reth_node_api::{ConfigureEvm, ConfigureEvmEnv, FullNodeComponents};
use reth_node_ethereum::{EthEvmConfig, EthereumNode};
use reth_primitives::{
    address, constants,
    revm::env::fill_tx_env,
    revm_primitives::{CfgEnvWithHandlerCfg, EVMError, ExecutionResult, ResultAndState},
    Address, Block, BlockWithSenders, Bytes, ChainSpec, ChainSpecBuilder, Genesis, Hardfork,
    Header, Receipt, SealedBlockWithSenders, TransactionSigned, U256,
};
use reth_provider::Chain;
use reth_revm::{
    db::{states::bundle_state::BundleRetention, BundleState},
    DatabaseCommit, StateBuilder,
};
use reth_tracing::tracing::{debug, error, info};
use rusqlite::Connection;
use std::sync::Arc;

mod db;

sol!(RollupContract, "rollup_abi.json");
use RollupContrac:{RollupContractCalls, RollupContractEvents};

const DATABASE_PATH: &str = "rollup.db";
const ROLLUP_CONTRACT_ADDRESS: Address = address!("74ae65DF20cB0e3BF8c022051d0Cdd79cc60890C");
const ROLLUP_SUBMITTER_ADDRESS: Address = address!("B01042Db06b04d3677564222010DF5Bd09C5A947");
const CHAIN_ID: u64 = 17001;
static CHAIN_SPEC: Lazy<Arc<ChainSpec>> = Lazy::new(|| {
    Arc::new(
        ChainSpecBuilder::default()
            .chain(CHAIN_ID.into())
            .genesis(Genesis::clique_genesis(CHAIN_ID, ROLLUP_SUBMITTER_ADDRESS))
            .shanghai_activated()
            .build(),
    )
});

struct Rollup<Node: FullNodeComponents> {
    ctx: ExExContext<Node>,
    db: Database,
}

impl<Node: FullNodeComponents> Rollup<Node> {
    fn new(ctx: ExExContext<Node>, connection: Connection) -> eyre::Result<Self> {
        let db = Database::new(connection)?;
        Ok(Self { ctx, db })
    }

    async fn start(mut self) -> eyre::Result<()> {
        // Process all new chain state notifications
        while let Some(notification) = self.ctx.notifications.recv().await {
            if let Some(reverted_chain) = notification.reverted_chain() {
                self.revert(&reverted_chain)?;
            }

            if let Some(committed_chain) = notification.committed_chain() {
                self.commit(&committed_chain)?;
                self.ctx.events.send(ExExEvent::FinishedHeight(committed_chain.tip().number))?;
            }
        }

        Ok(())
    }

    /// Process a new chain commit.
    ///
    /// This function decodes all transactions to the rollup contract into events, executes the
    /// corresponding actions and inserts the results into the database.
    fn commit(&mut self, chain: &Chain) -> eyre::Result<()> {
        let events = decode_chain_into_rollup_events(chain);

        for (_, tx, event) in events {
            match event {
                // A new block is submitted to the rollup contract.
                // The block is executed on top of existing rollup state and committed into the
                // database.
                RollupContractEvents::BlockSubmitted(_) => {
                    // ..
                }
                // A deposit of ETH to the rollup contract. The deposit is added to the recipient's
                // balance and committed into the database.
                RollupContractEvents::Enter(RollupContract::Enter {
                    token,
                    rollupRecipient,
                    amount,
                }) => {
                    // ..
                _ => (),
            }
        }

        Ok(())
    }

    /// Process a chain revert.
    ///
    /// This function decodes all transactions to the rollup contract into events, reverts the
    /// corresponding actions and updates the database.
    fn revert(&mut self, chain: &Chain) -> eyre::Result<()> {
        let mut events = decode_chain_into_rollup_events(chain);
        // Reverse the order of events to start reverting from the tip
        events.reverse();

        for (_, tx, event) in events {
            match event {
                // The block is reverted from the database.
                RollupContractEvents::BlockSubmitted(_) => {
                    // ..
                }
                // The deposit is subtracted from the recipient's balance.
                RollupContractEvents::Enter(RollupContract::Enter {
                    token,
                    rollupRecipient,
                    amount,
                }) => {
                    // ..
                }
                _ => (),
            }
        }

        Ok(())
    }
}

fn main() -> eyre::Result<()> {
    reth::cli::Cli::parse_args().run(|builder, _| async move {
        let handle = builder
            .node(EthereumNode::default())
            .install_exex("Rollup", move |ctx| async {
                let connection = Connection::open(DATABASE_PATH)?;
                Ok(Rollup::new(ctx, connection)?.start())
            })
            .launch()
            .await?;

        handle.wait_for_node_exit().await
    })
}

What can I build with Execution Extensions?

This question can be reframed as “What can be modeled as a post-execution hook?”. Turns out a lot of things!

We see a few valuable ExExes that should be built:

1. Rollup derivation pipelines such as Kona, with EVMs configured for L2 usage, similar to how Reth Alphanet’s EVM is set up. We also predict that composing L2 ExExes will provide the fastest path towards Stage 2 Rollup decentralization. Expect more from us here. We cannot wait to run OP Mainnet, Base, Zora, and other rollups on Reth as ExExes.
2. Out of process ExExes tightly integrated with the node’s services using gRPC, paving the way for multi-tenancy and Reth acting as a control plane for off-chain services.
3. Alternative VM integrations (e.g. MoveVM or Arbitrum Stylus), or complex execution pipelines similar to Artemis, SGX Revm or Shadow Logs.
4. Foundational infrastructure combined with re-staking such as oracles & bridges, or any other Actively Validated Services (AVS). This is possible because ExExes can peer with each other over Ethereum’s DiscV5 P2P network and have an elected set of participants with write permission to their state other than the node’s 'finalized' notifications.
5. Next-generation auxiliary infrastructure such as AI coprocessors, or decentralized/shared sequencers.

What’s next for Execution Extensions?

Currently, ExExes need to be installed on the node with a custom build in your main function. We aspire to make ExExes dynamically loaded as plugins, and expose a Docker Hub-esque reth pull API, such that developers can distribute their ExExes over the air to node operators easily.

We want to make Reth a platform that provides stability & performance on core node operations, while also being a launchpad for innovation.

The Reth project is hopefully going to change how people think about building high performance off-chain infra, and ExExes are just the beginning. We are excited to continue building infrastructure on Reth, and invest in it.

Reach out to georgios@paradigm.xyz if you want to build an ExEx project, or contribute directly on Github.