Stateless Block Functions
Stateless block functions lie at the core of Project Zero’s data transformation process, enabling protocol-specific customization and real-time enrichment of raw blockchain data. These functions are lightweight, modular components designed to process individual blocks independently, without requiring historical context or maintaining state. This stateless design ensures scalability and simplifies the implementation of new protocols, as each blockchain's unique data structures and logic can be defined through dedicated, customizable functions. To simply put, the block functions are javascript functions which are easy to define for filtering and transforming raw blockchain data on the fly into required custom needs and formats reducing multiple data pipeline overheads against contemporary solutions.
Block functions are executed in isolated environment, which provides access to such third party libraries as lodash and web3-eth-abi. lodash is accessible through the global variable _, while web3-eth-abi is accessible through ABI variable. Each block function should implement synchronous method main with two input parameters - block and blockMetadata (second one is not mandatory). First parameter contains raw block data (it's structure depends on selected dataset/blockchain), while second parameter have strict structure and contain two properties: number (block number) and hash (block hash).
Note
main function gets a separate block even if the batch size is bigger than 1, so it will always work in the context of the single block. To maintain acceptable performance, there is a pool of isolated filters that can process the blocks in parallel and guarantee data consistency. After filtering, the data will be processed by a batch former that creates batches according to the user's configurations and passes the data to the sender, which delivers the data to its destination. Here a simplified schema of the filtering process:
Value returned from main method will be delivered to the selected destination instead of raw block data. Block functions have executions limitations, such as execution time per block (30 seconds) and memory limit (128MB), you will get an appropriate error message if you reach these limitations.
In other words, block function works as and additional layer on top of regular stream, it takes the raw data and performs modifications provided by the user (data enrichment, filtering unnecessary data, data aggregation, decoding of the raw data etc.) and sends it to the client instead of the raw data. At the moment only javascript is supported, but in the future we have plans to add other languages. Also we have plans to allow async function execution, so users will be able to call some third party API to enrich blocks data.