> For the complete documentation index, see [llms.txt](https://docs.thehemera.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.thehemera.com/hemera-indexer/installation/install-and-run.md). # Install & Run Follow these steps to get started and run the Hemera Indexer: {% embed url="" %} ## Clone the Repository ``` git clone https://github.com/HemeraProtocol/hemera-indexer.git ``` ## Building and Running the Hemera Indexer To operate the Hemera Indexer, you essentially have two approaches to choose from, each suited to different levels of customization and control. **1. Use Docker** **2.Build from Source code** ## Run from Docker #### **1. Install Docker & Docker Compose** If you have trouble running the following commands, consider referring to the [official docker installation guide](https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository) for the latest instructions. ``` # Add Docker's official GPG key: sudo apt-get update sudo apt-get install ca-certificates curl sudo install -m 0755 -d /etc/apt/keyrings sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc sudo chmod a+r /etc/apt/keyrings/docker.asc # Add the repository to Apt sources: echo \ "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \ $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \ sudo tee /etc/apt/sources.list.d/docker.list > /dev/null sudo apt-get update ``` ``` # Install docker and docker compose sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin ``` #### **2. Run Docker Compose** ``` cd hemera_indexer cd docker-compose ``` Alternatively, you might want to edit environment variables in `docker-compose.yaml`. Please check out the [configuration manual](/hemera-indexer/configurations.md) on how to configure the environment variables. ``` vim docker-compose.yaml ``` Now, run the following command to spin up the containers. ``` sudo docker compose up ``` You should be able to see similar logs from your console that indicate Hemera Indexer is running properly. ``` [+] Running 2/0 ✔ Container postgresql Created 0.0s ✔ Container hemera Created 0.0s Attaching to hemera, postgresql postgresql | postgresql | PostgreSQL Database directory appears to contain a database; Skipping initialization postgresql | postgresql | 2024-06-24 08:18:48.547 UTC [1] LOG: starting PostgreSQL 15.7 (Debian 15.7-1.pgdg120+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit postgresql | 2024-06-24 08:18:48.548 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432 postgresql | 2024-06-24 08:18:48.549 UTC [1] LOG: listening on IPv6 address "::", port 5432 postgresql | 2024-06-24 08:18:48.554 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" postgresql | 2024-06-24 08:18:48.564 UTC [27] LOG: database system was shut down at 2024-06-24 06:44:23 UTC postgresql | 2024-06-24 08:18:48.575 UTC [1] LOG: database system is ready to accept connections hemera | 2024-06-24 08:18:54,953 - root [INFO] - Using provider https://eth.llamarpc.com hemera | 2024-06-24 08:18:54,953 - root [INFO] - Using debug provider https://eth.llamarpc.com hemera | 2024-06-24 08:18:55,229 - alembic.runtime.migration [INFO] - Context impl PostgresqlImpl. hemera | 2024-06-24 08:18:55,230 - alembic.runtime.migration [INFO] - Will assume transactional DDL. hemera | 2024-06-24 08:18:55,278 - alembic.runtime.migration [INFO] - Context impl PostgresqlImpl. hemera | 2024-06-24 08:18:55,278 - alembic.runtime.migration [INFO] - Will assume transactional DDL. hemera | 2024-06-24 08:18:56,169 - root [INFO] - Current block 20160303, target block 20137200, last synced block 20137199, blocks to sync 1 hemera | 2024-06-24 08:18:56,170 - ProgressLogger [INFO] - Started work. Items to process: 1. hemera | 2024-06-24 08:18:57,505 - ProgressLogger [INFO] - 1 items processed. Progress is 100%. hemera | 2024-06-24 08:18:57,506 - ProgressLogger [INFO] - Finished work. Total items processed: 1. Took 0:00:01.336310. hemera | 2024-06-24 08:18:57,529 - exporters.postgres_item_exporter [INFO] - Exporting items to table block_ts_mapper, blocks end, Item count: 2, Took 0:00:00.022562 hemera | 2024-06-24 08:18:57,530 - ProgressLogger [INFO] - Started work. ``` ## Run From Source Code #### **1. Install Python3 and pip** Skip this step if you already have both installed. ``` sudo apt update sudo apt install python3 sudo apt install python3-pip ``` #### **2. Initiate Python VENV** Skip this step if you don't want to have a dedicated python venv for Hemera Indexer. ``` sudo apt install python3-venv python3 -m venv ./venv ``` #### **3. Install pip Dependencies** ``` source ./venv/bin/activate sudo apt install libpq-dev pip install -e . ``` {% hint style="info" %} #### Running using Make script Another option is to set up and build from source using the make script You can use the command: `make development` {% endhint %} #### **Prepare Your PostgreSQL Instance** Hemera Indexer requires a PostgreSQL database to store all indexed data. You may skip this step if you already have a PostgreSQL set up. #### **5. Setup PostgreSQL** Follow the instructions about how to set up a PostgreSQL database here: [Setup PostgreSQL on Ubuntu](https://www.cherryservers.com/blog/how-to-install-and-setup-postgresql-server-on-ubuntu-20-04). #### **6. Configure** Configure the `OUTPUT` or `--output` parameter according to your PostgreSQL role information. Check out [Configure Hemera Indexer](/hemera-indexer/configurations.md) for details. E.g. `postgresql+psycopg2://${YOUR_USER}:${YOUR_PASSWORD}@${YOUR_HOST}:5432/${YOUR_DATABASE}`. #### **7. Run** Please check out [Configure Hemera Indexer](/hemera-indexer/configurations.md) on how to configure the indexer. ``` python hemera.py stream \ --provider-uri https://eth.llamarpc.com \ --debug-provider-uri https://eth.llamarpc.com \ --postgres-url postgresql+psycopg2://devuser:devpassword@localhost:5432/hemera_indexer \ --output jsonfile://output/eth_blocks_20000001_20010000/json,csvfile://output/hemera_indexer/csv,postgresql+psycopg2://devuser:devpassword@localhost:5432/eth_blocks_20000001_20010000 \ --start-block 20000001 \ --end-block 20010000 \ --output-types block,transaction,log \ --block-batch-size 200 \ --batch-size 50 \ --max-workers 8 ``` Once you have successfully bootstrapped Hemera Indexer, you should be able to view similar logs as below. ``` 2024-06-25 16:37:38,456 - root [INFO] - Using provider https://eth.llamarpc.com 2024-06-25 16:37:38,456 - root [INFO] - Using debug provider https://eth.llamarpc.com 2024-06-25 16:37:38,485 - alembic.runtime.migration [INFO] - Context impl PostgresqlImpl. 2024-06-25 16:37:38,485 - alembic.runtime.migration [INFO] - Will assume transactional DDL. 2024-06-25 16:37:38,502 - alembic.runtime.migration [INFO] - Context impl PostgresqlImpl. 2024-06-25 16:37:38,502 - alembic.runtime.migration [INFO] - Will assume transactional DDL. 2024-06-25 16:37:39,485 - root [INFO] - Current block 20167548, target block 20137200, last synced block 20137199, blocks to sync 1 2024-06-25 16:37:39,486 - ProgressLogger [INFO] - Started work. Items to process: 1. 2024-06-25 16:37:40,267 - ProgressLogger [INFO] - 1 items processed. Progress is 100%. 2024-06-25 16:37:40,268 - ProgressLogger [INFO] - Finished work. Total items processed: 1. Took 0:00:00.782177. 2024-06-25 16:37:40,283 - exporters.postgres_item_exporter [INFO] - Exporting items to table block_ts_mapper, blocks end, Item count: 2, Took 0:00:00.014799 2024-06-25 16:37:40,283 - ProgressLogger [INFO] - Started work. ```