Subquery Indexer Toolkit (aka Swiss Kinfe)
Objective of the project:
Provide a reliable and convenient environment for indexer owners and the opportunity to learn about the problem in a timely manner
What is it, what does it consist of?
The tool contains everything you need to index, monitor and properly access for you from any device
The solution is based on
- Postgres
- Subquerynetwork/indexer-coordinator
- Redis
- Subquerynetwork/indexer-proxy
- Trafik
- Grafana
- Node exporter
- Additional friendly exporters
- Wireguard
Is it opensource? Can i check?
Yep. The solution was created by indexers for indexers in order to improve the quality of work of indexers
https://github.com/web3cdnservices/subquery-indexer-toolkit/blob/master/README.md
What will I get after installation?
- Fully prepared server for indexing with featured metrics
- Secure access yo your coordinator panel via any device (pc,phone,etc)
- Secure access to monitoring & Alerting
How it will looklike?
-
Generating keys and passwords.
-
Instalation process
(All data will be saved to SubqueryIndexerHandbook.tar.gz, So, download it. Here - passwords and all required keys)
After setup, web server automatically will generate SSL cert for you subdomain and server 443 and 80 ports
Monitoring
Subquery toolkit dashboard
Traefik
Postgres
Node_exporter
Infrastructure description
By default Wireguard mask 10.253.1.0/24, but you can set you own in .env file Internal network mask: 172.29.13.0/24 All addresses and services accessible via private network!
| Service | Address | Annotation |
|---|---|---|
| Postgres | 172.29.13.2 | |
| Redis | 172.29.13.4 | |
| Indexer Coordinator | http://172.29.13.3:8000 | |
| Indexer Proxy | http://172.29.13.5:8375 | |
| Traefik | 172.29.13.6 | ports 8082(metrics), 80(public), 443(public with Letsencrypt) |
| Prometheus | http://172.29.13.8:9090 | |
| Node Exporter | http://172.29.13.9:9100 | |
| Grafana | http://172.29.13.10:3000 |
You can receive access to subnet via wireguard configuration from SubqueryIndexerHandbook.tar.gz
Wireguard use by default 816 port. But you can set it manually in .env file. This file NEVER automatically overwrites, Script preventing override.
Installation
- Prepare system
Debian/Ubuntu/Arch/Kali any apt-get based
Run command:
sudo apt update && sudo apt install -y docker-compose docker.io git curl bzip2
Centos/Fedora/Redhat any yum based
Run command
sudo yum install -y yum-utils git curl bzip2
sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo yum install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
- Cloning package
cd /opt
git clone https://github.com/web3cdnservices/subquery-indexer-toolkit.git
cd subquery-indexer-toolkit
- Generating keys
Run command
sudo bash ./tools/generate_initial_configuration
Command will request domain name and your email. Please paste only domain without protocol. Example: indexerer.myprofessionaldomain.com and make shure, you linked IP address to this domain
- Install Indexer and monitoring software
Run command
sudo bash ./tools/start_indexer_with_wireguard
After installation will complete you will have fully completed ecosystem with VPN server, monitoring, and indexer software. Please download from server file: SubqueryIndexerHandbook.tar.gz. It will contains All passwords, keys and addresses.
Upgrading
- Upgrade from repo
sudo git pull
- Run upgrading process
sudo bash ./tools/start_indexer_with_wireguard
Connection to server with wireguard
LInux users
1.Install wireguard client
- Ubuntu/Debian/Arch/Kali
sudo apt install wireguard-tools wireguard
- Centos/Fedora/Redhat
sudo yum install epel-release
sudo yum install 'dnf-command(config-manager)'
sudo yum config-manager --set-enabled PowerTools
sudo yum copr enable jdoss/wireguard
sudo yum install wireguard-dkms wireguard-tools
- Windows
Install WireGuard for Windows Downloads
- Macbook
Install WireGuard on the Mac App Store
- Iphone
Install WireGuard on the App Store
- Android
2 Configure vpn client
Open downloaded file SubqueryIndexerHandbook.tar.gz
Linux any dist
-
Open wireguard folder from archive. and copy client1.conf OR client2.conf to /etc/wireguard/ folder. It will looklike /etc/wireguard/client1.conf
-
Execute command
sudo systemctl enable wg-quick@client1
sudo systemctl start wg-quick@client1
Please change client1 to client2 if you want to use on this pc client2 profile.
MacOS
- Using GUI import content of client1.conf and run.(Import tunnel from file)
Windows
- Using GUI import content of client1.conf and run.(Import tunnel from file)
Iphone
- Open on PC archive, go to wireguard folder and open client2.png. That’s image. QR code. Then - open wireguard on phone on choose option. Scan from QR code.
Android
- Open on PC archive, go to wireguard folder and open client2.png. That’s image. QR code. Then - open wireguard on phone on choose option. Scan from QR code.
- Completing
- Go to
http://172.29.13.3:8000 and configure your indexer via GUI. (That’s coordinator internal GUI) - Go to http://172.29.13.10:3000 (Grafana monitoring) for checking health.
Ecosystem
Questions:
I have own cert, i don’t want to use free OR i want to use nginx as proxy with self configured ssl
Yes, you can use own proxy server for that cofiguration you need to use another run script
sudo bash ./tools/start_indexer_with_wireguard_without_traefik
Then, configure your nginx proxy to host http://172.29.13.5:8375
Docs: NGINX Reverse Proxy | NGINX Documentation
TODO
- Docs and scrips for migration from official Subquery scripts
- Docs for Postgres clustering/Migration
- Auto-resolving problems + Load balancing & reservation for nodes
- CLI interface for indexer management