đź“ť Onboarding to Kepler (Restoring Dictionary Projects)

Indexers can start preparing for Kepler

We are sharing the docs to help indexers start to prepare for Kepler by providing database snapshots of the first few dictionary projects. Indexers may wish to download and prepare these files in advance of our launch date on the 1st of May.

You may want to start the following processes:

Note, restoring a dictionary project may take around 2~4 days due to the number of constraints and indexes that need to be updated, so start this process when you can.

Indexers don’t need to restore all the projects, it depends on the resource indexers have. However, indexing more projects will have more chance to be pick by our sponsor plan.

Indexer Services

Please make sure you are running the correct indexer-services:

onfinality/subql-coordinator	v1.0.4
onfinality/subql-indexer-proxy	v1.0.0

About the DB restore

The DB restore has 2 stage:

  1. restore tables
  2. rebuild the index for the tables

The stage 1 is running in multi thread which will cost more cpu. You will see pg_restore threads in the result list of top cmd

The stage 2 cost less cpu but takes the most of the time for restoring. You will find postgres thread in the top list, this thread will be disappeared once the whole restore complete.

The whole db restore will takes around 2-4 days depends on your machine. Especially when you have more than one project running at the same time, it will take longer time. Recommend to wait until the restore finish to start indexing these projects.

6 Likes

I’ve used SubQuery’s Pre-Built Image in AWS with default settings. When I try to

sudo docker-compose up -d

I see one warn “Found orphan containers ([coordinator_proxy coordinator_service]) for this project”

and error - “Error response from daemon: driver failed programming external connectivity on endpoint indexer_coordinator (xxx): Bind for 0.0.0.0:8000 failed: port is already allocated”

The AWS image hasn’t been updated in some time. We suggest following our more recent guides above

I suggest to update docs before the Kepler start, or at least make one page instruction, because pages of the current documentation link to each other and there are conflicting paragraphs, although all pages have an update date of April 28, 2023 and people might think that this is actual information.

UPD: recent guides above leads to page about Pre-Built AWS Image

1 Like

I have re-note the SubQuery’s guide here, you can refer to it for more details, some steps have been customized for Linux: GitHub - duchuytb9x/NodeSubQueryKepler

2 Likes

Thank you! (You have “Extract Moonbeam Snapshot” several times in Readme.md)

What specific error did you get, please pay attention to the name of the snapshot

We have some issues with caching docs that are outdated and my be causing some confusion. Will try to resolve tomorrow

1 Like

is it necessary to change “secret-key” in docker-compose.yml? The docs only say “POSTGRES_PASSWORD” and “postgres-password”. I already started everything, but the guys in Discord write that the secret key also needs to be changed.

no answer to this question?

How to Become an Indexer | SubQuery Academy (Documentation) please refer to this part, all the keys mention in the documents need to set to your own value, then follow the instruction here Upgrade indexer services to restart the specific container needs to restart

Please also review ERROR Failed to index block 3433175: TypeError: Cannot read properties of undefined (reading 'transformer') - #3 by james.bayly regarding Moonbeam

1 Like

why not do table indexing first. then copy data.