Is this repo still active? And does it work for anyone?

[anizeani]

I've tried to run the 3 docker options (quick, public and full) for a week now, but unfortunately without any success.

1. question: did anyone succeed in the last few weeks to display data in the UI?

For me, the UI starts, but no data visible.
For:
docker-compose -p kusama -f docker-compose.kusama-quick.yml up --build

At localhost:8080 in the inspector I see on my PC (windows):
<GET https://explorer-31.polkascan.io/api/v1/kusama/balances/transfer net::ERR_CONNECTION_TIMED_OUT>
same timeout error for networkstats/latest and /block

2. question: is the API https://explorer-31.polkascan.io/api/v1/kusama working? And is there a documentation for the API?

on OSX I get:
Access to XMLHttpRequest at 'http://127.0.0.1:8080/api/v1/networkstats/latest' from origin 'http://localhost:8080' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: It does not have HTTP ok status.

if anyone knows how to solve this issue I would be pleased to know^^

At localhost:8081 in the inspector i see:
react_devtools_backend.js: RPC-CORE: getMetadata(at?: BlockHash): Metadata:: createType(Metadata):: Unable to create Enum via index 13, in V0, V1, V2, V3, V4, V5, V6, V7, V8, V9, V10, V11
3. question: Does this mean, that my docker container doesn't run the up-to date python files, because V12 is implemented in the python files of the repo?

For:
docker-compose -p kusama -f docker-compose.kusama-public.yml up --build

I see on localhost:8080 the same CORS error but on my PC
(all containers are up and running)
On the harvester monitor I see a failure for: app.tasks.start_sequencer, app.tasks.accumulate_block_recursive (args: ('0xbc1afbbec4d7d28f267ee2aa5b1f015548d81d59d3e0c32e4286f1b138f4e667', None) )

in the log of docker I see:
harvester-worker_1 | [2021-06-09 12:24:49,073: WARNING/ForkPoolWorker-2] Metadata: CACHE MISS
harvester-worker_1 | [2021-06-09 12:24:49,073: WARNING/ForkPoolWorker-2] 9040
harvester-worker_1 | [2021-06-09 12:24:49,159: WARNING/ForkPoolWorker-2] store version to db
harvester-worker_1 | [2021-06-09 12:24:49,159: WARNING/ForkPoolWorker-2] MetadataV13Decoder
harvester-worker_1 | [2021-06-09 12:24:49,504: INFO/ForkPoolWorker-8] Task app.tasks.start_sequencer[3bd9e224-c707-4e46-be87-132da8b70327] succeeded in 1.6601815999893006s: {'result': 'Chain not at genesis'}
harvester-worker_1 | [2021-06-09 12:24:53,817: WARNING/ForkPoolWorker-2] ! ERROR adding 0x842598b55b65e8aa30eaf89183ef0aca2c1c0438633cdd8b23d1d8fb9adbbf91
harvester-worker_1 | [2021-06-09 12:24:53,818: ERROR/ForkPoolWorker-2] Task app.tasks.accumulate_block_recursive[753031eb-56de-4924-89e8-5883e9d76800] raised unexpected: HarvesterCouldNotAddBlock('0x842598b55b65e8aa30eaf89183ef0aca2c1c0438633cdd8b23d1d8fb9adbbf91')

For:
docker-compose -p kusama -f docker-compose.kusama-full.yml up --build
same error as the one from public.

My ultimate goal is to run the docker-compose.substrate-node-template.yml for my own node.
If I understood correctly, I would only have to specify the template app/type_registry/custom_types.json and the endpoint of my nodes then the MySQL database should be able to store the extrinsics of my nodes and I should have the same behaviour as the polkascan.io view offers, right?

Sidenote1: I saw, that in the docker-compose files Substrate_Metadata_Version=9, shouldn't it be =12?
Sidenote2: the command <git submodule update --init --recursive> didn't update the submodules to the lastest version, so I checked out master in each of them.
Sidenote3: I am aware that this are many questions (sry for that). Should I split them into different issues?

[arjanz]

Thanks for reporting, the opensource version was a bit off the radar recently to be honest. Most of the time the cause is a missing version bump from the dependencies of libraries we are actively maintaining (like scalecodec and substrate-interface).

I updated the requirements.txt files and docker-compose configs, so it should work now.

[anizeani]

Great! Thank you very much!!
Just as a hint: if someone follows the readme and git checkout the latest tag (v0.6.3), its still outdated with the old api (explorer-31), so maybe set a new tag or recommend to checkout master. Thank you very much @arjanz

[anizeani]

So the quick deployment works.

But for the public Substrate RPC I still get errors. I can display Finalized block, transactions, module events, active accounts, runtime versions.
But I can't get Last blocks, last balance transfers.

At 8080 I still get the CORS error:
kusama:1 Access to XMLHttpRequest at 'http://127.0.0.1:8080/api/v1/block' from origin 'http://localhost:8080' has been blocked by CORS policy: Request header field content-type is not allowed by Access-Control-Allow-Headers in preflight response.

At 8081 I still get the:
RPC-CORE: getMetadata(at?: BlockHash): Metadata:: createType(Metadata):: Unable to create Enum via index 13, in V0, V1, V2, V3, V4, V5, V6, V7, V8, V9, V10, V11

In the docker logs I get the following error:
in short:
harvester-worker_1 | sqlalchemy.exc.ProgrammingError:
...
mysql_1: mbind: Operation not permitted
..
harvester-worker_1 | File "/usr/src/app/app/tasks.py", line 147, in accumulate_block_recursive
harvester-worker_1 | raise HarvesterCouldNotAddBlock(block_hash) from exc
harvester-worker_1 | app.processors.converters.HarvesterCouldNotAddBlock: 0x589484cfbbb3c5aa159fd33eb86d65b28752cc377b62df41216fe86ccbd33fb0
harvester-monitor_1 | Error:
harvester-monitor_1 | Unable to load celery application.
harvester-monitor_1 | The module scalecodec was not found.

The complete logs dump:

harvester-worker_1 | sqlalchemy.exc.ProgrammingError: (mysql.connector.errors.ProgrammingError) 1146 (42S02): Table 'polkascan.harvester_status' doesn't exist
harvester-worker_1 | [SQL: SELECT harvester_status.key AS harvester_status_key, harvester_status.value AS harvester_status_value, harvester_status.last_modified AS harvester_status_last_modified, harvester_status.notes AS harvester_status_notes
harvester-worker_1 | FROM harvester_status
harvester-worker_1 | WHERE harvester_status.key = %(key_1)s
harvester-worker_1 | LIMIT %(param_1)s]
harvester-worker_1 | [parameters: {'key_1': 'SEQUENCER_TASK_ID', 'param_1': 1}]
harvester-worker_1 | (Background on this error at: http://sqlalche.me/e/f405)
harvester-worker_1 | [2021-06-09 15:41:04,052: INFO/ForkPoolWorker-2] Task app.tasks.start_harvester[ad6a6084-5547-4edd-a6ac-646ff3a18e23] succeeded in 0.21016909999889322s: {'result': 'Harvester job started', 'block_sets': [{'start_block_hash': '0x589484cfbbb3c5aa159fd33eb86d65b28752cc377b62df41216fe86ccbd33fb0', 'end_block_hash': None}], 'sequencer_task_id': '7f423695-bec5-4f19-93cc-83a191486d42'}
harvester-worker_1 | [2021-06-09 15:41:04,052: INFO/MainProcess] Received task: app.tasks.accumulate_block_recursive[66a53688-80cd-430b-9d28-b2c15fd1910f]
mysql_1 | mbind: Operation not permitted
harvester-worker_1 | [2021-06-09 15:41:04,540: WARNING/ForkPoolWorker-6] ! ERROR adding 0x589484cfbbb3c5aa159fd33eb86d65b28752cc377b62df41216fe86ccbd33fb0
harvester-worker_1 | [2021-06-09 15:41:04,542: ERROR/ForkPoolWorker-6] Task app.tasks.accumulate_block_recursive[66a53688-80cd-430b-9d28-b2c15fd1910f] raised unexpected: HarvesterCouldNotAddBlock('0x589484cfbbb3c5aa159fd33eb86d65b28752cc377b62df41216fe86ccbd33fb0')
harvester-worker_1 | Traceback (most recent call last):
harvester-worker_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
harvester-worker_1 | self.dialect.do_execute(
harvester-worker_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
harvester-worker_1 | cursor.execute(statement, parameters)
harvester-worker_1 | File "/usr/local/lib/python3.8/site-packages/mysql/connector/cursor.py", line 569, in execute
harvester-worker_1 | self._handle_result(self._connection.cmd_query(stmt))
harvester-worker_1 | File "/usr/local/lib/python3.8/site-packages/mysql/connector/connection.py", line 553, in cmd_query
harvester-worker_1 | result = self._handle_result(self._send_cmd(ServerCmd.QUERY, query))
harvester-worker_1 | File "/usr/local/lib/python3.8/site-packages/mysql/connector/connection.py", line 442, in _handle_result
harvester-worker_1 | raise errors.get_exception(packet)
harvester-worker_1 | mysql.connector.errors.ProgrammingError: 1146 (42S02): Table 'polkascan.data_block' doesn't exist
harvester-worker_1 |
harvester-worker_1 | The above exception was the direct cause of the following exception:
harvester-worker_1 |
harvester-worker_1 | Traceback (most recent call last):
harvester-worker_1 | File "/usr/src/app/app/tasks.py", line 120, in accumulate_block_recursive
harvester-worker_1 | block = harvester.add_block(block_hash)
harvester-worker_1 | File "/usr/src/app/app/processors/converters.py", line 458, in add_block
harvester-worker_1 | if Block.query(self.db_session).filter_by(hash=block_hash).count() > 0:
harvester-worker_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3578, in count
harvester-worker_1 | return self.from_self(col).scalar()
harvester-worker_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3322, in scalar
harvester-worker_1 | ret = self.one()
harvester-worker_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3292, in one
harvester-worker_1 | ret = self.one_or_none()
harvester-worker_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3261, in one_or_none
harvester-worker_1 | ret = list(self)
harvester-worker_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3334, in iter
harvester-worker_1 | return self._execute_and_instances(context)
harvester-worker_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3359, in _execute_and_instances
harvester-worker_1 | result = conn.execute(querycontext.statement, self._params)
harvester-worker_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 988, in execute
harvester-worker_1 | return meth(self, multiparams, params)
harvester-worker_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection
harvester-worker_1 | return connection._execute_clauseelement(self, multiparams, params)
harvester-worker_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1101, in _execute_clauseelement
harvester-worker_1 | ret = self._execute_context(
harvester-worker_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1252, in _execute_context
harvester-worker_1 | self._handle_dbapi_exception(
harvester-worker_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1473, in _handle_dbapi_exception
harvester-worker_1 | util.raise_from_cause(sqlalchemy_exception, exc_info)
harvester-worker_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
harvester-worker_1 | reraise(type(exception), exception, tb=exc_tb, cause=cause)
harvester-worker_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 152, in reraise
harvester-worker_1 | raise value.with_traceback(tb)
harvester-worker_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
harvester-worker_1 | self.dialect.do_execute(
harvester-worker_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
harvester-worker_1 | cursor.execute(statement, parameters)
harvester-worker_1 | File "/usr/local/lib/python3.8/site-packages/mysql/connector/cursor.py", line 569, in execute
harvester-worker_1 | self._handle_result(self._connection.cmd_query(stmt))
harvester-worker_1 | File "/usr/local/lib/python3.8/site-packages/mysql/connector/connection.py", line 553, in cmd_query
harvester-worker_1 | result = self._handle_result(self._send_cmd(ServerCmd.QUERY, query))
harvester-worker_1 | File "/usr/local/lib/python3.8/site-packages/mysql/connector/connection.py", line 442, in _handle_result
harvester-worker_1 | raise errors.get_exception(packet)
harvester-worker_1 | sqlalchemy.exc.ProgrammingError: (mysql.connector.errors.ProgrammingError) 1146 (42S02): Table 'polkascan.data_block' doesn't exist
harvester-worker_1 | [SQL: SELECT count(*) AS count_1
harvester-worker_1 | FROM (SELECT data_block.id AS data_block_id, data_block.parent_id AS data_block_parent_id, data_block.hash AS data_block_hash, data_block.parent_hash AS data_block_parent_hash, data_block.state_root AS data_block_state_root, data_block.extrinsics_root AS data_block_extrinsics_root, data_block.count_extrinsics AS data_block_count_extrinsics, data_block.count_extrinsics_unsigned AS data_block_count_extrinsics_unsigned, data_block.count_extrinsics_signed AS data_block_count_extrinsics_signed, data_block.count_extrinsics_error AS data_block_count_extrinsics_error, data_block.count_extrinsics_success AS data_block_count_extrinsics_success, data_block.count_extrinsics_signedby_address AS data_block_count_extrinsics_signedby_address, data_block.count_extrinsics_signedby_index AS data_block_count_extrinsics_signedby_index, data_block.count_events AS data_block_count_events, data_block.count_events_system AS data_block_count_events_system, data_block.count_events_module AS data_block_count_events_module, data_block.count_events_extrinsic AS data_block_count_events_extrinsic, data_block.count_events_finalization AS data_block_count_events_finalization, data_block.count_accounts AS data_block_count_accounts, data_block.count_accounts_new AS data_block_count_accounts_new, data_block.count_accounts_reaped AS data_block_count_accounts_reaped, data_block.count_sessions_new AS data_block_count_sessions_new, data_block.count_contracts_new AS data_block_count_contracts_new, data_block.count_log AS data_block_count_log, data_block.range10000 AS data_block_range10000, data_block.range100000 AS data_block_range100000, data_block.range1000000 AS data_block_range1000000, data_block.datetime AS data_block_datetime, data_block.year AS data_block_year, data_block.month AS data_block_month, data_block.week AS data_block_week, data_block.day AS data_block_day, data_block.hour AS data_block_hour, data_block.full_month AS data_block_full_month, data_block.full_week AS data_block_full_week, data_block.full_day AS data_block_full_day, data_block.full_hour AS data_block_full_hour, data_block.logs AS data_block_logs, data_block.authority_index AS data_block_authority_index, data_block.slot_number AS data_block_slot_number, data_block.spec_version_id AS data_block_spec_version_id, data_block.debug_info AS data_block_debug_info
harvester-worker_1 | FROM data_block
harvester-worker_1 | WHERE data_block.hash = %(hash_1)s) AS anon_1]
harvester-worker_1 | [parameters: {'hash_1': '0x589484cfbbb3c5aa159fd33eb86d65b28752cc377b62df41216fe86ccbd33fb0'}]
harvester-worker_1 | (Background on this error at: http://sqlalche.me/e/f405)
harvester-worker_1 |
harvester-worker_1 | The above exception was the direct cause of the following exception:
harvester-worker_1 |
harvester-worker_1 | Traceback (most recent call last):
harvester-worker_1 | File "/usr/local/lib/python3.8/site-packages/celery/app/trace.py", line 385, in trace_task
harvester-worker_1 | R = retval = fun(*args, **kwargs)
harvester-worker_1 | File "/usr/src/app/app/tasks.py", line 70, in call
harvester-worker_1 | return super().call(*args, **kwargs)
harvester-worker_1 | File "/usr/local/lib/python3.8/site-packages/celery/app/trace.py", line 648, in protected_call
harvester-worker_1 | return self.run(*args, **kwargs)
harvester-worker_1 | File "/usr/src/app/app/tasks.py", line 147, in accumulate_block_recursive
harvester-worker_1 | raise HarvesterCouldNotAddBlock(block_hash) from exc
harvester-worker_1 | app.processors.converters.HarvesterCouldNotAddBlock: 0x589484cfbbb3c5aa159fd33eb86d65b28752cc377b62df41216fe86ccbd33fb0
harvester-monitor_1 | Error:
harvester-monitor_1 | Unable to load celery application.
harvester-monitor_1 | The module scalecodec was not found.
harvester-monitor_1 |
harvester-monitor_1 | Error:
harvester-monitor_1 | Unable to load celery application.
harvester-monitor_1 | The module scalecodec was not found.
harvester-monitor_1 |
kusama_harvester-monitor_1 exited with code 0