You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
While working on concurrency for scraping I hit a program crash. Now when I try to run the scraper this happens:
$ python __main__.py scrape
EthPM CLI v0.1.0a0
Traceback (most recent call last):
File "__main__.py", line 126, in <module>
main(parser, logger)
File "__main__.py", line 115, in main
scraper(args)
File "__main__.py", line 43, in scraper
last_scraped_block = trio.run(scrape, w3, ethpmcli_dir, start_block)
File "/home/piper/python-environments/ethpm-cli/lib/python3.6/site-packages/trio/_core/_run.py", line 1444, in run
raise runner.main_task_outcome.error
File "/home/piper/projects/ethpm-cli/ethpm_cli/scraper.py", line 32, in scrape
initialize_ethpm_dir(ethpm_dir, w3)
File "/home/piper/projects/ethpm-cli/ethpm_cli/scraper.py", line 85, in initialize_ethpm_dir
validate_chain_data_store(chain_data_path, w3)
File "/home/piper/projects/ethpm-cli/ethpm_cli/validation.py", line 68, in validate_chain_data_store
chain_data = json.loads(chain_data_path.read_text())
File "/home/piper/.pyenv/versions/3.6.5/lib/python3.6/json/__init__.py", line 354, in loads
return _default_decoder.decode(s)
File "/home/piper/.pyenv/versions/3.6.5/lib/python3.6/json/decoder.py", line 342, in decode
raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 26 column 2 (char 415)
What is wrong
While working on concurrency for scraping I hit a program crash. Now when I try to run the scraper this happens:
The underlying JSON document is:
Now this almost definitely from the file being written to by multiple concurrent threads, however, the program should gracefully handle corrupt files.
The text was updated successfully, but these errors were encountered: