Enviro (urban/weather/indoor) - Low Disk Space after Wifi Outage - Edge case #126
Comments
|
Thanks for raising this edge case. Yes, the board should keep attempting to upload files in this instance. Here's how I'd probably do it. I suspect there's some scenario I'm not thinking of with this though, not to mention the duplicated code. import enviro
import os
try:
# initialise enviro
enviro.startup()
# if the clock isn't set...
if not enviro.is_clock_set():
enviro.logging.info("> clock not set, synchronise from ntp server")
if not enviro.sync_clock_from_ntp():
# failed to talk to ntp server go back to sleep for another cycle
enviro.halt("! failed to synchronise clock")
# check disk space...
if enviro.low_disk_space():
# less than 10% of diskspace left, this probably means cached results
# are not getting uploaded so warn the user and halt with an error
# is an upload destination set?
if enviro.config.destination:
enviro.logging.error("! low disk space. Attempting to upload file(s)")
# if we have enough cached uploads...
enviro.logging.info(f"> {enviro.cached_upload_count()} cache file(s) need uploading")
if not enviro.upload_readings():
enviro.halt("! reading upload failed")
else:
# no destination so go to sleep
enviro.halt("! low disk space")
# TODO this seems to be useful to keep around?
filesystem_stats = os.statvfs(".")
enviro.logging.debug(f"> {filesystem_stats[3]} blocks free out of {filesystem_stats[2]}")
# TODO should the board auto take a reading when the timer has been set, or wait for the time?
# take a reading from the onboard sensors
enviro.logging.debug(f"> taking new reading")
reading = enviro.get_sensor_readings()
# here you can customise the sensor readings by adding extra information
# or removing readings that you don't want, for example:
#
# del readings["temperature"] # remove the temperature reading
#
# readings["custom"] = my_reading() # add my custom reading value
# is an upload destination set?
if enviro.config.destination:
# if so cache this reading for upload later
enviro.logging.debug(f"> caching reading for upload")
enviro.cache_upload(reading)
# if we have enough cached uploads...
if enviro.is_upload_needed():
enviro.logging.info(f"> {enviro.cached_upload_count()} cache file(s) need uploading")
if not enviro.upload_readings():
enviro.halt("! reading upload failed")
else:
enviro.logging.info(f"> {enviro.cached_upload_count()} cache file(s) not being uploaded. Waiting until there are {enviro.config.upload_frequency} file(s)")
else:
# otherwise save reading to local csv file (look in "/readings")
enviro.logging.debug(f"> saving reading locally")
enviro.save_reading(reading)
# go to sleep until our next scheduled reading
enviro.sleep()
# handle any unexpected exception that has occurred
except Exception as exc:
enviro.exception(exc)If you could give it a test that would be great. And yes, if you find that it does work for you, or need to make modifications, please raise a PR. |
|
Started testing by disabling wifi router and letting 2 x Urbans, 2 x Indoors and 1 x Weather build up files and fill up (readings evey 2 minutes with uplaods every 3 minutes). Sample log before turning wfi router back on, see entries at 17:12 which show not taking readings and attemtping to upload (162 files) but will fail as wifi still off. : All three types of devcies showing same Low disk space warning then attempeting to upload but failed as wifi off. Turned Wifi back on and all 5 devices successfully uplaod their 160 odd flies and returned ot normal operation. It did take about 20 minutes but as using SSL for MQTT it can take a lot of time Sample log on success: Data starting to upload: Data uploaded for all 5 devices: Pull request to cover change under c902806 |
|
Glad to see that the logging change worked and let the board recover! I'll look at getting the fix merged in. |
dave-ct
added a commit
to dave-ct/enviro
that referenced
this issue
Dec 13, 2022
In the event of low disk space caused by previous cache upload failures, dont take more readings and just try to upload. related to issue pimoroni#126
|
@ZodiusInfuser have done a seperate pull request here #128 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Version: 0.0.9
Image: https://github.com/pimoroni/enviro/actions/runs/3624876714
Summary: Uploads folder fills up and cannot recover without deleteing files.
Suggested Fix: Update so that if the file system is 90% full then do not take further readings but attempt to do uploads only on each wake so that it can eventually recover instead of sleeping.
While testing anothet issue had 2 x Urbans, 2 x Indoors and 1 x Weather running on battery doing readings every 2 minutes and uploads every 3 readings. At about 08:20 the wifi router seems to have gone down so when inspecting the devices 12 hours later all flashing read. After rebooting the wifi router the devices did not automtically recover. Tried poking but reverted quickly back to red light. On inspection of device logs it shows low disk space.
Uploads folder full of about 180 json files waiting to upload.
Simple fix was to delete the uplaods folder.
While this is an edge case due to frquency of readings. Possible solution is instead of
enviro.haltat 10% then can attepemt to upload the readings but dont take any new ones, can log a alternate mesasge as well. This could be useful in particular for the Urban or Weather as if they are attached to a roof or hard to reach area means you have to bring them inside to fix which is a pain.I think I could make pull request for this, just need to alter
main.pybut wanted to check if alternate approach should be done?The text was updated successfully, but these errors were encountered: