Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

getUSGS fails when recent measurement is not available in lookback window #24

Open
msweier opened this issue Jan 6, 2025 · 1 comment

Comments

@msweier
Copy link
Collaborator

msweier commented Jan 6, 2025

If the USGS gage stops transmitting and there recent no measurements available, the getUSGS script fails on line 182 in the getUSGS_ts function. The getUSGS function expects a name element in the timeseries attribute returned by the USGS API. The USGS API returns a blank array in this case. This causes the script to fail and not process any other USGS sites in the queue.

Code expects this:
image

But gets this:
image

@msweier msweier changed the title getUSGS fails when recent measurement is not avaiable in lookback window getUSGS fails when recent measurement is not available in lookback window Jan 6, 2025
@msweier
Copy link
Collaborator Author

msweier commented Jan 6, 2025

Hmm...in troubleshooting this I could not replicate. The gage is fixed and errors from similar non transmitting gages are being handed gracefully. I guess I will just keep my eyes open if it happens again.

This was the original error I got on the server:

2025-01-03 21:43:19;INFO;Attempting to write values for ts_id -->  Dodge.Flow.Inst.~15Minutes.0.Raw-USGS,05379500.00060
Traceback (most recent call last):
  File "/./localsoft/python3/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 3790, in get_loc
    return self._engine.get_loc(casted_key)
  File "index.pyx", line 152, in pandas._libs.index.IndexEngine.get_loc
  File "index.pyx", line 181, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 7080, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 7088, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: '05379500.00060'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/././scripts/getUsgs/getUSGS_CDA.py", line 350, in <module>
    main()
  File "/././scripts/getUsgs/getUSGS_CDA.py", line 346, in main
    CWMS_writeData(USGS_ts, USGS_data, USGS_data_method)
  File "/././scripts/getUsgs/getUSGS_CDA.py", line 224, in CWMS_writeData
    USGS_data_row = USGS_data.loc[USGS_Id_param]
  File "/./localsoft/python3/lib/python3.9/site-packages/pandas/core/indexing.py", line 1153, in __getitem__
    return self._getitem_axis(maybe_callable, axis=axis)
  File "/./localsoft/python3/lib/python3.9/site-packages/pandas/core/indexing.py", line 1393, in _getitem_axis
    return self._get_label(key, axis=axis)
  File "/./localsoft/python3/lib/python3.9/site-packages/pandas/core/indexing.py", line 1343, in _get_label
    return self.obj.xs(label, axis=axis)
  File "/./localsoft/python3/lib/python3.9/site-packages/pandas/core/generic.py", line 4236, in xs
    loc = index.get_loc(key)
  File "/./localsoft/python3/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 3797, in get_loc
    raise KeyError(key) from err
KeyError: '05379500.00060'

and the response from the USGS website triggering the error
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant