PWV HTTP Requests Error

[sanahabhimani]

Description

Addressing this error for the PWV agent:

Unable to format event {'log_logger': <Logger 'ocs.ocs_agent.OCSAgent'>, 'log_level': <LogLevel=info>, 'log_namespace': 'ocs.ocs_agent. OCSAgent', '10g_SOle': None, 'log_format': 'acq:0 CRASH: [Failure instance: Traceback: <class \'requests .exceptions .JSONDecodeError\'>: [Errno Expecting value] {"pwv": nan, "timestamp": nan}:

Motivation and Context

I'm 10% unsure that this is truly an HTTP requests failure because I don't know if I should expect/assume a 204 error to output nan's. Implementing a try/except to loop through and make the http request again. Added a self.log.info message to query the requests status code the next time this happens in the agent to know more.

How Has This Been Tested?

Hasn't been tested on the flask server at the site itself.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.

[mhasself]

Forgive the random review but this caught my eye! Couple other comments:

1. The JSONDecodeError you're seeing is because the server is working (it sends a json-like structure) but is sending "json" in a (non-)standard that the requests library doesn't like. Specifically it's representing NaN values as "nan". Probably you want to ignore those data, anyway, so it's fine to just catch the error like you're doing.

2. Later in the function, you have this:

            session.data = {"timestamp": last_timestamp,
                            "pwv": {}}
            session.data['pwv'] = last_pwv

That's a bit of an odd construction, to have pwv sometimes be an empty dict and sometimes be a float. It could cause trouble for anyone trying to use the session.data. Can you change it to simply:

            session.data = {"timestamp": last_timestamp,
                            "pwv": last_pwv}

?

[BrianJKoopman]

Thanks for the review @mhasself. I just wanted to mention what I think is going on on the backend here. Looking at the most recent two crashes, this happens at the first query after 00:00 -- 2023-11-05T00:00:08+0000 and 2023-11-07T00:00:10+0000 are the two that I'm seeing.

The server that this agent queries takes the median of the last four data points, and I expect that code is running into some error when there are less than four points in the file its reading, so we end up with a nan response.

[sanahabhimani]

thank you both for the input and catching my half-brained errors!

[mhasself]

Thanks for the fixes!

[BrianJKoopman]

Based on the logs it looks like the except statement needs to be handling the JSONDecodeError, not a ValueError. Where was the ValueError coming up?