Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

24h data missing #134

Open
mczapala opened this issue Apr 4, 2020 · 13 comments
Open

24h data missing #134

mczapala opened this issue Apr 4, 2020 · 13 comments

Comments

@mczapala
Copy link

mczapala commented Apr 4, 2020

Querying for the 24h data (https://data.sensor.community/static/v2/data.24h.json) returns empty json file.

@stevecrawshaw
Copy link

As also does the hourly data: https://data.sensor.community/static/v2/data.1h.json which I am using for a real time feed..

@ricki-z
Copy link
Member

ricki-z commented Apr 5, 2020

the real time data is in the data.json feed.
the data.1h.json contains aggregated values (average over 60 minutes).

At the moment there is a very high load in all data centers worldwide (including those of Amazon and Google !!! just in case why we aren't switching to one of them). Thats why we needed to reduce the load on our servers.
As soon as the data center is upgrading the hardware we will enable these feeds again.

@mczapala
Copy link
Author

mczapala commented Apr 5, 2020

Any ETA when that might happen (days, weeks, months)?

@ricki-z
Copy link
Member

ricki-z commented Apr 5, 2020

More than some days for the 24h values (but hopefully only a few weeks).
A few days for the 1h values.
Many data centers need new hardware which normally is build in? Yes, China.

@mczapala
Copy link
Author

mczapala commented Apr 5, 2020

Sure, I understand. Thank you for the information.

@stevecrawshaw
Copy link

Thank you, understood. And thanks for providing this facility.

@ricki-z
Copy link
Member

ricki-z commented Apr 6, 2020

At the moment the 1h stream is online again. We have to read half a million measurements for this, so the load has increased significantly.

@tommasosansone91
Copy link

Querying for the 24h data (https://data.sensor.community/static/v2/data.24h.json) returns empty json file.

Now it works, but the query on the area returns a void dictionary. E.g. See the Berlin query in the doc page https://data.sensor.community/airrohr/v1/filter/area=52.5200,13.4050,10

@ricki-z
Copy link
Member

ricki-z commented Apr 18, 2020

We had to change the filter script, as someone used this in an app. This app also send massive requests with a radius of 0 ...
If using the filter it's better to use the bbox. The area filter needs to calculate the distance from the center for many sensors. This is very time consuming.

@tommasosansone91
Copy link

tommasosansone91 commented Apr 20, 2020

Thanks. So which API should I use to get real-time data causing the least load possible on your servers?

@ricki-z
Copy link
Member

ricki-z commented Apr 29, 2020

Use the box query. And try to do a request every 2 or 3 minutes. The data returned contains all values of the last 5 minutes. So even 4 minutes may be enough to get all data.

@tommasosansone91
Copy link

At the moment my app makes a request to the real-time api address every time the user lands on a certain page (No one else than me has the app address), but I am trying to implement a scheduled script that will make a request only once every 5 minutes. I hope this latter configuration won't overload your server.

@ricki-z
Copy link
Member

ricki-z commented May 3, 2020

This should be okay in terms of server load. The pure data.json is a "static" file, recreated once per minute.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants