Google Drive rate limits

[nightkr]

Google Drive only allows 1000 queries per user per 100 seconds, which makes our download logic pretty unhappy, especially during development.

[nightkr]

Note that this only seems to affect full-size images, not thumbnails. I'm going to disable full-size downloads for now...

[indrif]

We should probably implement a configurable throttling mechanism for queues. This will not be the only case where we need to control the number of calls. Most APIs have some kind of rate limit and even though most of them might not be applicable to us there are some that will.

Probably needs some discussion.

[indrif]

Many of the ai services have pretty aggressive rate limits. Azure limits the calls to 20/s for example for their cognitive image recognition service.

[nightkr]

Yeah, absolutely.