[EPIC] pagination support

[soxofaan]

Epic for all things pagination

So far, pagination support has been absent (or somewhat and afterthought) in the design of openeo-python-driver. We already started working on pagination support for job listings to battle some operational challenges, but there is still more to cover, hence this epic.

as a reference, these are openEO endpoints and things that can be paginated (from Open-EO/openeo-python-client#677 (comment)):

• GET /collections
• GET /processes
• GET /jobs
• GET /process_graph
• GET /files
• GET /services
• Logs (Services and Jobs)
• Collection Items (if implemented)

[soxofaan]

Related directly to openeo-python-driver:

• https://github.com/eu-cdse/openeo-cdse-infra/issues/337
• Implement "federation:missing" on /collections openeo-aggregator#122 (not directly a pagination issue, but the necessary API refactor ideally prepares for that)
• Implement "federation:missing" on /process_graphs openeo-aggregator#125 (not directly a pagination issue, but the necessary API refactor ideally prepares for that)
• Support job listing pagination openeo-aggregator#164
• Pagination support in aggregator? openeo-aggregator#175
• Support paging in job listings openeo-geopyspark-driver#959
• Support paging in job listings #332
• Clean up legacy job listing code paths #337
• Pagination reports next page, but no additional data is available #350
• Introduce support for paged job listing #333

Related in other projects:

• Enforce pagination of resource listings from the server-side openeo-api#550
• Support paginated job listings openeo-python-client#677
• Request lists of resources with pagination openeo-web-editor#331