Paginated APIs x Python

This talk has been accepted for PyOhio 2022

Description

While working with making requests to external RESTful APIs using Python, we can use just one line to get data. What if the endpoints are paginated and have limitations as well? Developers may find it hard to code for paginated results, store them and/or add throttling between consecutive requests.

This talk will cover some useful and robust ways to deal efficiently with range of paginated API patterns using Python.

Talk Objectives

• How paginated APIs work and what are some common patterns
• How can we use Python to efficiently fetch paginated results
• What are some common ways to throttle our calls and respect limits
• Parallel fetch/get calls (only if time permits)

Examples:

I have included examples for processing different types of paginated apis in respective scripts