Skip to content

Commit

Permalink
Update docs/website/docs/tutorial/rest-api.md
Browse files Browse the repository at this point in the history
Co-authored-by: Akela Drissner-Schmid <[email protected]>
  • Loading branch information
burnash and akelad authored Sep 10, 2024
1 parent 0c11739 commit c9eeebc
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion docs/website/docs/tutorial/rest-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -254,7 +254,7 @@ pokemon_source = rest_api_source(

## Loading data incrementally

When working with some APIs, you may need to load data incrementally to avoid fetching the entire dataset every time and to reduce the load time. The API that support incremental loading usually provide a way to fetch only new or changed data (most often by using a timestamp field like `updated_at`, `created_at`, or incremental IDs).
When working with some APIs, you may need to load data incrementally to avoid fetching the entire dataset every time and to reduce the load time. APIs that support incremental loading usually provide a way to fetch only new or changed data (most often by using a timestamp field like `updated_at`, `created_at`, or incremental IDs).

To illustrate incremental loading, let's consider the GitHub API. In the `rest_api_pipeline.py` file, you can find an example of how to load data from the GitHub API incrementally. Let's take a look at the configuration:

Expand Down

0 comments on commit c9eeebc

Please sign in to comment.