Skip to content

Commit

Permalink
Update docs/website/docs/tutorial/rest-api.md
Browse files Browse the repository at this point in the history
Co-authored-by: Akela Drissner-Schmid <[email protected]>
  • Loading branch information
burnash and akelad committed Sep 11, 2024
1 parent 9c60f9f commit 4e884ad
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion docs/website/docs/tutorial/rest-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -254,7 +254,7 @@ pokemon_source = rest_api_source(

## Loading data incrementally

When working with some APIs, you may need to load data incrementally to avoid fetching the entire dataset every time and to reduce the load time. The API that support incremental loading usually provide a way to fetch only new or changed data (most often by using a timestamp field like `updated_at`, `created_at`, or incremental IDs).
When working with some APIs, you may need to load data incrementally to avoid fetching the entire dataset every time and to reduce the load time. APIs that support incremental loading usually provide a way to fetch only new or changed data (most often by using a timestamp field like `updated_at`, `created_at`, or incremental IDs).

To illustrate incremental loading, let's consider the GitHub API. In the `rest_api_pipeline.py` file, you can find an example of how to load data from the GitHub API incrementally. Let's take a look at the configuration:

Expand Down

0 comments on commit 4e884ad

Please sign in to comment.