add pagination support to list #26
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Switch the REST API implementation over to the requests python library. This will make it easier to handle things like pagination in future changes. This adds a new dependency to the pwclient. However this is a well known python library, and is even mentioned in the patchwork API documentation itself. Signed-off-by: Jacob Keller <[email protected]>
The patchwork REST API defaults to sending a maximum of 30 items for API requests which return a list. This makes pwclient list fail to return more than 30 items when using the REST API. To handle this, the API includes 'Link' headers in the response which indicate whether there is more data and what URL the data is available at. Processing the Link header is done automatically by the requests library. Handle this in the _list implementation by checking whether the response had a 'next' header, and request that data as well. This allows the pwclient to list everything instead of being limited to 30 patches. Signed-off-by: Jacob Keller <[email protected]>
|
This looks mostly good. My one request is to avoid using |
|
Also, you can fix the |
|
Makes sense. I'll have to dig into it a bit to see how to parse. I started that way but found it difficult to parse correctly (I ended up getting double queries somehow). It shouldn't be too tricky to figure out tho. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This series adds support for handling the pagination Link headers when
listing patches. Doing this allows handling replies which contain more than
30 patches.
To do this, I propose switching from using urllib to using the python
requests library. Alternatively we could process the Link headers manually
and continue using urllib without the extra dependency. This is possible but
could be somewhat tricky to get correct.
I only implemented Link handling in the _list implementation since that one
was the most clear place where it was important. I'm not sure what other
API endpoints can return Link headers.