Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix typing issues for typed Scrapy. #131

Merged
merged 13 commits into from
Nov 19, 2024
Merged

Fix typing issues for typed Scrapy. #131

merged 13 commits into from
Nov 19, 2024

Conversation

wRAR
Copy link
Contributor

@wRAR wRAR commented Sep 21, 2023

No description provided.

@codecov
Copy link

codecov bot commented Sep 21, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.89%. Comparing base (64ad254) to head (380fefd).
Report is 14 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #131      +/-   ##
==========================================
+ Coverage   97.87%   97.89%   +0.02%     
==========================================
  Files          14       14              
  Lines        1597     1615      +18     
  Branches      296      297       +1     
==========================================
+ Hits         1563     1581      +18     
  Misses         14       14              
  Partials       20       20              
Files with missing lines Coverage Δ
scrapy_zyte_api/_params.py 100.00% <100.00%> (ø)
scrapy_zyte_api/_request_fingerprinter.py 98.33% <100.00%> (ø)
scrapy_zyte_api/_session.py 100.00% <100.00%> (ø)
scrapy_zyte_api/addon.py 98.11% <100.00%> (+0.03%) ⬆️
scrapy_zyte_api/handler.py 95.26% <100.00%> (+0.12%) ⬆️
scrapy_zyte_api/providers.py 94.73% <100.00%> (+0.08%) ⬆️
scrapy_zyte_api/responses.py 96.11% <100.00%> (+0.11%) ⬆️
---- 🚨 Try these New Features:

scrapy_zyte_api/responses.py Outdated Show resolved Hide resolved
@wRAR wRAR marked this pull request as draft September 26, 2023 09:04
@wRAR wRAR marked this pull request as ready for review November 4, 2024 09:59
@Gallaecio Gallaecio closed this Nov 4, 2024
@Gallaecio Gallaecio reopened this Nov 4, 2024
@wRAR
Copy link
Contributor Author

wRAR commented Nov 4, 2024

Scrapy < 2.1 has a hidden bug, accidentally fixed in scrapy/scrapy#4481: scrapy.responsetypes.ResponseTypes.from_headers() expects headers keys to be case-insensitive so it can't work with simple dicts. Looks like we need to skip a couple of tests on Scrapy 2.0.1.

@wRAR
Copy link
Contributor Author

wRAR commented Nov 4, 2024

OTOH I think this breakage will also happen in user code. Unless we want to drop Scrapy 2.0.1, we need to use scrapy.http.Headers here.

@@ -393,7 +393,7 @@ class UseChecker(ConstantChecker):
def check(self, response: Response, request: Request) -> bool:
if response.meta.get(SESSION_INIT_META_KEY, False) is True:
return True
return super().check(request, response)
return super().check(response, request)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😮

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah.

It's a test though.

@wRAR wRAR merged commit 5fbf0bc into main Nov 19, 2024
19 checks passed
@wRAR wRAR deleted the scrapy-typing branch November 19, 2024 08:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants