Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Page Navigation Timeout Handling #70

Merged
merged 40 commits into from
Aug 18, 2023
Merged

Improve Page Navigation Timeout Handling #70

merged 40 commits into from
Aug 18, 2023

Conversation

bachdumpling
Copy link
Contributor

@bachdumpling bachdumpling commented Aug 2, 2023

This PR improves the reliability of page navigation in the collect function by better handling timeouts.

Changes

  • Implements Promise.race to timeout initial navigation after 10 seconds. If the navigation does not complete within this time, the code will automatically attempt to navigate again with the waitUntil: 'domcontentloaded' option. This prevents very slow pages from causing total failures.
  • After initial load, allows collecting partial data even if timeouts occur. Previously timeouts triggered an error and automatically stopped the program.
  • Increases default timeout from 30s to 35s for additional time to navigate.
  • This version enhances page navigation reliability by implementing a timeout mechanism for the fillForms function using Promise.race. If fillForms() doesn't complete within a specified time, it exits gracefully, allowing the program to continue. It checks if the page is closed before interacting with it and handles the "Execution context was destroyed" error gracefully.

Testing

Other Improvements

  • Extracted navigation logic into navigateWithTimeout function for better readability.
  • Added console logs to provide more info about navigation process and aid debugging.

@bachdumpling bachdumpling marked this pull request as draft August 2, 2023 18:45
@bachdumpling bachdumpling linked an issue Aug 2, 2023 that may be closed by this pull request
@bachdumpling bachdumpling marked this pull request as ready for review August 2, 2023 19:02
example.ts Outdated Show resolved Hide resolved
Copy link
Member

@dphiffer dphiffer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heyyyy let's merge it!

@bachdumpling bachdumpling merged commit 86d7f74 into main Aug 18, 2023
1 check passed
@dphiffer dphiffer deleted the fix-timeout branch August 18, 2023 20:55
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants