Whats Changed
Web page normalizing to prevent all duplicate content, crawl traps, and more pages from being crawled repeatedly.
We can now crawl websites that target ports outside 80 and 443.
- feat(page): add relative directory url handling
- chore(website): fix relative page merging links
- chore(serde): fix cron compile configuration
- chore(chrome): update [email protected]
- chore(page): add port validation links
- chore(website): fix signature compile non disk feature flag
- chore(rand): update [email protected]
- [chore(abs): clear query pairs [#257]
Full Changelog: v2.26.27...v2.27.50