All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
- The
Parser
now also trims hidden whitespace characters that aren't covered by PHP'strim()
function by default. Such characters at the beginning of a line can cause parsing to fail, because it's important that user-agent and rule lines actually start with the corresponding keywords.
- Method
isExplicitlyNotAllowedFor()
that ignores rules for wildcard user-agent (*
) and checks if some path is explicitly not allowed for a certain user-agent.
- Required PHP version is now 8.0.
- It now also parses
Sitemap:
lines. You can get all referenced sitemaps via thesitemaps()
method of theRobotsTxt
class.
- Also allow usage of crwlr/url 1.0 as it's not a problem at all and the PHP version requirement of this package is still
^7.4|^8.0
.
- Upgraded crwlr/url package version constraint.