Skip to content

Commit

Permalink
Remove WIP notice from readme
Browse files Browse the repository at this point in the history
  • Loading branch information
otsch committed Nov 7, 2021
1 parent 87897ce commit e23f1a3
Showing 1 changed file with 0 additions and 3 deletions.
3 changes: 0 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,6 @@
# Robots Exclusion Standard/Protocol Parser
## for Web Crawling/Scraping

ℹ️ This is currently still __work in progress__, but will receive a
first tag and will be registered on packagist very soon.

Use this library within crawler/scraper programs to parse robots.txt
files and check if your crawler user-agent is allowed to load certain
paths.
Expand Down

0 comments on commit e23f1a3

Please sign in to comment.