Go Link Crawler

Crawler for links and titles. It benchmarks requests per second.

Copy ./config/.go-link-crawler.example.yaml to ./config/.go-link-crawler.yaml

Build

make help

make run-docker

if you need use other file than list.txt run docker container with

docker run -v $PATH_TO_YOUR_FILE:/app/crawl_list.txt -it go-link-crawler ./main ./crawl_list.txt

MIT

Name		Name	Last commit message	Last commit date
Latest commit History 4 Commits
config		config
log		log
services		services
utils		utils
.gitignore		.gitignore
Dockerfile		Dockerfile
LICENSE		LICENSE
Makefile		Makefile
README.md		README.md
go.mod		go.mod
go.sum		go.sum
list.txt		list.txt
main.go		main.go