Onion Address Example

[AlexanderWeismannn]

Hi there, could you provide an example / instructions on performing a crawling command on a .onion address (say duckduckgo). Do i just need to have TOR running in the background? Is there anything special i need to do?

Cheers!

[kisenge]

I recently got it to work for me. You will have to find your Tor.exe file and run it. I used Powershell to do this. As well I got it to work after commenting out the check_tor() line in torcrawl.py. and including the "-y 1" or "-y 0" tag at the command line. Good luck.

[MikeMeliz]

You can find some simple examples and how to run the script on README.md. Try to follow first the Installation section. And if you're planning to use TOR to crawl or extract a website, then yes you'll need it! Or you can use the argument -w to just use the plain crawler.

Some simple examples would be:
python3 torcrawl.py -u google.com -> which will extract -using Tor relay- the front-page of Google into your terminal.
python3 torcrawl.py -u https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/ -> the same thing, but now with the .onion address of duckduckgo.
python3 torcrawl.py -u google.com -c -d 2 -p 5 -> to crawl Google with depth 2 and 5 seconds pause between requests, etc.