Skip to content

the-dvlpr/Web-Scraping-Tutorial

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

14 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Web Scraping/Crawling Tutorial

A tutorial on web scraping and crawling aimed to get you up and running in 15 minutes.

YouTube Videos

IMAGE ALT TEXT HERE IMAGE ALT TEXT HERE IMAGE ALT TEXT HERE

Credentials

Make sure to update your login info in the credentials.rb file.

How to Run from Command Line

To run each file navigate to the main directory (web_scraper) in your terminal and run the word ruby followed by the name of the file. Example: $ ruby auto_follow.rb or $ ruby auto_liker.rb

Updates (since videos were made)

I've made some fairly simple updates since the videos.

  • changed the while true loops to loop do loops with a break condition using a constant variable set at top
  • added more variables for counting, tracking, and info output
  • added top 100 users on instagram at the bottom of auto_follow.rb

Libraries Used

watir for crawling in live browser
pry ruby REPL
rb-readline ruby IRB and dependency of pry
awesome-print for clearer console output

About

Videos on youtube

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages