This repository contains code implementation for simple Web Crawler demonstration.
Jsoup java library is used for this purpose.Following Jsoup dependency snippet is added to pom.xml.
<groupId>org.jsoup</groupId>
<artifactId>jsoup</artifactId>
<version>1.7.2</version>
</dependency>
##Guidelines to run this web-crawler standalone app:
Clone the repository and follow the below simple steps to build this maven project and run
`$mvn package`
`$mvn install assembly:assembly`
Now you will have the jar with dependencies in your target folder.
Run the below java command by giving any URL as input argument
`$java -cp target/web-crawler-1.0-SNAPSHOT-jar-with-dependencies.jar com.webcrawler.core.App <URL>`
Example:
`$java -cp target/web-crawler-1.0-SNAPSHOT-jar-with-dependencies.jar com.webcrawler.core.App http://wiprodigital.com/`