Skip to content

gaurav031/Scrape

Repository files navigation


Project Banner
webscraping nextjs tailwindcss mongodb

A Ecom Price Tracking Application

  1. 🤖 Introduction
  2. ⚙️ Tech Stack
  3. 🔋 Features
  4. 🤸 Quick Start
  5. 🕸️ Snippets
  6. 🔗 Links
  7. 🚀 More

🚨 Tutorial

Developed using Next.js and Bright Data's webunlocker, this e-commerce product scraping site is designed to assist users in making informed decisions. It notifies users when a product drops in price and helps competitors by alerting them when the product is out of stock, all managed through cron jobs.

  • Next.js
  • Bright Data
  • Cheerio
  • Nodemailer
  • MongoDB
  • Headless UI
  • Tailwind CSS

👉 Header with Carousel: Visually appealing header with a carousel showcasing key features and benefits

👉 Product Scraping: A search bar allowing users to input Amazon product links for scraping.

👉 Scraped Projects: Displays the details of products scraped so far, offering insights into tracked items.

👉 Scraped Product Details: Showcase the product image, title, pricing, details, and other relevant information scraped from the original website

👉 Track Option: Modal for users to provide email addresses and opt-in for tracking.

👉 Email Notifications: Send emails product alert emails for various scenarios, e.g., back in stock alerts or lowest price notifications.

👉 Automated Cron Jobs: Utilize cron jobs to automate periodic scraping, ensuring data is up-to-date.

and many more, including code architecture and reusability

Follow these steps to set up the project locally on your machine.

Prerequisites

Make sure you have the following installed on your machine:

Cloning the Repository

git clone https://github.com/gaurav031/Scrape.git
cd Scrape

Installation

Install the project dependencies using npm:

npm install

Set Up Environment Variables

Create a new file named .env in the root of your project and add the following content:

#SCRAPER
BRIGHT_DATA_USERNAME=
BRIGHT_DATA_PASSWORD=

#DB
MONGODB_URI=

#OUTLOOK
EMAIL_USER=
EMAIL_PASS=

Replace the placeholder values with your actual credentials. You can obtain these credentials by signing up on these specific websites from BrightData, MongoDB, and Node Mailer

Running the Project

npm run dev

Open http://localhost:3000 in your browser to view the project.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published