Skip to content

Simple yet powerful web scraper to extract valuable data from the web effortlessly.

License

Notifications You must be signed in to change notification settings

imariom/TheHarvester

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 

Repository files navigation

TheHarvester 🚀

A simple yet powerful distributed key-value store database with features like replication, sharding, and fault tolerance. This project explores distributed systems concepts, networking, and advanced concurrency patterns in Go.

Table of Contents About the Project Key Features Getting Started Usage Examples Tech Stack Project Roadmap Contributing License Contact

About the Project

Motivation: Explain why you built this project and any specific problem it solves.

Overview: Briefly outline what this project does, how it stands out, and any unique approaches or techniques used.

Use Cases: - Example 1 - Example 2

Key Features

  • Hybrid storage support: In-Memory and Disk Persistence
  • Use LSM tree to model data
  • TTL Support
  • Batching and Atomic Operations support
  • Snapshotting for Faster Recovery
  • Concurrency Control with Optimistic Locking

Getting Started

Installing

To start using NexusDB, install Go 1.23 or above. NexusDB needs go modules. From your project, run the following command

$ go get github.com/imariom/nexusdb

This will retrieve the library.

Installing NexusDB Command Line Tool

NexusDB provides a CLI tool which can perform certain operations like offline backup/restore. To install the NexusDB CLI, retrieve the repository and checkout the desired version. Then run

$ cd nexusc
$ go install .

This will install the NexusDB command line utility into your $GOBIN path.

Tech Stack

Languages: Python, JavaScript, etc. Frameworks: Django, React, Flask, etc. Libraries/Tools: BeautifulSoup, TensorFlow, Docker, etc.

Project Roadmap

A list of any upcoming features or improvements planned for the project.

Initial setup Feature enhancements Documentation updates Future plans (if any)

Contributing

Contributions are welcome! If you'd like to collaborate, please:

  1. Fork the repository.
  2. Create your feature branch (git checkout -b feature/YourFeature).
  3. Commit your changes (git commit -m 'Add YourFeature').
  4. Push to the branch (git push origin feature/YourFeature).
  5. Open a Pull Request.

License

Distributed under the MIT License. See LICENSE for more information.

Contact

Mário Moiane - [email protected]

About

Simple yet powerful web scraper to extract valuable data from the web effortlessly.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages