-
Notifications
You must be signed in to change notification settings - Fork 107
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Browse files
Browse the repository at this point in the history
This reverts commit 485b11e.
- Loading branch information
Showing
20 changed files
with
1,229 additions
and
1,969 deletions.
There are no files selected for viewing
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,21 +1,12 @@ | ||
# Building Blocks | ||
|
||
Building blocks are the atomic units of creating a vector retrieval stack. If you want to create a vector retrieval | ||
stack that's ready for production, you'll need to have a few key components in place. These include: | ||
Building blocks are the atomic units of creating a vector retrieval stack. If you want to create a vector retrieval stack that's ready for production, you'll need to have a few key components in place. These include: | ||
|
||
- Data sources: You can get your data from a variety of sources, including relational databases like PSQL and MySQL, | ||
data pipeline tools like Kafka and GCP pub-sub, data warehouses like Snowflake and Databricks, and customer data | ||
platforms like Segment. The goal here is to extract and connect your data so that it can be used in your vector stack. | ||
- Vector computation: This involves turning your data into vectors using models from Huggingface or your own custom | ||
models. You'll also need to know where to run these models and how to bring all of your computing infrastructure | ||
together using tools like custom spark pipelines or products like Superlinked. The ultimate goal is to have | ||
production-ready pipelines and models that are ready to go. | ||
- Vector search & management: This is all about querying and retrieving vectors from Vector DBs like Weaviate and | ||
Pinecone, or hybrid DBs like Redis and Postgres (with pgvector). You'll also need to use search tools like Elastic and | ||
Vespa to rank your vectors. The goal is to make the vectors indexable and search for relevant vectors when needed. | ||
- Data sources: You can get your data from a variety of sources, including relational databases like PSQL and MySQL, data pipeline tools like Kafka and GCP pub-sub, data warehouses like Snowflake and Databricks, and customer data platforms like Segment. The goal here is to extract and connect your data so that it can be used in your vector stack. | ||
- Vector computation: This involves turning your data into vectors using models from Huggingface or your own custom models. You'll also need to know where to run these models and how to bring all of your computing infrastructure together using tools like custom spark pipelines or products like Superlinked. The ultimate goal is to have production-ready pipelines and models that are ready to go. | ||
- Vector search & management: This is all about querying and retrieving vectors from Vector DBs like Weaviate and Pinecone, or hybrid DBs like Redis and Postgres (with pgvector). You'll also need to use search tools like Elastic and Vespa to rank your vectors. The goal is to make the vectors indexable and search for relevant vectors when needed. | ||
|
||
## Contents | ||
|
||
- [Data Sources](https://hub.superlinked.com/data-sources) | ||
- [Vector Compute](https://hub.superlinked.com/vector-compute) | ||
- [Vector Search & Management](https://hub.superlinked.com/vector-search) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,52 +1,47 @@ | ||
# Contributing | ||
|
||
VectorHub is a learning hub that lives on its contributors. We are always looking for people to help others, especially | ||
as we grow. You can contribute in many ways, either by creating new content or by letting us know if content needs | ||
updating. | ||
VectorHub is a learning hub that lives on its contributors. We are always looking for people to help others, especially as we grow. You can contribute in many ways, either by creating new content or by letting us know if content needs updating. | ||
|
||
## How is VectorHub organised | ||
|
||
VectorHub's content is organized into three major areas: | ||
|
||
1. Building Blocks: These cover the broad field of vector creation and retrieval. We take a step by step approach to | ||
creating a vector stack: Data Sources -> Vector Compute -> Vector Search & Management. | ||
1. Building Blocks: These cover the broad field of vector creation and retrieval. We take a step by step approach to creating a vector stack: Data Sources -> Vector Compute -> Vector Search & Management. | ||
|
||
1. Blog: This is where contributors can share examples of things they have been working on, research and solutions to | ||
problems they have encountered while working on Information Retrieval problems | ||
2. Blog: This is where contributors can share examples of things they have been working on, research and solutions to problems they have encountered while working on Information Retrieval problems | ||
|
||
1. Toolkit (coming soon): These are interesting apps, links, videos, tips, & tricks that aid in vector creation and | ||
retrieval. | ||
3. Toolkit (coming soon): These are interesting apps, links, videos, tips, & tricks that aid in vector creation and retrieval. | ||
|
||
## How to contribute | ||
|
||
[This loom](https://www.loom.com/share/aae75e4746f24453af0f3ae276f9ac56?sid=28db5254-f95f-48ae-8bf9-e13ed201bbce) | ||
explains how to set up your contributing workflow. | ||
[This loom](https://www.loom.com/share/aae75e4746f24453af0f3ae276f9ac56?sid=28db5254-f95f-48ae-8bf9-e13ed201bbce) explains how to set up your contributing workflow. | ||
|
||
To summarise: | ||
|
||
1. Fork the VectorHub repo | ||
1. Push all commits to your fork in the appropriate section for your content | ||
1. Open a PR to merge content from their fork to the remote repo (superlinked/vectorhub) | ||
2. Push all commits to your fork in the appropriate section for your content | ||
3. Open a PR to merge content from their fork to the remote repo (superlinked/vectorhub) | ||
|
||
When contributing an article please include the following at the start: | ||
|
||
1. One sentence to explain their topic / use case | ||
1. One-two sentences on why your use case is valuable to the reader | ||
1. A brief outline of what each section will discuss (can be bulletpointed) | ||
1) One sentence to explain their topic / use case | ||
2) One-two sentences on why your use case is valuable to the reader | ||
3) A brief outline of what each section will discuss (can be bulletpointed) | ||
|
||
## Get involved | ||
|
||
We constantly release bounties looking for content contributions. Keep an eye out for items with bounty labels on our | ||
GitHub. | ||
We constantly release bounties looking for content contributions. Keep an eye out for items with bounty labels on our GitHub. | ||
|
||
### Other ways you can get involved | ||
|
||
::::link-array :::link-array-item{headerImage headerColor} | ||
[Report an error/bug/typo](https://github.com/superlinked/VectorHub/issues) ::: | ||
::::link-array | ||
:::link-array-item{headerImage headerColor} | ||
[Report an error/bug/typo](https://github.com/superlinked/VectorHub/issues) | ||
::: | ||
|
||
:::link-array-item{headerImage headerColor} | ||
[Create new or update existing content](https://github.com/superlinked/VectorHub) ::: :::: | ||
[Create new or update existing content](https://github.com/superlinked/VectorHub) | ||
::: | ||
:::: | ||
|
||
:::hint{type="info"} Thank you for your suggestions! If you think there is anything to improve on VectorHub, feel free | ||
to contact us on [email protected], or check our [GitHub repository](https://github.com/superlinked/VectorHub). | ||
:::hint{type="info"} | ||
Thank you for your suggestions! If you think there is anything to improve on VectorHub, feel free to contact us on arunesh\@superlinked.com, or check our [GitHub repository](https://github.com/superlinked/VectorHub). | ||
::: |
Oops, something went wrong.