-
Notifications
You must be signed in to change notification settings - Fork 481
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #2965 from opensearch-project/main
Syncing CFP end date changes to prod, and one event edit.
- Loading branch information
Showing
7 changed files
with
158 additions
and
3 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
--- | ||
name: Nate McCall | ||
short_name: zznate | ||
photo: '/assets/media/community/members/zznate.jpg' | ||
title: 'OpenSearch Community Member: Nate McCall' | ||
primary_title: Nate McCall | ||
breadcrumbs: | ||
icon: community | ||
items: | ||
- title: Community | ||
url: /community/index.html | ||
- title: Members | ||
url: /community/members/index.html | ||
- title: 'Nate McCall's Profile' | ||
url: '/community/members/zznate.html' | ||
twitter: 'zznate' | ||
github: zznate | ||
job_title_and_company: 'Product Research and Development at DataStax' | ||
personas: | ||
- author | ||
permalink: '/community/members/zznate.html' | ||
redirect_from: '/authors/zznate/' | ||
--- | ||
Nate is currently in product research and development at DataStax. He is a Vice President emeritus at The Apache Software Foundation and is a committer and PMC member on Apache Cassandra. In the off hours he can be found building high-end custom roller skates for customers all over the world at his shop Seaside Skates in Paraparaumu, Aotearoa New Zealand. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,83 @@ | ||
--- | ||
calendar_date: '2024-06-18' | ||
eventdate: 2024-06-18 10:30:00 -0700 | ||
primary_title: Development Backlog & Triage Meeting - ml-commons - 2024-06-18 | ||
title: Development Backlog & Triage Meeting - ml-commons - 2024-06-18 | ||
online: true | ||
signup: | ||
url: https://www.meetup.com/opensearch/events/301599479 | ||
title: Join on Meetup | ||
|
||
--- | ||
|
||
Join the OpenSearch ml-commons team for their next backlog & triage planning meeting. | ||
|
||
(hosts: [Yaliang Wu](https://github.com/ylwu-amzn), [Dhrubo Saha](https://github.com/dhrubo-os), [Jing Zhang](https://github.com/jngz-es), & [Xun Zhang](https://github.com/Zhangxunmt)) | ||
|
||
--- | ||
|
||
**Join Zoom Meeting** | ||
```json | ||
https://us02web.zoom.us/j/82164218920 | ||
|
||
Meeting ID: 821 6421 8920 | ||
Passcode: 259735 | ||
|
||
--- | ||
|
||
One tap mobile | ||
+12532050468,,82164218920# US | ||
+12532158782,,82164218920# US (Tacoma) | ||
|
||
--- | ||
Dial by your location | ||
• +1 253 205 0468 US | ||
• +1 253 215 8782 US (Tacoma) | ||
• +1 346 248 7799 US (Houston) | ||
• +1 669 444 9171 US | ||
• +1 669 900 9128 US (San Jose) | ||
• +1 719 359 4580 US | ||
• +1 646 558 8656 US (New York) | ||
• +1 646 931 3860 US | ||
• +1 689 278 1000 US | ||
• +1 301 715 8592 US (Washington DC) | ||
• +1 305 224 1968 US | ||
• +1 309 205 3325 US | ||
• +1 312 626 6799 US (Chicago) | ||
• +1 360 209 5623 US | ||
• +1 386 347 5053 US | ||
• +1 507 473 4847 US | ||
• +1 564 217 2000 US | ||
• 877 853 5247 US Toll-free | ||
• 888 788 0099 US Toll-free | ||
|
||
Meeting ID: 821 6421 8920 | ||
|
||
Find your local number: https://us02web.zoom.us/u/kcboT3QOI | ||
|
||
``` | ||
|
||
--- | ||
|
||
**Agenda:** | ||
|
||
**Triage issues** *(add the triaged label once reviewed/ready. They can be also labelled as sprint backlog if we are looking to queueing them up next, or good first issue / help wanted when appropriate.)* | ||
|
||
* [Backend ml-commons](https://github.com/opensearch-project/ml-commons/issues) | ||
* [Dashboards ml-commons](https://github.com/opensearch-project/ml-commons-dashboards/issues) | ||
|
||
**Sprint backlog** *(Examine if it still reflects the work that we are committing to doing and is it in the right priority order)* | ||
|
||
* [Backend ml-commons](https://github.com/opensearch-project/ml-commons/issues) | ||
* [Dashboards ml-commons](https://github.com/opensearch-project/ml-commons-dashboards/issues) | ||
|
||
**Backlog** *(anything we should move to sprint backlog? anything we should tag asking for help from the community?)* | ||
|
||
* [Backend ml-commons](https://github.com/opensearch-project/ml-commons/issues) | ||
* [Dashboards ml-commons](https://github.com/opensearch-project/ml-commons-dashboards/issues) | ||
|
||
|
||
***Please see Meetup link for URL and required passcode.*** | ||
|
||
|
||
*By joining the Development Backlog & Triage Meeting, you grant OpenSearch, and our affiliates the right to record, film, photograph, and capture your voice and image during the Development Backlog & Triage Meeting (the “Recordings”). You grant to us an irrevocable, nonexclusive, perpetual, worldwide, royalty-free right and license to use, reproduce, modify, distribute, and translate, for any purpose, all or any part of the Recordings and Your Materials. For example, we may distribute Recordings or snippets of Recordings via our social media outlets.* |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
48 changes: 48 additions & 0 deletions
48
_posts/2024-06-06-opensearch-partnering-with-datastax-on-generative-ai.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,48 @@ | ||
--- | ||
layout: post | ||
title: "Announcing an OpenSearch and DataStax generative AI partnership" | ||
authors: | ||
- zznate | ||
date: 2024-06-13 | ||
categories: | ||
- community | ||
- partners | ||
meta_keywords: Generative AI, retrieval augmented generation , DataStax HCDP, OpenSearch integrations | ||
meta_description: Learn about the collaboration between open source startup DataStax and the OpenSearch Project on integration efforts to support Generative AI developers. | ||
excerpt: | ||
has_math: false | ||
has_science_table: false | ||
--- | ||
|
||
DataStax and the OpenSearch Project are announcing a series of integration efforts to support generative AI developers. Retrieval-augmented generation (RAG) is a key design pattern in generative AI. RAG applications work by assembling context from a variety of sources, which is then processed by a large language model (LLM) to provide an intelligent and relevant response. Serving these applications requires a mix of data retrieval and storage capabilities, and we, OpenSearch and DataStax, are committed to working together to serve the broad needs of generative AI developers. | ||
|
||
To power the explosive growth within the generative AI space, we need to keep innovating on the tooling available to developers. These tools require access to a variety of enterprise data, and we want to be there to provide that access in whatever common format is required. Being able to retrieve data in the most flexible ways possible is a necessary catalyst for getting RAG and generative AI knowledge applications to production. | ||
|
||
Amazon sponsors the OpenSearch Project to ensure the continuing existence of an open-source search engine that users can use, modify, and extend however they wish. In addition to AWS, the OpenSearch community is full of active contributors, maintainers, and partners. For generative AI specifically, OpenSearch offers the following benefits: | ||
|
||
* **Ease of use**: OpenSearch provides easy-to-use indexing and search capabilities and has built-in features for text analysis, tokenization, and relevance scoring. | ||
* **Optimized for text retrieval**: OpenSearch makes it easy to find and rank documents based on keyword queries | ||
* **Versatility**: OpenSearch can handle a wide variety of data types and formats | ||
* **AI/ML integration**: OpenSearch supports semantic search with vector embeddings, multi-modal search, hybrid search with score normalization, and sparse vector search | ||
|
||
DataStax is a leading contributor to a range of open source projects, including [Langflow](https://langflow.org/), [Apache Cassandra](https://cassandra.apache.org/_/index.html), and [JVector](https://github.com/jbellis/jvector), which provides vector search through DiskANN and advanced GenAI techniques like COLBert. Generative AI developers seek this database and vector combination to provide: | ||
|
||
* **Context assembly**: Langflow delivers a UI to discover ecosystem components and compose the workflows that back Generative AI applications | ||
* **Similarity search**: JVector offers high-performance vector similarity search and can handle embedding-based queries which require low latency and high relevance | ||
* **Scalability**: Cassandra offers scalable persistence for structured and semi-structured data | ||
|
||
The combination of these technologies enable semantic and keyword searches as well as hybrid query processing. Context is assembled using: | ||
* Keyword queries which are directed to OpenSearch to retrieve relevant documents | ||
* Semantic queries use JVector and Cassandra to find the most relevant data points based on vector similarity | ||
* Database queries which provide known personalization, profile, and transactional data | ||
|
||
### **Moving Forward** | ||
DataStax will maintain a JVector integration for OpenSearch and offer OpenSearch as part of its self-managed offering platform, HCDP (Hyper Converged Data Platform), and as an integration for its cloud service, Astra. | ||
|
||
Enterprises have spent years investing in search infrastructure. With the inclusion of OpenSearch, DataStax can provide developers the most flexible information retrieval possible using applications already familiar to many enterprises. OpenSearch bridges the gap between single-document Q&A and open-domain Q&A, essentially providing the ability to reason across multiple diverse documents and texts by combining keyword search in OpenSearch with the dense vector search of JVector in Astra and HCDP. | ||
|
||
For generative AI, relevance is critical, and through this partnership we will ensure that your enterprise data estate can act as context for RAG and generative AI workflows to provide as much data to the context as possible. For more information, see the [HCDP announcement](https://www.datastax.com/fr/blog/introducing-vector-search-for-self-managed-modern-architecture). | ||
|
||
|
||
|
||
|
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters