Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add elasticsearch component #211

Merged
merged 27 commits into from
Aug 8, 2024

Conversation

zidanehakim
Copy link
Collaborator

@zidanehakim zidanehakim commented Jul 11, 2024

Because

  • We need to integrate Elasticsearch into our data pipeline.
  • This will allow us to cover data movement user stories

This commit

  • Allow client to connect to their Elasticsearch database through VDP

  • Auth using passwordless API Key and cloud ID through elasticsearch go sdk

  • Create execution function to handle these following tasks

  • TASK_INDEX (INSERT)

  • TASK_UPDATE

  • TASK_SEARCH

  • TASK_DELETE

  • TASK_CREATE_INDEX

  • TASK_DELETE_INDEX

Copy link

codecov bot commented Jul 21, 2024

Codecov Report

Attention: Patch coverage is 51.00154% with 318 lines in your changes missing coverage. Please review.

Project coverage is 40.55%. Comparing base (219c77e) to head (f300438).
Report is 6 commits behind head on main.

Files Patch % Lines
data/elasticsearch/v0/tasks.go 55.00% 177 Missing and 84 partials ⚠️
data/elasticsearch/v0/main.go 26.08% 32 Missing and 2 partials ⚠️
data/elasticsearch/v0/client.go 0.00% 22 Missing ⚠️
store/store.go 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #211      +/-   ##
==========================================
+ Coverage   39.75%   40.55%   +0.80%     
==========================================
  Files         123      128       +5     
  Lines       12351    13490    +1139     
==========================================
+ Hits         4910     5471     +561     
- Misses       6808     7228     +420     
- Partials      633      791     +158     
Flag Coverage Δ
unittests 40.55% <51.00%> (+0.80%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

data/elasticsearch/v0/tasks.go Outdated Show resolved Hide resolved
data/elasticsearch/v0/config/tasks.json Outdated Show resolved Hide resolved
data/elasticsearch/v0/config/tasks.json Outdated Show resolved Hide resolved
@donch1989 donch1989 merged commit eb492ca into instill-ai:main Aug 8, 2024
8 checks passed
donch1989 pushed a commit that referenced this pull request Aug 13, 2024
🤖 I have created a release *beep* *boop*
---


##
[0.25.0-beta](v0.24.0-beta...v0.25.0-beta)
(2024-08-13)


### Features

* add a hook to avoid we miss make document
([#244](#244))
([4c4531d](4c4531d))
* add elasticsearch component
([#211](#211))
([eb492ca](eb492ca))
* add Fireworks AI component
([#237](#237))
([0c40652](0c40652))
* add Groq component
([#269](#269))
([1401220](1401220))
* add mongodb component
([#198](#198))
([2cb550f](2cb550f))
* add qdrant component
([#271](#271))
([bd2b9e6](bd2b9e6))
* add weaviate component
([#246](#246))
([cb3e667](cb3e667))
* add WhatsApp component
([#226](#226))
([28d0de8](28d0de8))
* **artifact:** add artifact component
([#268](#268))
([dabf472](dabf472))
* **artifact:** add artifact component
([#275](#275))
([15fc0d2](15fc0d2))
* **document:** integrate pdf2md in document operator
([#277](#277))
([07360d1](07360d1))
* **groq, fireworksai:** take out the unsupported models from instill
credit ([#283](#283))
([8978acd](8978acd))
* make component ID accessible on IExecution
([#257](#257))
([dd63656](dd63656))
* **openai:** support `gpt-4o-2024-08-06` and structured output
([#280](#280))
([8bdaef7](8bdaef7))
* **sql:** add TASK_INSERT_MANY and fix sql query validation
([#252](#252))
([3a93cea](3a93cea))
* **text:** add tokenizer for cohere & new gpt-4o
([#276](#276))
([5d8cec3](5d8cec3))
* **text:** revert "add tokenizer for cohere & new gpt-4o
([#276](#276))"
([910a330](910a330))


### Bug Fixes

* **artifact:** add the description to remind users to add file
extension ([#281](#281))
([5ff5d7a](5ff5d7a))
* ignore bold case and add all line to result
([#272](#272))
([219c77e](219c77e))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: 👋 Done
3 participants