Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add sql component #193

Merged
merged 13 commits into from
Jul 26, 2024
Merged

Conversation

zidanehakim
Copy link
Collaborator

@zidanehakim zidanehakim commented Jul 2, 2024

Because

  • We need to integrate sql into our data pipeline.
  • This will allow us to cover data movement user stories

This commit

  • Allow client to connect to their sql database through VDP

  • Create execution function to handle following SQL tasks

  • TASK_INSERT

  • TASK_UPDATE

  • TASK_SELECT

  • TASK_DELETE

  • TASK_CREATE_TABLE

  • TASK_DROP_TABLE

@zidanehakim
Copy link
Collaborator Author

zidanehakim commented Jul 2, 2024

I have opened a new PR regarding the AWS RDS component. I had GPG key (signed commit) and component test issues with the previous PR (#191 ). This PR will be the final one and I have also added component test on the commit. Thank you!

Copy link
Member

@chuang8511 chuang8511 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I finished my first review.
I am not super familiar with SQL.
Please bring up some discussion points if I miss something.

data/awsrds/v0/config/definition.json Outdated Show resolved Hide resolved
data/awsrds/v0/config/tasks.json Outdated Show resolved Hide resolved
data/awsrds/v0/config/tasks.json Outdated Show resolved Hide resolved
data/awsrds/v0/config/tasks.json Outdated Show resolved Hide resolved
data/awsrds/v0/main.go Outdated Show resolved Hide resolved
data/awsrds/v0/main.go Outdated Show resolved Hide resolved
data/awsrds/v0/main.go Outdated Show resolved Hide resolved
data/awsrds/v0/main.go Outdated Show resolved Hide resolved
data/awsrds/v0/main.go Outdated Show resolved Hide resolved
data/awsrds/v0/component_test.go Outdated Show resolved Hide resolved
@zidanehakim
Copy link
Collaborator Author

@chuang8511

I have fixed most of the problems. Please kindly review it for the second round. Thanks!

@donch1989
Copy link
Member

A quick question: any reason that we choose sqlx for the db client?

@zidanehakim
Copy link
Collaborator Author

@donch1989

It might just be my preference, both sqxl and sql are actually the same, but I prefer sqlx more since it simpler and make the code cleaner

@zidanehakim zidanehakim changed the title feat: add aws rds component feat: add sql component Jul 8, 2024
@donch1989 donch1989 merged commit 9a373f3 into instill-ai:main Jul 26, 2024
6 checks passed
donch1989 pushed a commit that referenced this pull request Jul 31, 2024
🤖 I have created a release *beep* *boop*
---


##
[0.24.0-beta](v0.23.0-beta...v0.24.0-beta)
(2024-07-31)


### Features

* add audio operator
([#236](#236))
([fe8abff](fe8abff))
* add handler to auto-fill missing default values
([#210](#210))
([dcad3f0](dcad3f0))
* add HubSpot component
([#199](#199))
([b3936a8](b3936a8))
* add Jira component
([#205](#205))
([51f3ed7](51f3ed7))
* add Ollama component
([#224](#224))
([810f850](810f850))
* add sql component
([#193](#193))
([9a373f3](9a373f3))
* add token count for each chunk
([#235](#235))
([bb69104](bb69104))
* add video operator to fulfil unstructured data process
([#238](#238))
([a1459d7](a1459d7))
* **document:** add docx doc pptx ppt html to transform to text in
markdown format
([#232](#232))
([2932db9](2932db9))
* **document:** move ConvertToText task from text operator to document
operator ([#248](#248))
([699ca70](699ca70))
* introduce event handler interface
([#253](#253))
([9599b42](9599b42))
* **restapi:** recategorize the restapi component as a generic component
([#249](#249))
([fbfc3a3](fbfc3a3))
* **website:** add scrape sitemap function
([#239](#239))
([8648326](8648326))


### Bug Fixes

* bug of duplicate document
([#256](#256))
([e028a6e](e028a6e))
* bug of json without setting array for images
([#259](#259))
([4aeae69](4aeae69))
* change md format to html tag for correct frontend link
([#240](#240))
([7e16b2b](7e16b2b))
* revert the alias because they are same as package name
([#243](#243))
([1d9c42d](1d9c42d))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: 👋 Done
4 participants