Support for cloud storage connectors

[Lekhanrao]

Ticket Contents

To develop connectors for AWS and GCP storage services

Goals

Currently we have Local storage and Azure connectors. We need to enable connectors for Storages services in Google Cloud (GCP) and Amazon Web Services (AWS), so that it will become possible to use alternate Storage options as per specific use case requirements.

Expected Outcome

The stack should have capabilities to store data in GCP and AWS by providing appropriate storage keys in the environment.

Acceptance Criteria

1. There should be two folders named gcp and aws under jb-lib/lib/file_storage folder and two implementations have to be written for sync and async storage for both GCP and AWS connectors. The current two folders (local and azure) can be used as reference for implementing them.
2. Appropriate test cases should also be written for these two new connectors and the tests should also follow the same existing directory structure as local and azure storage.

Implementation Details

The current implementation supports local storage and Azure storage. Write connectors for storage services in GCP and AWS by following the Azure storage connector file. The new files should be put under jb-lib/lib/file_storage folder.

Mockups/Wireframes

Product Name

Jugalbandi

Organisation Name

OpenNyAI

Domain

Legal

Tech Skills Needed

Python, Other

Mentor(s)

@Lekhanrao @KaranrajM

Complexity

Low

Category

Backend

[c4gt-community-support]

Hi!
Important Details - These following details are helpful for contributors to effectively identify and contribute to tickets.

• Sub-Category - Please mention the sub-category if any for the ticket

Please update the ticket

[Lekhanrao]

@c4gt-community-support, What are the sub-category types and where do we update that in the Issue template?

[shreyp135]

Hi @Lekhanrao,
I would like to work on this issue, so could you please assign it to me.
Thanks.

[Lekhanrao]

Assigned ticket to @shreyp135

[Alekh-Srivastava]

Hey @Lekhanrao I can also work on the AWS and GCP part, can I take up the issue as well?

[Blacksujit]

@Lekhanrao

hello , can you please assign the issue to me i can figure it out

[Lekhanrao]

@Alekh-Srivastava and @Blacksujit , assigned this ticket to you both as well. We will pick the best solution/code. Thanks!

[Lekhanrao]

@shreyp135, Please test the code with the Cloud keys for both GCP and AWS by uploading the file and reading the file.

[Lekhanrao]

@shreyp135, @Alekh-Srivastava and @Blacksujit, Please let us know if you have any update on this.

[shreyp135]

@shreyp135, @Alekh-Srivastava and @Blacksujit, Please let us know if you have any update on this.

Hi @Lekhanrao,
Sorry I got caught up with something else, I just have to resolve the issue in 2 tests. I'll try to update it by tomorrow.

[VedantKhairnar]

Hello @shreyp135,
Any update on the contribution?
Pls do let us know if you need any assistance.
Thanks.

[shreyp135]

Hello @shreyp135,
Any update on the contribution?
Pls do let us know if you need any assistance.
Thanks.

Hi @VedantKhairnar,
I am stuck at an issue, I have commented on the pull request #184.

[Blacksujit]

@Lekhanrao please assign this to someone else I am stuck with my college exams if I got free I will take the issue immediately sorry for inconvenience.

[Lekhanrao]

@shreyp135 and @KaranrajM , is there an update on this please?

[RishiikeshBalaji]

hi, can I look into this issue? @Lekhanrao

[legionJP]

Hi @Lekhanrao , I would like to contribute on this issue, so if the issue is still open you can assign it to me.
Thanks.

[Lekhanrao]

@legionJP, assigned ticket to you.

[Lekhanrao]

@RishiikeshBalaji, Assigned ticket to you as well. We will pick the best code! All the best!

[Lekhanrao]

Assigned to @ananyaagrawal23 and @Sunandhita

[Lekhanrao]

@KaranrajM , hello Karan, wanted to understand why the link to the Pull request was removed? So is this Issue #172 still open?

[Lekhanrao]

@ananyaagrawal23 and @Sunandhita , wanted to check if any work has started on this please?

[Sunandhita]

Code for this is already done Lekha.... we need the AWS keys to test the code, then we can push code... we are still waiting for the keys

[Sunandhita]

The connectors for GCP storage services has been developed, pushed and PR has been raised. The connectors for AWS storage service hasn't been pushed yet since the AWS keys weren't available

[Lekhanrao]

@Sunandhita is the PR for GCP completed?
And regarding AWS - I think our plan was to use the personal account to test and close this, isnt it?