Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: set up FOCA #13

Merged
merged 44 commits into from
Aug 19, 2024
Merged

chore: set up FOCA #13

merged 44 commits into from
Aug 19, 2024

Conversation

psankhe28
Copy link
Collaborator

@psankhe28 psankhe28 commented Aug 6, 2024

Description

Checklist

  • My code follows the contributing guidelines
    of this project, including, in particular, with regard to any style guidelines
  • The title of my PR complies with the
    Conventional Commits specification; in particular, it clearly
    indicates that a change is a breaking change
  • I acknowledge that all my commits will be squashed into a single commit,
    using the PR title as the commit message
  • I have performed a self-review of my own code
  • I have commented my code in hard-to-understand areas
  • I have updated the user-facing documentation to describe any new or
    changed behavior
  • I have added type annotations for all function/class/method interfaces
    or updated existing ones (only for Python, TypeScript, etc.)
  • I have provided appropriate documentation
    (Google-style Python docstrings) for all
    packages/modules/functions/classes/methods or updated existing ones
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature
    works
  • New and existing unit tests pass locally with my changes
  • I have not reduced the existing code coverage

Comments

Summary by Sourcery

Add configuration and OpenAPI specification files for the TUS storage handler, enabling file handling using TUS and Minio (S3 Storage).

New Features:

  • Introduce a new configuration file for the TUS storage handler, including server, security, API specifications, and exception handling settings.
  • Add an OpenAPI specification file for the TUS storage handler API, detailing endpoints, responses, and metadata.

Copy link
Contributor

sourcery-ai bot commented Aug 6, 2024

Reviewer's Guide by Sourcery

This pull request introduces the initial setup for the Tus Storage Handler API. It includes configuration settings in 'config.yaml' and an OpenAPI specification in 'storage_handler.yaml'. The configuration file sets up server parameters, security settings, API specifications, and exception handling. The OpenAPI file defines the API's metadata and a basic GET endpoint that returns a welcome message.

File-Level Changes

Files Changes
tus_storagehandler/config.yaml
tus_storagehandler/storage_handler.yaml
Introduced configuration and OpenAPI specification files for the new Tus Storage Handler API, including server settings, security configurations, API metadata, and a basic GET endpoint.

Tips
  • Trigger a new Sourcery review by commenting @sourcery-ai review on the pull request.
  • Continue your discussion with Sourcery by replying directly to review comments.
  • You can change your review settings at any time by accessing your dashboard:
    • Enable or disable the Sourcery-generated pull request summary or reviewer's guide;
    • Change the review language;
  • You can always contact us if you have any questions or feedback.

@psankhe28 psankhe28 requested a review from uniqueg August 6, 2024 19:02
Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @psankhe28 - I've reviewed your changes - here's some feedback:

Overall Comments:

  • The PR title 'feat: added foca' doesn't accurately describe the changes. Consider updating it to reflect the addition of configuration files for Tus Storage Handler.
  • Several checklist items are unchecked, including adding tests, updating documentation, and adding type annotations. Please address these points or explain why they're not applicable.
  • The PR adds configuration files but lacks implementation code. Is this intentional, or is there more code to be added in this feature?
Here's what I looked at during the review
  • 🟡 General issues: 2 issues found
  • 🟡 Security: 2 issues found
  • 🟢 Testing: all looks good
  • 🟢 Complexity: all looks good
  • 🟢 Documentation: all looks good

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment to tell me if it was helpful.

tus_storagehandler/config.yaml Outdated Show resolved Hide resolved
tus_storagehandler/config.yaml Outdated Show resolved Hide resolved
tus_storagehandler/storage_handler.yaml Outdated Show resolved Hide resolved
tus_storagehandler/storage_handler.yaml Outdated Show resolved Hide resolved
psankhe28 and others added 2 commits August 7, 2024 00:34
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
Copy link

codecov bot commented Aug 6, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (0516517) to head (83cddfc).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff            @@
##              main       #13   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            2         3    +1     
  Lines            8        23   +15     
=========================================
+ Hits             8        23   +15     
Flag Coverage Δ
test_integration 100.00% <100.00%> (ø)
test_unit 100.00% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Prati28 <[email protected]>
@psankhe28 psankhe28 changed the title feat: added foca feat: add foca for tus-storagehandler Aug 7, 2024
Copy link
Member

@uniqueg uniqueg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty good for a start :)

However, I think for "adding FOCA" we want the result to be a service that fires up and is available via Swagger UI etc. Of course no controllers need to be implemented (we just need one stub/dummy) - but we want to be sure that everything is in place to implement the controllers next.

To do that (apart from the individual comments in the code), please also add

  • The latest FOCA version as a dependency to pyproject.toml
  • A stub for the home controller for the GET / operation at tus_storagehandler/api/elixircloud/cloud_storage_handler/controllers.py (or replace cloud_storage_handler with csh); note that that FOCA looks for the controller for a given operation by putting together the value of x-openapi-router-controller (set in the api section in config.yaml; see my comment there) and the operationId from the specs). It needs to be either a class or a function. In our case, I think a function will do. Just have it return a 200 response for your integration test (see next point). Don't forget to add __init__.py to each of the new (sub)packages you create, i.e., api, elixircloud, cloud_storage_handler (or csh).
  • An API server entry point in tus_storagehandler/app.py similar to this one
  • An integration/end-to-end test for your service. Just write a module test_operations.py in tests/test_integration that uses pytest and requests to send a request to GET / and asserts that the response is 200.
  • A way of firing up the service in the CI workflow before the integration test runs. The most robust is to write a simple deployment/Dockerfile similar to this one and a deployment/docker-compose.yaml file similar to this one (you can omit the mongodb service).

tus_storagehandler/storage_handler.yaml Outdated Show resolved Hide resolved
tus_storagehandler/storage_handler.yaml Outdated Show resolved Hide resolved
tus_storagehandler/storage_handler.yaml Outdated Show resolved Hide resolved
tus_storagehandler/storage_handler.yaml Outdated Show resolved Hide resolved
tus_storagehandler/storage_handler_policies.conf Outdated Show resolved Hide resolved
tus_storagehandler/config.yaml Outdated Show resolved Hide resolved
tus_storagehandler/config.yaml Outdated Show resolved Hide resolved
@uniqueg
Copy link
Member

uniqueg commented Aug 7, 2024

Looks pretty good for a start :)

However, I think for "adding FOCA" we want the result to be a service that fires up and is available via Swagger UI etc. Of course no controllers need to be implemented (we just need one stub/dummy) - but we want to be sure that everything is in place to implement the controllers next.

To do that (apart from the individual comments in the code), please also add

  • The latest FOCA version as a dependency to pyproject.toml
  • A stub for the home controller for the GET / operation at tus_storagehandler/api/elixircloud/cloud_storage_handler/controllers.py (or replace cloud_storage_handler with csh); note that that FOCA looks for the controller for a given operation by putting together the value of x-openapi-router-controller (set in the api section in config.yaml; see my comment there) and the operationId from the specs). It needs to be either a class or a function. In our case, I think a function will do. Just have it return a 200 response for your integration test (see next point). Don't forget to add __init__.py to each of the new (sub)packages you create, i.e., api, elixircloud, cloud_storage_handler (or csh).
  • An API server entry point in tus_storagehandler/app.py similar to this one
  • An integration/end-to-end test for your service. Just write a module test_operations.py in tests/test_integration that uses pytest and requests to send a request to GET / and asserts that the response is 200.
  • A way of firing up the service in the CI workflow before the integration test runs. The most robust is to write a simple deployment/Dockerfile similar to this one and a deployment/docker-compose.yaml file similar to this one (you can omit the mongodb service).

Actually, it would be nice to write a small tutorial on that and add it to the FOCA docs. It might also be useful to perhaps optionally connect this with the Cookiecutter somehow, @JaeAeich - as this is really all just boilerplate/always the same.

@uniqueg uniqueg changed the title feat: add foca for tus-storagehandler chore: set up FOCA Aug 7, 2024
@uniqueg uniqueg mentioned this pull request Aug 7, 2024
Signed-off-by: Prati28 <[email protected]>
Signed-off-by: Prati28 <[email protected]>
Signed-off-by: Prati28 <[email protected]>
Signed-off-by: Prati28 <[email protected]>
Signed-off-by: Prati28 <[email protected]>
Signed-off-by: Prati28 <[email protected]>
Copy link
Collaborator

@JaeAeich JaeAeich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@psankhe28 IK you just asked me help with your failing test, but I had to make below some of below changes to make them run on my local machine. I understand this PR might not have been ready for a review but while checking your code, I just noticed couple of things which I found could be worth looking into so I commented so they don't go unnoticed. But to narrow down why tests are failing:

  • Wrong placement of config.yaml and tus_storagehandler.yaml
  • Your test need the server to be running.

deployment/Dockerfile Outdated Show resolved Hide resolved
deployment/Dockerfile Outdated Show resolved Hide resolved
deployment/Dockerfile Outdated Show resolved Hide resolved
deployment/Dockerfile Outdated Show resolved Hide resolved
deployment/Dockerfile Outdated Show resolved Hide resolved
tus_storagehandler/app.py Show resolved Hide resolved
tus_storagehandler/config.yaml Outdated Show resolved Hide resolved
tus_storagehandler/storage_handler.yaml Outdated Show resolved Hide resolved
tus_storagehandler/app.py Outdated Show resolved Hide resolved
tests/test_integration/test_operations.py Outdated Show resolved Hide resolved
@JaeAeich
Copy link
Collaborator

Documentation check is also failing, please run make d when you are finished with the this PR and that should pass that CI. Also maybe don't add docker and docker-compose yet, we could make a template for it and then use it, if its necessary rn then sure go ahead. Just a thought :).

Signed-off-by: Prati28 <[email protected]>
Signed-off-by: Prati28 <[email protected]>
Signed-off-by: Prati28 <[email protected]>
Signed-off-by: Prati28 <[email protected]>
Signed-off-by: Prati28 <[email protected]>
@psankhe28 psankhe28 requested review from JaeAeich and uniqueg August 10, 2024 20:05
Signed-off-by: Prati28 <[email protected]>
Copy link
Member

@uniqueg uniqueg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks mostly good. Please address the integration test comment, then I think we're good to go

pyproject.toml Outdated Show resolved Hide resolved
Signed-off-by: Pratiksha Sankhe <[email protected]>
Signed-off-by: Pratiksha Sankhe <[email protected]>
@psankhe28 psankhe28 requested a review from uniqueg August 17, 2024 10:25
@JaeAeich
Copy link
Collaborator

Not a review, but I was testing some templating for CI which required me running precommit in this repo, you're PR should fail pre-commit checks, please configure the repository to require it running all the workflows defined. pre-commit check is in pr-evaluation.yaml. You can check the error by running make pc in the root repo.

@JaeAeich
Copy link
Collaborator

@psankhe28 poetry add types-requests --group=types

Signed-off-by: Pratiksha Sankhe <[email protected]>
@psankhe28
Copy link
Collaborator Author

psankhe28 commented Aug 19, 2024

Not a review, but I was testing some templating for CI which required me running precommit in this repo, you're PR should fail pre-commit checks, please configure the repository to require it running all the workflows defined. pre-commit check is in pr-evaluation.yaml. You can check the error by running make pc in the root repo.

There is no pr-evaluation.yaml. Do you mean pr-validation.yaml?
In pr-validation.yaml, there is already pre-commit checks.

Copy link
Member

@uniqueg uniqueg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Getting really close...

.github/workflows/run-application.yaml Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Show resolved Hide resolved
Signed-off-by: Pratiksha Sankhe <[email protected]>
@psankhe28 psankhe28 requested a review from uniqueg August 19, 2024 17:27
Copy link
Collaborator Author

@psankhe28 psankhe28 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done with changes

Copy link
Member

@uniqueg uniqueg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have marked the locations that need changes to remove support for ENVIRONMENT. Please double check them and resolve the issues, then also resolve the previous conversation about this topic and finally tag me for a review 🙏

tus_storagehandler/app.py Outdated Show resolved Hide resolved
tus_storagehandler/app.py Outdated Show resolved Hide resolved
tus_storagehandler/app.py Outdated Show resolved Hide resolved
tus_storagehandler/app.py Outdated Show resolved Hide resolved
tus_storagehandler/app.py Outdated Show resolved Hide resolved
tus_storagehandler/app.py Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
tus_storagehandler/app.py Outdated Show resolved Hide resolved
tus_storagehandler/app.py Outdated Show resolved Hide resolved
tus_storagehandler/app.py Outdated Show resolved Hide resolved
tus_storagehandler/app.py Outdated Show resolved Hide resolved
Signed-off-by: Pratiksha Sankhe <[email protected]>
Signed-off-by: Pratiksha Sankhe <[email protected]>
README.md Outdated Show resolved Hide resolved
@uniqueg uniqueg merged commit cb97130 into main Aug 19, 2024
12 checks passed
@uniqueg uniqueg deleted the foca-structure branch August 19, 2024 18:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

feat: exceptions in case of errors feat: integration of foca
3 participants