Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

APIs for metadata #273

Merged
merged 5 commits into from
Jan 7, 2024
Merged

APIs for metadata #273

merged 5 commits into from
Jan 7, 2024

Conversation

loicknuchel
Copy link
Contributor

@loicknuchel loicknuchel commented Jan 6, 2024

Summary by CodeRabbit

  • New Features

    • Enhanced map manipulation with new utility functions.
    • Simplified API route paths for easier access.
    • Added endpoints for managing project metadata.
    • Introduced JSON schema validation for API requests.
  • Refactor

    • Streamlined JSON schema validation with the new JsonSchema module.
    • Updated project, table, and column metadata structures.
  • Documentation

    • Updated Swagger definitions for clearer API documentation.
  • Tests

    • Added new tests for map utility functions to ensure reliability.
  • Chores

    • Removed outdated API versioning from endpoint paths.

Copy link

coderabbitai bot commented Jan 6, 2024

Walkthrough

The recent updates to the Azimutt application focus on enhancing map manipulation utilities and refining the API's structure and functionality. New map manipulation functions have been introduced, providing more flexibility in working with maps. The API endpoints have undergone path simplifications and the removal of versioning prefixes. Enhanced JSON schema validation is now in place, and the handling of project metadata has been expanded, including Swagger documentation for better API clarity.

Changes

File Path Change Summary
.../azimutt/utils/mapx.ex
.../azimutt_web/utils/json_schema.ex
Introduces new map manipulation functions and JSON schema validation module.
.../azimutt_web/controllers/api/...
.../azimutt_web/router.ex
Updates API endpoint paths, adds metadata management functionality, and introduces new routes with updated base path for API versioning.
.../azimutt_web/utils/project_schema.ex Adds metadata structure definitions and accessor functions.
.../azimutt_web/views/api/metadata_view.ex Provides functionality for rendering metadata in JSON.
.../azimutt/utils/mapx_test.exs Adds tests for new map manipulation functions.

🐇✨
To the maps and routes we hop and bound,
Through the schema fields, our data is found.
With metadata rich, our projects gleam,
In Azimutt's realm, we code and dream.
🌟📈

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit-tests for this file.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit tests for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository from git and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit tests.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • The JSON schema for the configuration file is available here.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/coderabbit-overrides.v2.json

CodeRabbit Discord Community

Join our Discord Community to get help, request features, and share feedback.

@loicknuchel loicknuchel marked this pull request as ready for review January 7, 2024 08:09
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 3

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between aa778d7 and de94b73.
Files selected for processing (10)
  • backend/lib/azimutt/utils/mapx.ex (2 hunks)
  • backend/lib/azimutt_web/controllers/api/gallery_controller.ex (1 hunks)
  • backend/lib/azimutt_web/controllers/api/metadata_controller.ex (1 hunks)
  • backend/lib/azimutt_web/controllers/api/project_controller.ex (2 hunks)
  • backend/lib/azimutt_web/controllers/api/source_controller.ex (4 hunks)
  • backend/lib/azimutt_web/router.ex (2 hunks)
  • backend/lib/azimutt_web/utils/json_schema.ex (1 hunks)
  • backend/lib/azimutt_web/utils/project_schema.ex (2 hunks)
  • backend/lib/azimutt_web/views/api/metadata_view.ex (1 hunks)
  • backend/test/azimutt/utils/mapx_test.exs (1 hunks)
Additional comments: 19
backend/lib/azimutt_web/views/api/metadata_view.ex (3)
  • 5-7: The render function for "index.json" in AzimuttWeb.Api.MetadataView directly returns the metadata without any transformation or additional structure. Ensure this is the intended behavior and that the client expects the metadata in this raw format.

  • 9-15: The render function for "table.json" in AzimuttWeb.Api.MetadataView conditionally deletes the "columns" key from the metadata map based on the expand parameter. This is a good use of conditional logic to control the shape of the response based on the client's request.

  • 18-20: The render function for "column.json" in AzimuttWeb.Api.MetadataView directly returns the metadata without any transformation or additional structure. Ensure this is the intended behavior and that the client expects the metadata in this raw format.

backend/test/azimutt/utils/mapx_test.exs (2)
  • 28-32: The tests for the put_in function in Azimutt.Utils.MapxTest are correctly checking the behavior of the function when manipulating nested maps. It's good to see both a case where the nested structure already exists and one where it needs to be created.

  • 35-39: The tests for the update_in function in Azimutt.Utils.MapxTest are correctly checking the behavior of the function when updating nested maps. The use of the Map.merge function within the update_in call is a good way to ensure that the existing structure is preserved while updating.

backend/lib/azimutt/utils/mapx.ex (2)
  • 60-72: The put_in/3 function in Azimutt.Utils.Mapx is a custom implementation that does not raise an error when keys are missing. This is a deviation from the standard Kernel.put_in/3 behavior. Ensure that this behavior is documented and understood by the team to prevent confusion.

  • 74-88: The update_in/3 function in Azimutt.Utils.Mapx is a custom implementation that handles missing keys differently than Kernel.update_in/3. It's important to ensure that the team is aware of this custom behavior and that it's covered by tests.

backend/lib/azimutt_web/controllers/api/project_controller.ex (2)
  • 12-12: The route path for the index action in the AzimuttWeb.Api.ProjectController module has been updated to remove the /api/v1 prefix. This change should be cross-verified with the frontend and any other clients to ensure they are updated to use the new route.

  • 39-39: The route path for the create action in the AzimuttWeb.Api.ProjectController module has been updated to remove the /api/v1 prefix. This change should be cross-verified with the frontend and any other clients to ensure they are updated to use the new route.

Verification successful

The executed scripts did not find any occurrences of the old route /api/v1/organizations/:organization_id/projects in the frontend files, configuration files, documentation, or test files within the codebase. This suggests that there are no references to the outdated route that would need to be updated in response to the changes made in the backend.

Scripts Executed

The following scripts were executed for the analysis:


Script:

#!/bin/bash
# Verify that the frontend and any other clients are updated to use the new route.
# This is a placeholder for the actual verification steps, which would depend on the specific project setup.

Length of output: 0



Script:

#!/bin/bash
# Search for the old route in frontend files
rg "/api/v1/organizations/:organization_id/projects"

# Search for the old route in configuration files
rg "/api/v1/organizations/:organization_id/projects" --glob '*.{json,yml,yaml,config,ini}'

# Search for the old route in documentation
rg "/api/v1/organizations/:organization_id/projects" --glob '*.{md,txt}'

# Search for the old route in test files
rg "/api/v1/organizations/:organization_id/projects" --glob '*_test.exs'

Length of output: 289

backend/lib/azimutt_web/controllers/api/source_controller.ex (1)
  • 9-9: The introduction of the JsonSchema module alias in AzimuttWeb.Api.SourceController is a good refactor, centralizing JSON schema validation logic. Ensure that all instances of validate_json_schema have been replaced with JsonSchema.validate.
backend/lib/azimutt_web/utils/project_schema.ex (1)
  • 6-28: The addition of @column_meta, @table_meta, and @metadata module attributes in AzimuttWeb.Utils.ProjectSchema defines metadata structures for columns, tables, and general project metadata. Ensure that these structures are in sync with the frontend and other parts of the application that may use them.
backend/lib/azimutt_web/controllers/api/metadata_controller.ex (6)
  • 13-27: The swagger_path declaration for the index action in AzimuttWeb.Api.MetadataController is well-documented with a summary, description, and response codes. Ensure that the Swagger definitions are up-to-date and reflect the actual API behavior.

  • 35-50: The swagger_path declaration for the update action in AzimuttWeb.Api.MetadataController is well-documented with a summary, description, and response codes. Ensure that the Swagger definitions are up-to-date and reflect the actual API behavior.

  • 58-74: The swagger_path declaration for the table action in AzimuttWeb.Api.MetadataController is well-documented with a summary, description, and response codes. Ensure that the Swagger definitions are up-to-date and reflect the actual API behavior.

  • 82-98: The swagger_path declaration for the table_update action in AzimuttWeb.Api.MetadataController is well-documented with a summary, description, and response codes. Ensure that the Swagger definitions are up-to-date and reflect the actual API behavior.

  • 120-136: The swagger_path declaration for the column action in AzimuttWeb.Api.MetadataController is well-documented with a summary, description, and response codes. Ensure that the Swagger definitions are up-to-date and reflect the actual API behavior.

  • 144-161: The swagger_path declaration for the column_update action in AzimuttWeb.Api.MetadataController is well-documented with a summary, description, and response codes. Ensure that the Swagger definitions are up-to-date and reflect the actual API behavior.

backend/lib/azimutt_web/router.ex (2)
  • 207-214: The addition of new routes for metadata handling is a significant change. Ensure that the corresponding controller actions (index, update, table, table_update, column, column_update) are implemented and handle the expected parameters correctly. Also, verify that the commented-out delete routes are intentionally left for future implementation or should be removed if not needed.

  • 271-271: The swagger_info function now includes a basePath with the value "/api/v1". This change suggests that the API versioning has been centralized here. Confirm that this change is consistent with the intended API versioning strategy and that all necessary client-side adjustments have been made to accommodate this new base path.

Verification successful

The basePath: "/api/v1" setting in the swagger_info function is consistently used within the router.ex file, and there are no conflicting API versioning paths present. This suggests that the API versioning has been centralized as intended.

Scripts Executed

The following scripts were executed for the analysis:


Script:

#!/bin/bash
# Verify that the new base path is consistently used across the application.
rg --type elixir 'basePath: "/api/v1"'

Length of output: 101



Script:

#!/bin/bash
# Check if the routes in `router.ex` are prefixed with `/api/v1`
rg --type elixir 'scope "/api/v1"' backend/lib/azimutt_web/router.ex

# Search for other API versioning patterns to ensure there's no conflicting versioning
rg --type elixir 'scope "/api/v' backend/lib/azimutt_web/router.ex

Length of output: 300

backend/lib/azimutt_web/utils/json_schema.ex Show resolved Hide resolved
backend/lib/azimutt_web/utils/project_schema.ex Outdated Show resolved Hide resolved
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between de94b73 and 1fe10d0.
Files selected for processing (5)
  • backend/lib/azimutt/projects.ex (1 hunks)
  • backend/lib/azimutt_web/controllers/api/metadata_controller.ex (1 hunks)
  • backend/lib/azimutt_web/router.ex (3 hunks)
  • backend/lib/azimutt_web/utils/swagger_common.ex (1 hunks)
  • backend/lib/azimutt_web/views/api/metadata_view.ex (1 hunks)
Files skipped from review as they are similar to previous changes (3)
  • backend/lib/azimutt_web/controllers/api/metadata_controller.ex
  • backend/lib/azimutt_web/router.ex
  • backend/lib/azimutt_web/views/api/metadata_view.ex
Additional comments: 4
backend/lib/azimutt_web/utils/swagger_common.ex (3)
  • 6-7: The authorization function correctly adds an "auth-token" header parameter to the Swagger path object. This is a common practice for API authentication.

  • 10-10: The project_path function provides a path helper that includes path parameters for organization and project IDs. This is a useful utility for DRYing up Swagger path definitions.

  • 12-15: The project_params function correctly adds parameters for "organization_id" and "project_id" to the path object, including UUID validation. This ensures that the Swagger documentation will correctly reflect the required path parameters for endpoints dealing with projects.

backend/lib/azimutt/projects.ex (1)
  • 150-157: The update_project_content/4 function follows a clear and logical flow, using Elixir's with statement to chain operations that may fail. It retrieves a project, gets its content, decodes the JSON, applies a transformation function, encodes the JSON back, and updates the project file. If all steps succeed, it returns the updated JSON. This function should be tested to ensure that each step handles errors appropriately and that the transformation function f is always valid.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between 1fe10d0 and 4959d6e.
Files selected for processing (11)
  • backend/lib/azimutt/accounts/user_auth_token.ex (2 hunks)
  • backend/lib/azimutt_web/controllers/api/gallery_controller.ex (1 hunks)
  • backend/lib/azimutt_web/controllers/api/metadata_controller.ex (1 hunks)
  • backend/lib/azimutt_web/controllers/api/organization_controller.ex (2 hunks)
  • backend/lib/azimutt_web/controllers/api/project_controller.ex (3 hunks)
  • backend/lib/azimutt_web/controllers/api/source_controller.ex (8 hunks)
  • backend/lib/azimutt_web/controllers/api/user_controller.ex (1 hunks)
  • backend/lib/azimutt_web/controllers/user_auth.ex (2 hunks)
  • backend/lib/azimutt_web/router.ex (2 hunks)
  • backend/lib/azimutt_web/utils/project_schema.ex (2 hunks)
  • backend/lib/azimutt_web/utils/swagger_common.ex (1 hunks)
Files skipped from review as they are similar to previous changes (5)
  • backend/lib/azimutt_web/controllers/api/metadata_controller.ex
  • backend/lib/azimutt_web/controllers/api/project_controller.ex
  • backend/lib/azimutt_web/router.ex
  • backend/lib/azimutt_web/utils/project_schema.ex
  • backend/lib/azimutt_web/utils/swagger_common.ex
Additional comments: 19
backend/lib/azimutt_web/controllers/api/gallery_controller.ex (1)
  • 1-8: Please confirm that the removal of the swagger_path declaration for the "index" action is intentional and that the API documentation is updated accordingly to reflect this change.
backend/lib/azimutt/accounts/user_auth_token.ex (1)
  • 7-7: The addition of the Azimutt.Utils.Uuid alias and the is_valid?/1 function for UUID validation is a good enhancement for input validation.
backend/lib/azimutt_web/controllers/api/user_controller.ex (3)
  • 4-4: The addition of AzimuttWeb.Utils.SwaggerCommon alias supports the new Swagger documentation.

  • 7-16: Swagger documentation for the /users/current endpoint is well-defined and includes appropriate responses.

  • 23-42: The swagger_definitions for the User schema are comprehensive and provide clear examples for each field.

backend/lib/azimutt_web/controllers/api/organization_controller.ex (3)
  • 10-10: The addition of AzimuttWeb.Utils.SwaggerCommon alias supports the new Swagger documentation.

  • 13-22: Swagger documentation for the index action is well-defined and includes appropriate responses.

  • 56-77: The swagger_definitions for the Organization and Organizations schemas are comprehensive and provide clear examples for each field.

backend/lib/azimutt_web/controllers/user_auth.ex (2)
  • 8-8: The addition of the Azimutt.Accounts.UserAuthToken alias supports the new token validation functionality.

  • 129-135: The modification to fetch_current_user to include a check for the validity of the authentication token is a good security practice.

backend/lib/azimutt_web/controllers/api/source_controller.ex (9)
  • 9-11: The addition of AzimuttWeb.Utils.JsonSchema and AzimuttWeb.Utils.SwaggerCommon aliases supports the new Swagger documentation and JSON schema validation functionality.

  • 14-24: Swagger documentation for the index action is well-defined and includes appropriate responses.

  • 35-49: Swagger documentation for the show action is comprehensive and follows the Swagger specification.

  • 61-88: Swagger documentation for the create action is detailed and provides a clear structure for the expected request body.

  • 110-110: The replacement of validate_json_schema with JsonSchema.validate in the create action is a good refactor, centralizing JSON schema validation.

  • 122-148: Swagger documentation for the update action is well-structured and provides a clear schema for the request body.

  • 167-167: The replacement of validate_json_schema with JsonSchema.validate in the update action is consistent with the changes in the create action.

  • 179-193: Swagger documentation for the delete action is clear and includes the expected responses.

  • 225-409: The swagger_definitions function is extensive and provides a comprehensive set of schemas for source-related entities.

@loicknuchel loicknuchel merged commit 71d752e into main Jan 7, 2024
2 checks passed
@loicknuchel loicknuchel deleted the metadata-apis branch January 7, 2024 14:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant