Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: use schema versioning for import/export features #1475

Merged
merged 19 commits into from
Feb 11, 2025

Conversation

nas-tabchiche
Copy link
Contributor

@nas-tabchiche nas-tabchiche commented Feb 6, 2025

Summary by CodeRabbit

  • New Features
    • Introduced schema versioning for file uploads and data backups, ensuring that backups now include version metadata for enhanced compatibility.
    • Added error logging for server response data during backup file uploads to improve debugging.
    • Enhanced version comparison functionality, allowing for more robust validation of schema versions.
    • Added a new field for schema_version in the metadata structure for serialized backup data.
  • Bug Fixes
    • Streamlined version validation with improved error messaging for backup restoration and file uploads, offering clearer feedback when files are incompatible with the current system.
    • Enhanced error handling for invalid version formats and comparisons, standardizing the validation process.

Copy link

coderabbitai bot commented Feb 6, 2025

Walkthrough

This pull request enhances schema and version handling across the project. Key changes include improved version validation during file uploads and backup restorations, the introduction of utilities for parsing and comparing version strings, and the addition of a custom exception (VersionFormatError) for managing version formatting errors. The update removes regex-based checks in favor of streamlined logic using the compare_schema_versions function, while also integrating schema version logging and configuration updates in settings and serializers. Test coverage has been expanded to ensure robust validation of the new functionality.

Changes

File(s) Summary
backend/core/views.py, backend/serdes/views.py Enhanced file upload and backup restoration processes by integrating schema version checks, logging, and streamlined version comparison logic (using compare_schema_versions) while removing regex validations and dev-version handling.
backend/core/utils.py, backend/core/tests/test_helpers.py Added VersionFormatError, parse_version, and compare_versions functions with comprehensive tests for version parsing, comparison, and error handling; replaced wildcard imports with explicit ones.
backend/ciso_assistant/settings.py, enterprise/backend/enterprise_core/settings.py, backend/ciso_assistant/meta.py Introduced SCHEMA_VERSION sourced from the meta module, added logging for schema version, and included new configuration variables to standardize version handling.
backend/serdes/serializers.py Added schema_version field in MetaSerializer and updated export logic in ExportSerializer to incorporate schema version into backup data.
frontend/src/routes/(app)/(internal)/backup-restore/+page.server.ts Added a console error log to capture and display error responses during backup file processing.

Sequence Diagram(s)

sequenceDiagram
    participant Client as Client
    participant View as API Handler
    participant Util as Version Utils
    participant Logger as Logger

    Client->>View: Upload file with (optional) schema_version
    alt schema_version present
        View->>Logger: Log uploaded schema_version
        View->>Util: compare_schema_versions(uploaded, current)
        alt Incompatible version
            Util-->>View: Raise ValidationError
            View->>Client: Return error response
        else Compatible version
            Util-->>View: Return success
            View->>Client: Process file normally
        end
    else Legacy file
        View->>Util: Use legacy version comparison logic
        View->>Client: Process file normally
    end
Loading

Possibly related PRs

Suggested reviewers

  • ab-smith
  • Mohamed-Hacene

Poem

I'm a rabbit coding through the night,
Hopping over bugs with newfound light.
Schema and version now dance in tune,
Logging their steps beneath the moon.
Together we hop, our code so sleek—
A joyful leap for every geek!

✨ Finishing Touches
  • 📝 Generate Docstrings (Beta)

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (2)
backend/core/utils.py (2)

6-6: Remove unused import.

The re module is imported but never used in the code.

-import re
🧰 Tools
🪛 Ruff (0.8.2)

6-6: re imported but unused

Remove unused import: re

(F401)


91-103: Improve exception handling by using raise ... from err.

The function is well-implemented, but the exception handling could be improved to preserve the exception chain.

     try:
         return [int(part) for part in parts]
     except ValueError as err:
-        raise VersionFormatError(f"Non-numeric version component in {version}")
+        raise VersionFormatError(f"Non-numeric version component in {version}") from err
🧰 Tools
🪛 Ruff (0.8.2)

103-103: Within an except clause, raise exceptions with raise ... from err or raise ... from None to distinguish them from errors in exception handling

(B904)

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 0753602 and 5be3c23.

📒 Files selected for processing (4)
  • backend/core/tests/test_helpers.py (2 hunks)
  • backend/core/utils.py (2 hunks)
  • backend/core/views.py (2 hunks)
  • backend/serdes/views.py (2 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • backend/serdes/views.py
🧰 Additional context used
🪛 Ruff (0.8.2)
backend/core/utils.py

6-6: re imported but unused

Remove unused import: re

(F401)


103-103: Within an except clause, raise exceptions with raise ... from err or raise ... from None to distinguish them from errors in exception handling

(B904)

⏰ Context from checks skipped due to timeout of 90000ms (9)
  • GitHub Check: startup-functional-test (3.12)
  • GitHub Check: startup-docker-compose-test
  • GitHub Check: functional-tests (3.12, chromium)
  • GitHub Check: enterprise-startup-functional-test (3.12)
  • GitHub Check: build (3.12)
  • GitHub Check: Analyze (python)
  • GitHub Check: enterprise-startup-docker-compose-test
  • GitHub Check: enterprise-functional-tests (3.12, chromium)
  • GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (4)
backend/core/utils.py (1)

106-157: LGTM! Well-documented and robust implementation.

The compare_versions function is well-implemented with:

  • Clear docstring and examples
  • Proper error handling
  • Flexible comparison levels (major, minor, patch)
  • Correct padding of version components
backend/core/tests/test_helpers.py (2)

159-180: LGTM! Comprehensive test coverage for parse_version.

The tests cover all important scenarios:

  • Single, two, and three component versions
  • Invalid version format (missing 'v')
  • Non-numeric components

187-240: LGTM! Excellent test coverage for compare_versions.

The tests are well-structured with:

  • Parameterized tests for multiple scenarios
  • Coverage of all comparison levels (major, minor, patch)
  • Edge cases with version padding
  • Invalid inputs
backend/core/views.py (1)

2270-2279: LGTM! Version comparison logic improved.

The changes correctly implement minor-level version comparison:

  • Uses the new compare_versions function
  • Clear error handling for incompatible versions
  • Aligns with PR objective to check only major and minor versions

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
backend/core/utils.py (1)

106-157: Add return type hint to compare_versions function.

The function has comprehensive documentation and examples but could benefit from an explicit return type hint.

 def compare_versions(
     version_a: str, version_b: str, level: Literal["major", "minor", "patch"] = "patch"
-) -> int:
+) -> Literal[-1, 0, 1]:
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 5be3c23 and ea9cfed.

📒 Files selected for processing (8)
  • backend/ciso_assistant/meta.json (1 hunks)
  • backend/ciso_assistant/settings.py (2 hunks)
  • backend/core/utils.py (2 hunks)
  • backend/core/views.py (4 hunks)
  • backend/serdes/serializers.py (2 hunks)
  • backend/serdes/views.py (2 hunks)
  • enterprise/backend/enterprise_core/settings.py (2 hunks)
  • frontend/src/routes/(app)/(internal)/backup-restore/+page.server.ts (1 hunks)
✅ Files skipped from review due to trivial changes (1)
  • backend/ciso_assistant/meta.json
🚧 Files skipped from review as they are similar to previous changes (2)
  • backend/serdes/views.py
  • backend/core/views.py
⏰ Context from checks skipped due to timeout of 90000ms (10)
  • GitHub Check: startup-functional-test (3.12)
  • GitHub Check: startup-docker-compose-test
  • GitHub Check: enterprise-startup-functional-test (3.12)
  • GitHub Check: functional-tests (3.12, chromium)
  • GitHub Check: build (3.12)
  • GitHub Check: enterprise-startup-docker-compose-test
  • GitHub Check: test (3.12)
  • GitHub Check: enterprise-functional-tests (3.12, chromium)
  • GitHub Check: Analyze (python)
  • GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (8)
frontend/src/routes/(app)/(internal)/backup-restore/+page.server.ts (1)

36-36: LGTM! Enhanced error logging.

The addition of error logging improves debugging capabilities for backup file upload issues.

backend/serdes/serializers.py (2)

40-40: LGTM! Added schema version support.

The schema_version field is properly integrated into the MetaSerializer.


121-121: LGTM! Schema version included in backup metadata.

The schema_version is correctly retrieved from settings and included in the backup metadata.

backend/core/utils.py (2)

85-88: LGTM! Well-defined custom exception.

The VersionFormatError exception is properly defined with a clear docstring.


91-103: LGTM! Robust version parsing.

The parse_version function includes proper error handling and clear docstring.

backend/ciso_assistant/settings.py (1)

110-110: LGTM! Added schema version logging.

The schema version is properly logged along with other configuration values.

enterprise/backend/enterprise_core/settings.py (2)

113-113: LGTM!

The logging statement is consistent with the existing logging pattern for version information.


14-14: Verify the path resolution logic.

The path resolution for COMMUNITY_BACKEND_DIR uses multiple .parent calls which can be fragile. Consider using a more robust path resolution method.

Run this script to verify the path resolution:

Also applies to: 22-22

✅ Verification successful

🏁 Script executed:

#!/bin/bash
# Description: Verify that the path resolution leads to the expected directory containing meta.json

# Test: Check if meta.json exists at the resolved path
python3 -c '
from pathlib import Path
import os
path = Path(__file__).resolve().parent.parent.parent.parent / "backend" / "ciso_assistant" / "meta.json"
print(f"Resolved path: {path}")
print(f"Path exists: {path.exists()}")
'

Length of output: 370


Verify the path resolution using a simulated file context.

The inline Python command previously failed because file isn’t defined when using Python’s “-c” flag. To mimic the behavior as if we were running the code from enterprise/backend/enterprise_core/settings.py, please run the following shell script. This script manually sets a fake file value to simulate the file’s location and then computes the resolved path for meta.json. This should help verify that the path resolution logic leads to the expected directory.

Please run this script and verify the output. If the resolved path is correct and points to the location where meta.json is expected, then the path resolution logic is working as intended. If not, further adjustments in the path construction may be required.


🏁 Script executed:

#!/bin/bash
# Adjusted script: simulate __file__ based on the known location of settings.py

python3 -c '
from pathlib import Path
# Simulate the location of "enterprise/backend/enterprise_core/settings.py"
fake_file = "enterprise/backend/enterprise_core/settings.py"
# Emulate the same parent traversal as in the production code
resolved_path = (Path(fake_file).resolve()).parent.parent.parent.parent / "backend" / "ciso_assistant" / "meta.json"
print(f"Simulated __file__ based path: {resolved_path}")
print("Path exists:", resolved_path.exists())
'

Length of output: 561


Path resolution verified.

The simulation confirms that the resolved path:

/home/jailuser/git/backend/ciso_assistant/meta.json

correctly points to an existing file. This indicates that the path traversal logic in the file is working as expected.

Comment on lines 30 to 33
META = {}
with open(BASE_DIR / "ciso_assistant" / "meta.json") as f:
META = json.load(f)
SCHEMA_VERSION = META.get("schema_version", "unset")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Add error handling for meta.json loading.

The code should handle cases where meta.json is missing or invalid.

 META = {}
-with open(BASE_DIR / "ciso_assistant" / "meta.json") as f:
-    META = json.load(f)
+meta_file = BASE_DIR / "ciso_assistant" / "meta.json"
+try:
+    with open(meta_file) as f:
+        META = json.load(f)
+except (FileNotFoundError, json.JSONDecodeError) as e:
+    logger.warning("Failed to load meta.json: %s", e)
 SCHEMA_VERSION = META.get("schema_version", "unset")
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
META = {}
with open(BASE_DIR / "ciso_assistant" / "meta.json") as f:
META = json.load(f)
SCHEMA_VERSION = META.get("schema_version", "unset")
META = {}
meta_file = BASE_DIR / "ciso_assistant" / "meta.json"
try:
with open(meta_file) as f:
META = json.load(f)
except (FileNotFoundError, json.JSONDecodeError) as e:
logger.warning("Failed to load meta.json: %s", e)
SCHEMA_VERSION = META.get("schema_version", "unset")

Comment on lines 29 to 32
META = {}
with open(COMMUNITY_BACKEND_DIR / "backend" / "ciso_assistant" / "meta.json") as f:
META = json.load(f)
SCHEMA_VERSION = META.get("schema_version", "unset")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Add error handling for meta.json loading.

The code should handle potential file access and JSON parsing errors gracefully.

Apply this diff to add error handling:

 META = {}
-with open(COMMUNITY_BACKEND_DIR / "backend" / "ciso_assistant" / "meta.json") as f:
-    META = json.load(f)
+try:
+    with open(COMMUNITY_BACKEND_DIR / "backend" / "ciso_assistant" / "meta.json") as f:
+        META = json.load(f)
+except (FileNotFoundError, json.JSONDecodeError, OSError) as e:
+    logger.error("Failed to load meta.json", error=str(e))
 SCHEMA_VERSION = META.get("schema_version", "unset")
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
META = {}
with open(COMMUNITY_BACKEND_DIR / "backend" / "ciso_assistant" / "meta.json") as f:
META = json.load(f)
SCHEMA_VERSION = META.get("schema_version", "unset")
META = {}
try:
with open(COMMUNITY_BACKEND_DIR / "backend" / "ciso_assistant" / "meta.json") as f:
META = json.load(f)
except (FileNotFoundError, json.JSONDecodeError, OSError) as e:
logger.error("Failed to load meta.json", error=str(e))
SCHEMA_VERSION = META.get("schema_version", "unset")

@nas-tabchiche nas-tabchiche force-pushed the fix/compare-major-minor-on-import branch from ea9cfed to e397228 Compare February 7, 2025 18:33
@nas-tabchiche nas-tabchiche changed the title fix: check only major and minor version when loading data fix: use schema versioning for import/export features Feb 7, 2025
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e397228 and 766392f.

📒 Files selected for processing (3)
  • backend/ciso_assistant/meta.py (1 hunks)
  • backend/ciso_assistant/settings.py (2 hunks)
  • enterprise/backend/enterprise_core/settings.py (2 hunks)
✅ Files skipped from review due to trivial changes (1)
  • backend/ciso_assistant/meta.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • backend/ciso_assistant/settings.py
🧰 Additional context used
🪛 Ruff (0.8.2)
enterprise/backend/enterprise_core/settings.py

14-14: json imported but unused

Remove unused import: json

(F401)

⏰ Context from checks skipped due to timeout of 90000ms (10)
  • GitHub Check: startup-functional-test (3.12)
  • GitHub Check: startup-docker-compose-test
  • GitHub Check: functional-tests (3.12, chromium)
  • GitHub Check: enterprise-startup-functional-test (3.12)
  • GitHub Check: enterprise-functional-tests (3.12, chromium)
  • GitHub Check: build (3.12)
  • GitHub Check: enterprise-startup-docker-compose-test
  • GitHub Check: test (3.12)
  • GitHub Check: Analyze (python)
  • GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (2)
enterprise/backend/enterprise_core/settings.py (2)

20-20: LGTM! Schema version configuration looks good.

The schema version is properly configured:

  • Imported from the meta module
  • Path configuration for community backend is correctly set
  • Schema version is retrieved from the meta module

Also applies to: 23-23, 29-29


109-109: LGTM! Schema version logging is properly integrated.

The schema version is now logged alongside other version information during startup, which will help with debugging and monitoring.

@@ -11,18 +11,22 @@

from pathlib import Path
import os
import json
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Remove unused import.

The json module is imported but not used in this file.

Apply this diff to remove the unused import:

-import json
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
import json
# (The unused json import has been removed.)
🧰 Tools
🪛 Ruff (0.8.2)

14-14: json imported but unused

Remove unused import: json

(F401)

Copy link
Collaborator

@eric-intuitem eric-intuitem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

COMMUNITY_BACKEND_DIR is defined but not used.
LGTM otherwise.

@nas-tabchiche
Copy link
Contributor Author

COMMUNITY_BACKEND_DIR is a relic from a previous iteration where I tried a meta.json file located in backend/ciso_assistant, before switching to a .py file which proved to be less of a headache because one could simply import it regardless of the deployment type (CE or EE). Removing.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (3)
backend/serdes/views.py (1)

11-11: Remove unused import.

The ValidationError import is not directly used in this file as it's raised by compare_schema_versions.

-from rest_framework.exceptions import ValidationError
🧰 Tools
🪛 Ruff (0.8.2)

11-11: rest_framework.exceptions.ValidationError imported but unused

Remove unused import: rest_framework.exceptions.ValidationError

(F401)

backend/core/utils.py (1)

218-218: Fix typo in error message.

There's a typo in the error message: "schena" should be "schema".

-                "Backup schema version greater than current schena version",
+                "Backup schema version greater than current schema version",
backend/core/views.py (1)

3016-3016: Add missing docstrings for functions.

The TODO comment indicates that proper docstrings are missing for several functions. This should be addressed to improve code maintainability and documentation.

Would you like me to help generate docstrings for these functions? I can analyze the code and propose appropriate documentation.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between fbbf25b and 7e015e9.

📒 Files selected for processing (4)
  • backend/core/tests/test_helpers.py (2 hunks)
  • backend/core/utils.py (2 hunks)
  • backend/core/views.py (4 hunks)
  • backend/serdes/views.py (3 hunks)
🧰 Additional context used
🪛 Ruff (0.8.2)
backend/serdes/views.py

11-11: rest_framework.exceptions.ValidationError imported but unused

Remove unused import: rest_framework.exceptions.ValidationError

(F401)

backend/core/tests/test_helpers.py

334-334: pytest.raises(Exception) should be considered evil

(B017)

⏰ Context from checks skipped due to timeout of 90000ms (10)
  • GitHub Check: startup-functional-test (3.12)
  • GitHub Check: startup-docker-compose-test
  • GitHub Check: functional-tests (3.12, chromium)
  • GitHub Check: enterprise-functional-tests (3.12, chromium)
  • GitHub Check: enterprise-startup-functional-test (3.12)
  • GitHub Check: enterprise-startup-docker-compose-test
  • GitHub Check: build (3.12)
  • GitHub Check: test (3.12)
  • GitHub Check: Analyze (python)
  • GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (9)
backend/serdes/views.py (2)

38-40: LGTM!

The backup metadata now includes both media and schema versions, which aligns with the PR's objective of implementing schema versioning.


123-133: LGTM!

The version validation logic has been improved by:

  1. Extracting both media and schema versions from metadata.
  2. Delegating version validation to the compare_schema_versions utility.
backend/core/utils.py (3)

97-109: LGTM!

The function is well-implemented with:

  1. Clear input validation
  2. Proper error handling
  3. Comprehensive docstring

112-163: LGTM!

The function is well-designed with:

  1. Input validation
  2. Efficient tuple comparison
  3. Proper version padding
  4. Comprehensive docstring with examples

166-246: LGTM!

The function is well-implemented with:

  1. Comprehensive schema version validation
  2. Fallback to semantic version comparison
  3. Proper error handling
  4. Detailed logging
  5. Comprehensive docstring with examples
backend/core/tests/test_helpers.py (1)

165-336: LGTM!

The test suite is well-designed with:

  1. Comprehensive test coverage for all version handling utilities
  2. Effective use of parameterization for multiple test cases
  3. Clear test descriptions and comments
  4. Edge case handling
🧰 Tools
🪛 Ruff (0.8.2)

334-334: pytest.raises(Exception) should be considered evil

(B017)

backend/core/views.py (3)

15-18: LGTM! Clean import statement focusing on email settings.

The imports have been streamlined to include only the required email-related settings.


84-88: LGTM! Proper import organization for schema version comparison.

The addition of compare_schema_versions import supports the new schema version validation functionality.


2306-2307: Verify schema version validation during file uploads.

The schema version validation has been added to ensure compatibility during file uploads. However, there are a few points to consider:

  1. The schema_version is optional (using .get()) but there's no explicit handling for when it's missing.
  2. The import_version is required but there's no error handling if it's missing from the meta section.

Run this script to check how other parts of the codebase handle missing schema versions:

Also applies to: 2315-2315

✅ Verification successful

🏁 Script executed:

#!/bin/bash
# Search for schema_version usage patterns
rg -A 5 'schema_version'

# Search for compare_schema_versions error handling
ast-grep --pattern 'try {
  $$$
  compare_schema_versions($_, $_)
  $$$
}'

Length of output: 11123


Schema version validation appears to be working as intended.

The tests in backend/core/tests/test_helpers.py confirm that both cases—when the backup schema version is present and when it’s absent—are properly handled by the compare_schema_versions function. In particular:

  • If the extracted schema_version is missing (i.e. None), the function uses the semantic version comparison fallback behavior (as tested by the “happy” and “unhappy” semver tests).
  • The extraction via json_dump["meta"].get("schema_version") intentionally makes the field optional, and downstream processing in compare_schema_versions handles the None case appropriately.

Regarding the concern over a missing import_version in the meta section: our search indicates that other parts of the codebase expect import_version to be set elsewhere before calling compare_schema_versions. There is no explicit error handling here because the validation logic in compare_schema_versions assumes that either a valid semantic version or a None is provided. Any missing or invalid import_version that causes a version comparison error is caught by the tests (for instance, raising a ValidationError with "importVersionNotCompatibleWithCurrentVersion").

This behavior is consistent with the established patterns elsewhere in the codebase. No additional error handling appears necessary based on the current test coverage and usage patterns.

"invalid_version" # This should trigger a formatting error in compare_versions.
)
version_b = "1.2.3"
with pytest.raises(Exception):
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Use specific exception type in pytest.raises.

Using a generic Exception in pytest.raises is considered bad practice as it can mask unexpected errors.

-    with pytest.raises(Exception):
+    with pytest.raises(VersionFormatError):
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
with pytest.raises(Exception):
with pytest.raises(VersionFormatError):
🧰 Tools
🪛 Ruff (0.8.2)

334-334: pytest.raises(Exception) should be considered evil

(B017)

backend/core/utils.py Outdated Show resolved Hide resolved
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (4)
backend/core/utils.py (2)

97-109: Add input validation for None or empty strings.

The function should validate that the input is not None or empty before processing.

 def parse_version(version: str) -> list[int]:
+    if not version:
+        raise VersionFormatError("Version string cannot be None or empty")
     if not version.startswith("v"):

166-252: Add return type hint.

The function should specify its return type for better type safety.

 def compare_schema_versions(
     schema_ver_a: int | None,
     version_a: str | None,
     version_b: str = settings.VERSION.split("-")[0],
     schema_ver_b: int = settings.SCHEMA_VERSION,
     level: Literal["major", "minor", "patch"] = "patch",
-):
+) -> None:
backend/core/tests/test_helpers.py (2)

165-186: Consider using parameterization for parse_version tests.

Consolidate the version parsing tests using pytest's parameterize decorator for better organization and maintainability.

-def test_parse_version_single_component():
-    assert parse_version("v1") == [1]
-
-def test_parse_version_two_components():
-    assert parse_version("v1.2") == [1, 2]
-
-def test_parse_version_three_components():
-    assert parse_version("v1.2.3") == [1, 2, 3]
-
-def test_parse_version_invalid_missing_v():
-    with pytest.raises(VersionFormatError) as exc_info:
-        parse_version("1.2.3")
-    assert "must start with 'v'" in str(exc_info.value)
-
-def test_parse_version_invalid_non_numeric():
-    with pytest.raises(VersionFormatError) as exc_info:
-        parse_version("v1.2.a")
-    assert "Non-numeric version component" in str(exc_info.value)
+@pytest.mark.parametrize(
+    "version, expected_result, expected_error",
+    [
+        ("v1", [1], None),
+        ("v1.2", [1, 2], None),
+        ("v1.2.3", [1, 2, 3], None),
+        ("1.2.3", None, "must start with 'v'"),
+        ("v1.2.a", None, "Non-numeric version component"),
+    ],
+)
+def test_parse_version(version, expected_result, expected_error):
+    if expected_error:
+        with pytest.raises(VersionFormatError) as exc_info:
+            parse_version(version)
+        assert expected_error in str(exc_info.value)
+    else:
+        assert parse_version(version) == expected_result

272-314: Consider parameterizing error test cases.

Consolidate the error test cases using pytest's parameterize decorator for better organization.

-def test_schema_version_mismatch():
-    schema_ver_a = 3
-    schema_ver_b = 2
-    version_a = "v1.2.3"
-    version_b = "v1.2.3"
-    with pytest.raises(ValidationError) as excinfo:
-        compare_schema_versions(schema_ver_a, version_a, version_b, schema_ver_b)
-    assert excinfo.value.args[0]["error"] == "backupGreaterVersionError"
-
-def test_semver_backup_greater():
-    schema_ver_a = None
-    version_a = "v1.3.0"
-    version_b = "v1.2.9"
-    with pytest.raises(ValidationError) as excinfo:
-        compare_schema_versions(schema_ver_a, version_a, version_b, schema_ver_b=1)
-    assert excinfo.value.args[0]["error"] == "backupGreaterVersionError"
-
-def test_semver_backup_less():
-    schema_ver_a = None
-    version_a = "v1.1.0"
-    version_b = "v1.2.0"
-    with pytest.raises(ValidationError) as excinfo:
-        compare_schema_versions(schema_ver_a, version_a, version_b, schema_ver_b=1)
-    assert excinfo.value.args[0]["error"] == "importVersionNotCompatibleWithCurrentVersion"
+@pytest.mark.parametrize(
+    "schema_ver_a, schema_ver_b, version_a, version_b, expected_error",
+    [
+        (3, 2, "v1.2.3", "v1.2.3", "backupGreaterVersionError"),
+        (None, 1, "v1.3.0", "v1.2.9", "backupGreaterVersionError"),
+        (None, 1, "v1.1.0", "v1.2.0", "importVersionNotCompatibleWithCurrentVersion"),
+    ],
+)
+def test_schema_version_errors(schema_ver_a, schema_ver_b, version_a, version_b, expected_error):
+    with pytest.raises(ValidationError) as excinfo:
+        compare_schema_versions(schema_ver_a, version_a, version_b, schema_ver_b)
+    assert excinfo.value.args[0]["error"] == expected_error
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 7e015e9 and d2bb4db.

📒 Files selected for processing (2)
  • backend/core/tests/test_helpers.py (2 hunks)
  • backend/core/utils.py (2 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (10)
  • GitHub Check: startup-functional-test (3.12)
  • GitHub Check: startup-docker-compose-test
  • GitHub Check: enterprise-startup-functional-test (3.12)
  • GitHub Check: functional-tests (3.12, chromium)
  • GitHub Check: enterprise-startup-docker-compose-test
  • GitHub Check: enterprise-functional-tests (3.12, chromium)
  • GitHub Check: build (3.12)
  • GitHub Check: test (3.12)
  • GitHub Check: Analyze (python)
  • GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (4)
backend/core/utils.py (2)

91-94: LGTM!

The exception class is well-defined and properly documented.


112-163: LGTM!

The function is well-implemented with:

  • Comprehensive documentation
  • Clear error handling
  • Efficient tuple comparison
  • Good examples in docstring
backend/core/tests/test_helpers.py (2)

193-246: LGTM!

The tests are well-organized with:

  • Effective use of parameterization
  • Comprehensive coverage of comparison scenarios
  • Clear test cases with good comments

334-334: Use specific exception type in pytest.raises.

Using a generic Exception in pytest.raises is considered bad practice as it can mask unexpected errors.

Copy link
Collaborator

@Mohamed-Hacene Mohamed-Hacene left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0k

@Mohamed-Hacene Mohamed-Hacene merged commit f4bcfcf into main Feb 11, 2025
19 of 20 checks passed
@Mohamed-Hacene Mohamed-Hacene deleted the fix/compare-major-minor-on-import branch February 11, 2025 07:19
@github-actions github-actions bot locked and limited conversation to collaborators Feb 11, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants