Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: bitbucket ocean integration #1416

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

dev-habib-nuhu
Copy link

@dev-habib-nuhu dev-habib-nuhu commented Feb 20, 2025

User description

Description

What

  • This PR adds a Bitbucket Cloud integration for Port's Ocean framework, enabling users to ingest and manage Bitbucket projects, repositories, pull requests, and components in Port.

Why

How -

  • Uses Ocean’s async HTTP client for API calls.
  • Implements Bitbucket API authentication via App Passwords.
  • Supports resync and scheduled sync for accurate data ingestion.
  • Enables webhook event handling for real-time updates.

Type of change

Please leave one option from the following and delete the rest:

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • New Integration (non-breaking change which adds a new integration)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Non-breaking change (fix of existing functionality that will not change current behavior)
  • Documentation (added/updated documentation)

All tests should be run against the port production environment(using a testing org).

Core testing checklist

  • Integration able to create all default resources from scratch
  • Resync finishes successfully
  • Resync able to create entities
  • Resync able to update entities
  • Resync able to detect and delete entities
  • Scheduled resync able to abort existing resync and start a new one
  • Tested with at least 2 integrations from scratch
  • Tested with Kafka and Polling event listeners
  • Tested deletion of entities that don't pass the selector

Integration testing checklist

  • Integration able to create all default resources from scratch
  • Resync able to create entities
  • Resync able to update entities
  • Resync able to detect and delete entities
  • Resync finishes successfully
  • If new resource kind is added or updated in the integration, add example raw data, mapping and expected result to the examples folder in the integration directory.
  • If resource kind is updated, run the integration with the example data and check if the expected result is achieved
  • If new resource kind is added or updated, validate that live-events for that resource are working as expected
  • Docs PR link here

Preflight checklist

  • Handled rate limiting
  • Handled pagination
  • Implemented the code in async
  • Support Multi account

Screenshots

Include screenshots from your environment showing how the resources of the integration will look.

API Documentation

Provide links to the API documentation used for this integration.


PR Type

Enhancement, Tests, Documentation


Description

  • Introduced Bitbucket integration for Port's Ocean framework.

  • Implemented authentication using App Passwords and Base64 encoding.

  • Added data ingestion for projects, repositories, and pull requests.

  • Provided webhook handling for real-time event updates.

  • Included unit tests for authentication, integration, and webhook functionality.

  • Configured dependencies and testing setup for the integration.


Changes walkthrough 📝

Relevant files
Enhancement
4 files
auth.py
Added custom authentication client for Bitbucket API.       
+11/-0   
integration.py
Implemented core integration logic for Bitbucket.               
+72/-0   
main.py
Added entry point for running Bitbucket integration.         
+10/-0   
webhook.py
Added webhook handler for Bitbucket events.                           
+6/-0     
Configuration changes
2 files
config.py
Configured environment-based settings for Bitbucket integration.
+23/-0   
pytest.ini
Configured pytest settings for integration tests.               
+6/-0     
Tests
3 files
test_auth.py
Added unit test for authentication token generation.         
+5/-0     
test_integration.py
Added unit test for fetching Bitbucket projects.                 
+13/-0   
test_webhook.py
Added unit test for webhook event handling.                           
+16/-0   
Dependencies
2 files
pyproject.toml
Defined project metadata and dependencies.                             
+20/-0   
requirements.txt
Listed required dependencies for the integration.               
+5/-0     
Additional files
3 files
.DS_Store [link]   
.DS_Store [link]   
__init__.py [link]   

Need help?
  • Type /help how to ... in the comments thread for any questions about Qodo Merge usage.
  • Check out the documentation for more information.
  • Copy link
    Contributor

    PR Reviewer Guide 🔍

    Here are some key observations to aid the review process:

    ⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
    🧪 PR contains tests
    🔒 Security concerns

    Sensitive information exposure:
    The integration logs API responses at DEBUG level (line 48 in integration.py) which could potentially expose sensitive data in logs. Consider removing or sanitizing sensitive information before logging.

    ⚡ Recommended focus areas for review

    Error Handling

    The _fetch_paginated_data method silently retries on any exception without max retries limit. This could lead to infinite retries on permanent errors.

    try:
        response = await self.client.get(url)
        response.raise_for_status()
        data = response.json() if callable(getattr(response, "json", None)) else response
        logging.debug(f"API Response: {data}")
        results.extend(data.get("values", []))
        url = data.get("next", None)
    except Exception as e:
        logging.warning(f"Request error, retrying after 10 seconds... Error: {e}")
        await asyncio.sleep(10)
    Resource Cleanup

    The HTTP client is not properly closed/cleaned up. Should implement proper cleanup in a close() or exit method.

    self.client = OceanAsyncClient(base_url=BITBUCKET_API_BASE, headers={
        "Authorization": f"Basic {auth_token}",
        "Content-Type": "application/json"
    })
    Config Validation

    No validation of required environment variables. Missing environment variables could cause runtime errors.

    workspace=os.getenv("BITBUCKET_WORKSPACE"),
    username=os.getenv("BITBUCKET_USERNAME"),
    app_password=os.getenv("BITBUCKET_APP_PASSWORD")

    Copy link
    Contributor

    qodo-merge-pro bot commented Feb 20, 2025

    PR Code Suggestions ✨

    Explore these optional code suggestions:

    CategorySuggestion                                                                                                                                    Impact
    Possible issue
    Improve API error handling

    Add error handling for response.raise_for_status() with specific HTTP status
    code handling and proper backoff strategy instead of a fixed 10-second delay

    integrations/bitbucket-ocean/bitbucket_ocean/integration.py [44-53]

     try:
         response = await self.client.get(url)
         response.raise_for_status()
         data = response.json() if callable(getattr(response, "json", None)) else response
         logging.debug(f"API Response: {data}")
         results.extend(data.get("values", []))
         url = data.get("next", None)
    +except aiohttp.ClientResponseError as e:
    +    if e.status in (429, 503):
    +        retry_after = int(response.headers.get('Retry-After', 10))
    +        logging.warning(f"Rate limited, waiting {retry_after} seconds...")
    +        await asyncio.sleep(retry_after)
    +    else:
    +        logging.error(f"HTTP error {e.status}: {e.message}")
    +        break
     except Exception as e:
    -    logging.warning(f"Request error, retrying after 10 seconds... Error: {e}")
    -    await asyncio.sleep(10)
    +    logging.error(f"Unexpected error: {e}")
    +    break
    • Apply this suggestion
    Suggestion importance[1-10]: 9

    __

    Why: The suggestion significantly improves error handling by adding specific HTTP status code handling and proper retry logic for rate limiting, which is crucial for API reliability and robustness.

    High
    General
    Add request timeout protection

    Add request timeout to prevent infinite waiting on API calls that might hang

    integrations/bitbucket-ocean/bitbucket_ocean/integration.py [45]

    -response = await self.client.get(url)
    +response = await self.client.get(url, timeout=30)
    • Apply this suggestion
    Suggestion importance[1-10]: 8

    __

    Why: Adding a timeout is critical for preventing hung requests that could block the application indefinitely, significantly improving the application's reliability and resource management.

    Medium
    Security
    Validate authentication credentials

    Add input validation for username and password to prevent encoding empty or
    invalid credentials

    integrations/bitbucket-ocean/bitbucket_ocean/auth.py [10-11]

     def get_auth_token(username: str, password: str) -> str:
    +    if not username or not password:
    +        raise ValueError("Username and password cannot be empty")
         return base64.b64encode(f"{username}:{password}".encode()).decode()
    • Apply this suggestion
    Suggestion importance[1-10]: 7

    __

    Why: Input validation for authentication credentials is important for security and prevents potential issues with empty credentials, though the base64 encoding would work regardless.

    Medium
    • Update

    Copy link

    This pull request is automatically being deployed by Amplify Hosting (learn more).

    Access this pull request here: https://pr-1416.d1ftd8v2gowp8w.amplifyapp.com

    @github-actions github-actions bot added size/XXL and removed size/M labels Feb 23, 2025
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    1 participant