Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expand testing across all supported SDKs #468

Open
xmonader opened this issue Aug 15, 2024 · 5 comments
Open

Expand testing across all supported SDKs #468

xmonader opened this issue Aug 15, 2024 · 5 comments
Assignees
Labels
feature New feature or request
Milestone

Comments

@xmonader
Copy link
Contributor

We need to thoroughly test the Pulumi plugin to ensure it works correctly with all supported SDKs. This testing will help us validate the plugin's functionality, compatibility, and performance.

Tasks:

Test the plugin with each of the following SDKs:

  • Python
  • Go
  • YAML

For each SDK, test the following operations:

  • Resource creation (e.g. networks, vms)
  • Resource updates
  • Resource deletion
  • State import
  • Output values
  • Secret management

Verify ThreeFold Grid specific functionalities:

  • Node selection and filtering
  • Network creation and management
  • VM deployment and configuration (with different configs / networks)

Test error handling and edge cases:

  • Invalid configurations
  • Network interruptions
  • Resource conflicts

Performance testing:

  • Measure and compare execution times across SDKs.
  • Test with large-scale deployments (if applicable).

Documentation review:

  • Ensure examples and documentation are up-to-date for each SDK.

Acceptance Criteria:

  • The plugin functions correctly and consistently across all tested SDKs.
  • All ThreeFold Grid specific functionalities work as expected.
  • Error handling is robust and informative.
  • Performance is acceptable and consistent across SDKs.
  • Documentation is accurate and complete for each SDK.
  • No critical bugs or compatibility issues are found.

Please document any bugs, inconsistencies, or performance issues (in new tickets, mentioning this ticket).

@xmonader xmonader added the feature New feature or request label Aug 18, 2024
@xmonader xmonader added this to 3.15.x Aug 18, 2024
@xmonader xmonader moved this to In Progress in 3.15.x Aug 18, 2024
@xmonader xmonader added this to the v0.6.x milestone Aug 18, 2024
@khaledyoussef24 khaledyoussef24 mentioned this issue Aug 19, 2024
9 tasks
@rawdaGastan rawdaGastan modified the milestones: v0.6.x, v0.8.0 Aug 20, 2024
@rawdaGastan rawdaGastan modified the milestones: v0.8.0, v0.7.x Sep 25, 2024
@khaledyoussef24
Copy link

khaledyoussef24 commented Sep 30, 2024

estimated time for testing and documenting all of the requirements 3 days

@khaledyoussef24
Copy link

khaledyoussef24 commented Oct 7, 2024

  • Test the plugin with each of the following SDKs:
    • Python
      • VMs
      • k8s
      • gateways
      • network
Screenshot 2024-10-07 at 7 53 37 AM
  • Go

  • YAML

    • VMs
    • k8s
    • multiplevms
    • gateways
    • network
      (facing an issue in multiple vms will open a seperated one)
  • For each SDK, test the following operations:

    • Resource creation (e.g. networks, vms)
Screenshot 2024-10-07 at 7 27 42 AM
  • Resource updates
Screenshot 2024-10-07 at 7 38 07 AM
  • Resource deletion (facing an issue always the network contract does not get deleted)
Screenshot 2024-10-07 at 7 28 40 AM - [x] State import - [x] Output values Screenshot 2024-10-07 at 7 27 42 AM - [x] Secret management Screenshot 2024-10-07 at 7 35 10 AM
  • Verify ThreeFold Grid specific functionalities:

    • Node selection and filtering
    • Network creation and management
    • VM deployment and configuration
  • Test error handling and edge cases:

    • Invalid configurations
    • Network interruptions
Screenshot 2024-10-07 at 7 18 49 AM - [x] Resource conflicts
  • Performance testing:

    • Measure and compare execution times across SDKs.(they are all working fine it only varies couple of seconds and not consistent)
    • Test with large-scale deployments (if applicable).
  • Documentation review:

  • Ensure examples and documentation are up-to-date for each SDK.

Screenshot 2024-10-07 at 8 02 30 AM Screenshot 2024-10-07 at 8 02 37 AM

Acceptance tests :

  • the repo is reachable .
  • the docs are clear and match the criteria of most of the threefold deploying methodologies.
  • The plugin functions correctly and consistently across all tested SDKs
  • No critical bugs or compatibility issues are found.
  • Performance is acceptable and consistent across SDKs.

@khaledyoussef24
Copy link

nodejs examples were also added will test them.

@xmonader
Copy link
Contributor Author

xmonader commented Oct 8, 2024

nodejs examples were also added will test them.

Can create a separate ticket for node.js and we can track in 3.16

@xmonader
Copy link
Contributor Author

xmonader commented Oct 8, 2024

There you go #536

@ashraffouda ashraffouda removed this from 3.15.x Nov 5, 2024
@rawdaGastan rawdaGastan modified the milestones: v0.7.x, v0.8.x Nov 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants