Skip to content

Conversation

pleoxconfusa
Copy link

Introduce pkgs processing logic for model component installation


Description

This pull request introduces the pkgs integration into Firewheel, enabling model components to define external packages in a structured way using metadata files within the INSTALL/pkgs directory. These packages are processed during installation using Ansible tasks, replacing redundant logic in tasks.yml.

Key Changes

  • Added pkgs_tasks.yml to handle package processing, including downloading, verifying, compressing, and cleaning up package directories.
  • Updated main.yml to include logic for finding and processing package metadata files in INSTALL/pkgs.
  • Introduced the optional INSTALL/pkgs directory for defining package metadata files.
  • Updated documentation (mc_install.rst) to describe the pkgs integration, including examples for package metadata files, pkgs_tasks.yml, and simplified tasks.yml.

Benefits

  • Modularity: Simplifies package handling by centralizing logic in pkgs_tasks.yml.
  • Idempotence: Ensures installation tasks can be run multiple times without changing the final state.
  • Reproducibility: Packages are defined with exact versions and checksums, ensuring consistent experimental outcomes.
  • Offline Accessibility: Supports caching and retrieval of packages for offline installations.
  • Cleanup: Automatically removes intermediate files and directories created during package processing.

Related Issue

This pull request addresses the need for modular and idempotent handling of external packages in Firewheel. It is part of the broader effort to improve reproducibility and offline accessibility for model components.


Type of Change

Please select the type of change your pull request introduces:

  • Bugfix
  • New feature
  • Documentation update
  • Other (please describe):

Checklist

  • This PR conforms to the process detailed in the Contributing Guide.
  • I have included no proprietary/sensitive information in my code.
  • I have performed a self-review of my code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have made corresponding changes to the documentation.
  • My changes generate no new warnings.
  • I have tested my code.

Additional Notes

This update introduces a significant improvement to the installation process for model components. By transitioning to the pkgs integration, Firewheel now supports more modular and reproducible handling of external packages. This change has been tested with multiple model components, including dns.dns_objects, to ensure compatibility with both traditional tasks.yml logic and the new pkgs integration.

pleoxconfusa and others added 2 commits September 25, 2025 06:04
- Added pkgs_tasks.yml to handle package processing, including downloading, verifying, compressing, and cleaning up package directories.
- Updated main.yml to include logic for finding and processing package metadata files in INSTALL/pkgs.
- Introduced the optional INSTALL/pkgs directory for defining package metadata files.
- Updated documentation (mc_install.rst) to describe the pkgs integration, including examples for package metadata files, pkgs_tasks.yml, and simplified tasks.yml.
- This feature enables more modular and idempotent handling of external packages, improving reproducibility and offline accessibility for model components.
@github-actions github-actions bot added the documentation Improvements or additions to documentation label Sep 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant