Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve agent code transport #8729

Open
wouterdb opened this issue Feb 11, 2025 · 1 comment
Open

Improve agent code transport #8729

wouterdb opened this issue Feb 11, 2025 · 1 comment
Assignees

Comments

@wouterdb
Copy link
Contributor

wouterdb commented Feb 11, 2025

Goal

The way code is transported between compile and agent is in need of a rework.

Problems

  • Code is now registered per handler type, not per model
    • all handler in the same module get the same code
    • executors now have weird names, as we don't know which module are on there
  • Files with the same content but different path go missing on upload

Proposed solution

Build a new code transport mechanism from scratch

In DB structure

  1. table for modules
    • a module corresponds to a specific version of an inmanta module (e.g. std)
    • a module has a name, version and a list of requirements
    • we will have two ways of versioning: source based or packaged
      • packaged get a version like 1.3.6 and gets installed per pip (don't have that yet)
      • source based gets a hash (hash of all files)
  2. a table of files, joining all files (in the existing file store) to a module (from above)
  • file has a path, a hash (link to content) and module/version it is contained in
  1. join table joining agent, model version to module, module version

API

  • api to request all module/versions for a agent on a specific version
  • api to download all files for a module/version
  • update codemanager of the scheduler to use them
  • api to upload code/version (like current code upload)
  • add module/version argument to put_version to populate the join table and to put_partial to check source version remain the same

TODO

@wouterdb
Copy link
Contributor Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants