feat(backend): move some static fields out of queue

[uael]

This commit move raw_* fields from the queue table to a new
job_definition table. For backward compatibility, they are left
unchanged in the queue table, hence all parameters fetch use coalesce
or a fallback to the queue table. As such, all previously pushed jobs
(before migration) will work as intended.

Important

This PR introduces a job_params table to store raw_code and raw_flow separately, optimizing database operations and updating related code to use this new structure.

• Database Changes:
  • Introduce job_params table in 20241004112544_create_params_args_result_tables.up.sql to store raw_code and raw_flow.
  • Update 20241004112544_create_params_args_result_tables.down.sql to drop job_params table.
• Code Changes:
  • Add QueuedJobEx struct in jobs.rs to extend QueuedJob with raw_code and raw_flow.
  • Replace get_queued_job with get_queued_job_ex in jobs.rs and other files to fetch extended job data.
  • Modify handle_dependency_job and handle_flow_dependency_job in worker_lockfiles.rs to use job_params for raw_code and raw_flow.
• Query Updates:
  • Update SQL queries in jobs.rs and worker_flow.rs to join job_params with queue for fetching raw_code and raw_flow.
  • Adjust generate_get_job_query in jobs.rs to include job_params fields.
• Miscellaneous:
  • Remove raw_code and raw_flow from QueuedJob struct in jobs.rs.
  • Update handle_code_execution_job in worker.rs to use job_params for raw_code.

^{This description was created by for 12f7d6a. It will automatically update as commits are pushed.}

[github-actions]

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

[uael]

I have read the CLA Document and I hereby sign the CLA

[ellipsis-dev]

👍 Looks good to me! Reviewed everything up to 12f7d6a in 2 minutes and 15 seconds

More details

• Looked at 901 lines of code in 10 files
• Skipped 0 files when reviewing.
• Skipped posting 6 drafted comments based on config settings.

1. backend/windmill-worker/src/worker_lockfiles.rs:204

• Draft comment:
  The is_some_and method is not available in stable Rust. Consider using map and unwrap_or for better compatibility and readability.
• Reason this comment was not posted:
  Comment was on unchanged code.

2. backend/windmill-worker/src/worker_flow.rs:308

• Draft comment:
  The is_some_and method is not available in stable Rust. Consider using map and unwrap_or for better compatibility and readability.
• Reason this comment was not posted:
  Marked as duplicate.

3. backend/windmill-worker/src/worker_flow.rs:937

• Draft comment:
  The is_some_and method is not available in stable Rust. Consider using map and unwrap_or for better compatibility and readability.
• Reason this comment was not posted:
  Marked as duplicate.

4. backend/windmill-worker/src/worker_flow.rs:981

• Draft comment:
  The is_some_and method is not available in stable Rust. Consider using map and unwrap_or for better compatibility and readability.
• Reason this comment was not posted:
  Marked as duplicate.

5. backend/windmill-worker/src/worker_flow.rs:1049

• Draft comment:
  The is_some_and method is not available in stable Rust. Consider using map and unwrap_or for better compatibility and readability.
• Reason this comment was not posted:
  Marked as duplicate.

6. backend/windmill-worker/src/worker_flow.rs:1114

• Draft comment:
  The is_some_and method is not available in stable Rust. Consider using map and unwrap_or for better compatibility and readability.
• Reason this comment was not posted:
  Marked as duplicate.

Workflow ID: wflow_9L6CEztEYwwCZe5A

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.