Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for Dynamic Workers Stored in Deno KV (deno compile) #27640

Open
chabanov opened this issue Jan 12, 2025 · 0 comments
Open

Support for Dynamic Workers Stored in Deno KV (deno compile) #27640

chabanov opened this issue Jan 12, 2025 · 0 comments

Comments

@chabanov
Copy link

Summary

We would like the ability to store worker code in a database so that users can dynamically create and manage their own scripts. This feature would allow launching/updating workers without requiring physical files on disk.

Details & Requirements

  1. Motivation

Many use cases require users to independently add or edit small modules/scripts for specialized tasks.
Storing scripts on disk complicates deployment and administration.
Storing code as text in a database or KV makes it easy to update or replace worker logic without restarting the entire application.

  1. Current Behavior & Limitations

Currently, workers are typically kept as files in a designated folder, requiring manual file management to add or edit them.
When trying to store worker code in KV/a database and load it via new Worker with Blob or data: URLs, errors like Module not found can occur (especially if the code has relative imports or if deno compile is used).
For “dynamic” workers under deno compile, any code not explicitly listed in static import statements does not get included in the resulting binary.

  1. Desired Behavior

The ability to retrieve worker code from a database (as a string) and run it dynamically.
Ideally, relative or absolute imports would work (at least optionally), so that the logic inside a worker is not disrupted.
If deno compile is used, there should be a way to either include these workers in the binary or allow dynamic import from KV/database without errors.

  1. Possible Approaches

External KV Support at Runtime: Let deno compile have a “mode” or configuration in which the compiled binary can fetch worker code from an external source (database/KV) without triggering Module not found or similar errors.

  1. Benefits

Users can add or modify worker processes without recompiling or restarting the entire application.
Simplified management and versioning of worker code within a single database or KV store.
Flexible permission settings for workers, since the code source is centralized.

  1. Additional Context / Workarounds

Running via deno run (instead of deno compile) can work with dynamic or external code, but that loses the convenience of having a self-contained executable.
One can bundle workers into a single file, but that prevents truly “dynamic” additions of new workers.
Storing workers as files alongside the binary with --allow-read=./workers/ is another workaround, but it’s less convenient than database-based management.

Conclusion

We respectfully ask the development team to consider official support (or a simpler workaround) for dynamically storing and launching worker scripts directly from a database or KV store, enhancing developer experience and offering more flexible management of worker code.

Sincerely,
Stels Laboratory Team

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant