-
Notifications
You must be signed in to change notification settings - Fork 68
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Uncouple bundle information parsing from initialization #526
Comments
Tbh I think the main issue is just that you need to actually evaluate
something to get the config, which makes it a bit harder to e.g. query for
info or a search etc safely.
In Aisu, I literally just parsed the MoonScript and extracted the data from
there.
It'd be kinda nice if we had something like jsonnet, just...some way to
have a functional-ish subset of MoonScript syntax that can easily be
evaluated.
…On Mon, Aug 26, 2019, 5:34 PM Janfel ***@***.***> wrote:
The Problem
Currently, a bundles metadata (author, license, etc) can only be parsed by
loading the bundle and therefore executing it. If a bundle requires a
certain dependency, such as a library or another bundle, then even just
parsing it's metadata might fail because of an unmet dependency. This makes
it impossible for an external tool such as a package manager to parse and
meet these dependencies before loading the bundle, without a separate
package manager specific metadata file (e.g. aisu.moon).
The Proposal
To circumvent this issue, I propose a separate lua/moonscript file, which,
if it exists, will be parsed before loading the bundle. The metadata will
then be expected to be in this file.
When loading the bundle, the metadata table will be inserted into the
exports table under the info key, thus keeping backwards compatibility
and preventing naming conflicts between the tables. The resulting table
will then be made accessible under _G.bundles as usual.
The Prospect
This would allow tool developers to specify tool-specific metadata to be
placed in this file, maybe under a standardized location such as
info.tools.[toolname], which can then be parsed without executing the
bundle itself, making dependency resolution possible. Continuing with the
aisu example, the build function would be placed under info.aisu.build or
info.tools.aisu.build, making aisu.moon unnecessary.
Proposed names for the file:
- bundle.lua
- meta.lua
- info.lua
- project.lua (after pyproject.toml)
- package.lua (after package.json)
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#526?email_source=notifications&email_token=AAM4YSNEBVOSVRI3LDPISSTQGRK6VA5CNFSM4IPXKCN2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HHP53VA>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAM4YSOKQIKI35VWTDY4L53QGRK6VANCNFSM4IPXKCNQ>
.
|
You can probably use something like the Rockspec Format used by LuaRocks. They are loading the files like regular Lua modules, just with an empty environment. This, however, would be rather declarative than functional and would not work with Moonscript out of the box (An issue with moonscripts |
Hmm, I'm still not entirely sure about the sandboxing approach. Whether bundles are directly grabbed off GitHub like now or go on some online bundle site, someone's going to have to run the config script without installing it for stuff like listing and searches to work. Even with the example sandbox, a lot can go wrong: a malicious script could easily e.g. run forever or eat up a ton of RAM. A simple but more advanced sandbox from the Lua mailing list is a bit more thorough, but the memory part at minimum won't work on LuaJIT (which doesn't support __gc) and won't handle the case of high CPU usage. If this is running on a server (e.g. for the package repository idea), it would be possible to use an external, more hardened sandbox, such as nsjail. However, if it's the client systems that are of concern, then it's really hard to rely on a sandbox being present on all platforms. Python worked around this for a long time by auto generated EGG-INFO files as part of the package build process, but nowadays the preference is to define the package config in a basic config format via pyproject.toml / Pipfile.toml; any extra scripts go elsewhere. Nimble works around this by maintaining a package list in JSON with separate metadata. IMO it's an awkward solution. All this seems like a lot of effort to be able to use Lua for defining basic metadata... Btw for the part about adding extensions, I personally just add new models that derive from the other ones, e.g.: mode.register
name: 'bazel'
extensions: {'bazel', 'bzl', 'BUILD'}
patterns: { 'BUILD$', 'WORKSPACE$' }
parent: 'python'
create: -> {} |
One could always just use a weak sandbox on the client and expect the user to know what they are downloading (like Vim) or provide the majority of bundles in trustworthy repositories (like Emacs) that validate the scripts with a strong sandbox. |
Oh yeah it's not perfect, I'm just saying that's it's a tad more complicated that it can appear. I just don't want to see a big bundle API change that we have to later redo for some other reason... |
I agree. Making up an API on the way is almost always a bad idea. Should this issue be closed then, or stay open as a reminder? |
What's the downside of using json for something like this? |
I mean json is kinda painful to write by hand.
That being said...I just realized, can't we use GKeyFile for this? GLib is
already a dependency so this requires nothing extra on our part, and it's
imo flexible enough for this basic use case.
…On Sun, Oct 20, 2019, 10:54 PM Shalabh Chaturvedi ***@***.***> wrote:
What's the downside of using json for something like this?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#526?email_source=notifications&email_token=AAM4YSPDHVZOK5T2MDTGPGTQPURXVA5CNFSM4IPXKCN2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBY7HJI#issuecomment-544338853>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAM4YSLO3ZR32VAEDULHVGDQPURXVANCNFSM4IPXKCNQ>
.
|
I hadn't heard of GKeyFile before - looks like some kind of ini file format? I think in this case the ubiquity of json trumps other factors. I hope we're not writing gobs and gobs of this stuff - we write one small file for every package - so being a bit clunky isn't a big deal. Maybe a good way forward to first nail down the exact metadata that will be stored in this file? |
I was thinking another way to manage this is to have the bundle writer run a 'howl bundle' command that imports the bundle locally and introspects it to write out the metadata (then the format doesn't matter too much). The tool could even export things like the registered commands and config values, without having the bundle writer have to repeat those in the metadata config. WDYT? |
I believe Arch does this with the AUR: you need to write a .SRCINFO file
which contains metadata from the PKGBUILD script. The one con is that it
can be *really* easy to accidentally push updates without updating .SRCINFO
unless you set up some local Git hooks.
…On Mon, Nov 4, 2019, 2:58 PM Shalabh Chaturvedi ***@***.***> wrote:
I was thinking another way to manage this is to have the bundle writer run
a 'howl bundle' command that imports the bundle locally and introspects it
to write out the metadata (then the format doesn't matter too much). The
tool could even export things like the registered commands and config
values, without having the bundle writer have to repeat those in the
metadata config. WDYT?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#526?email_source=notifications&email_token=AAM4YSKMPLLD7P4UW4JM33TQSCEJDA5CNFSM4IPXKCN2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDAWA7Q#issuecomment-549544062>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAM4YSJKQH7V33I5DLXRUILQSCEJDANCNFSM4IPXKCNQ>
.
|
Yeah that's a good point. What if the build command produced a zip file that you'd have to upload to a repo somewhere to make it available? Versioning of bundles is another issue we don't currently have a solution for. I'm imagining in the future we'd want some tools to build/upload/register the bundle anyway so might as well tie in the metadata generation to that. Also for development on trunk, you could continue to git push the changes and even if the metadata wasn't regenerated I'd imagine it would still work in most cases. |
The Problem
Currently, a bundles metadata (author, license, etc) can only be parsed by loading the bundle and therefore executing it. If a bundle requires a certain dependency, such as a library or another bundle, then even just parsing it's metadata might fail because of an unmet dependency. This makes it impossible for an external tool such as a package manager to parse and meet these dependencies before loading the bundle, without a separate package manager specific metadata file (e.g.
aisu.moon
).The Proposal
To circumvent this issue, I propose a separate lua/moonscript file, which, if it exists, will be parsed before loading the bundle. The metadata will then be expected to be in this file.
When loading the bundle, the metadata table will be inserted into the exports table under the
info
key, thus keeping backwards compatibility and preventing naming conflicts between the tables. The resulting table will then be made accessible under_G.bundles
as usual.The Prospect
This would allow tool developers to specify tool-specific metadata to be placed in this file, maybe under a standardized location such as
info.tools.[toolname]
, which can then be parsed without executing the bundle itself, making dependency resolution possible. Continuing with the aisu example, the build function would be placed underinfo.aisu.build
orinfo.tools.aisu.build
, makingaisu.moon
unnecessary.Proposed names for the file:
bundle.lua
meta.lua
info.lua
project.lua
(afterpyproject.toml
)package.lua
(afterpackage.json
)The text was updated successfully, but these errors were encountered: