Dependency management implementation design

[yassinebenaid]

As stated in the readme (as well as many other places). We want to add a dependency management system. And by dependency management, I don't mean modules only. But anything else in this regard including builtin commands, standard library and so on.

What do we mean by builtin commands?

Most of builtin commands in bash should be builtin in bunster, such as echo, true, false, cd, pwd and so on. we might also make some popular commands builtin, such as ls, ps etc.

What do we mean by standard library?

In addition to popular commands that we can make builtin. We want to offer a wide collection of utilities builtin. Especially that we use the Go programming language as target. It's so cool that we can expose the Go's standard library to the shell environment.

For example, string manipulation, numbers conversion, network management, various encoding and decoding formats json, html, base64 etc, file management, encryption and anything else offered by the Go's standard library .

What do we mean by modules?

We want to allow users to publish and consume scripts as libraries. Just like npm, composer, go get etc. This topic is not a concern now and will be discussed separately later. I just wanted to mention it to make my point below.

My Vision:

I imaging that bash builtin commands as well as popular commands (those in GNU utils) and others are good to stay as they are. Users won't notice that they're using a builtin version of them. We will try to implement the exact same spec.

The standard library will look like this

/std/proc/cpunum                           #  prints number of CPUs

/std/encoding/htmlescape                   # escapes special characters like "<" to become "&lt;" in html text.
/std/encoding/htmlunescape                 # the opposite of above


/std/time/now                              # prints current datetime
/std/time/cmp time1 time2                  # compares two datetimes

/std/cypt/rand                             # print random unique string

...

This is just an example of how the API looks like. the actual commands will be discussed separately later.

Why do we have this /std/xxx/xxx format?

Because I want to introduce namespaces. Especially that we want to support modules later. Two scripts may define functions with same name, which will cause conflicts. Thats why namespaces are important.

This is not a final decision, but modules are expected to look like /ext/<LIBRARY>/<FUNCTION>. So that we have a consistent design.

Concerns !!!

I can imagine that the standard library is going to be very big. That is why I'm very concerned that the binary size will be a bit high. the binary size will affect the ram usage because we might need to load references to all functions so that we can look them up.

The solution would be to add directives at the op of the script:

use ( "proc", "time", "crypt" )

...

This will also serve us in modules:

use ( "github.com/yassinebenaid/docker-utils@f8b815857" )

...

Again, this is not a discussion about modules, we can discuss them later. But most importantly is the use of the use keyword. very consistent. and allow as to only compile needed/required modules.

This was my vision of the design. I'm a very open for suggestions.

[yassinebenaid]

any thoughts @dscho @vanodevium

[vanodevium]

@yassinebenaid

I think it looks very nice and convenient.
The only thing I would probably call it not an ext, but a module or plugin.
The keyword use is also convenient (although it seems to me that it will be possible to assemble it yourself at the stage of code interpretation), but it can also be shifted to the responsibility of the script author. Let him think what he is doing!
You can even do this: if the script author used at least some std, then load the entire library. Otherwise, let him use use himself.

Anyway, I like it! Really awesome.

[yassinebenaid]

Glad that you like it. few notes :

The only thing I would probably call it not an ext, but a module or plugin.

We can think of this later when we work on modules.

The keyword use is also convenient (although it seems to me that it will be possible to assemble it yourself at the stage of code interpretation), but it can also be shifted to the responsibility of the script author. Let him think what he is doing!
You can even do this: if the script author used at least some std, then load the entire library. Otherwise, let him use use himself.

I have to correct you !!
First, we cannot know weather the user is using the standard library or not. because the use can type this:

var=/std/foo/bar


$var argument

The command name is the result of the substitution of the variable $var. We don't know the result of the substitution at compile time. This was an example with variables, but it can be command substitution, expansion or whatever else. all of which are only known at runtime. Thats why the user needs to tell us what he needs exactly.

You can even do this: if the script author used at least some std, then load the entire library. Otherwise, let him use use himself.

The goal behind all of this is to avoid loading the entire stdlib. The user will choose which parts are required: use ( "crypt" ). The compiler will only link the /std/crypt library. At compile time.

[vanodevium]

@yassinebenaid Oh, I see. Sounds very good! Lets do it :)