allow users to write obscure shell operators with function call syntax

[andychu]

For users who care about readability and not compatibility. Exact syntax TBD.

• ${x#/} => stripPrefix(x, '/')
• ${x^^} => uppercase(x)
• ${x@P} => promptFormat(x)
• ${x//pat/replace} => replace(x, 'pat', 'replace', 'all') (although I wanted this to use the left-to-right "pass" syntax, which I'm not sure if we'll get to)

Could also add an option to not strip trailing newline, e.g. commandSub(cmd, newline=true)

https://oilshell.zulipchat.com/#narrow/stream/121540-oil-discuss/topic/Trailing.20Newline.20in.20Command.20Sub

Also:

• glob() for dynamic globbing
• split() for IFS splitting and other kinds of splitting
• maybe() for empty elision

[shamrin]

What is the advantage of new syntax:

replace(x, 'pat', 'replace', 'all')

When this exits (it's even shorter!):

$(echo $x | sed 's/pat/replace/g')

Similarly, why support uppercase(x) when there's $(echo $x | tr '[a-z]' '[A-Z]')?

Contrast with fish shell approach. Fish does not have function calls with round brackets, only commands.

In other words, what's Oil philosophy regarding function calls versus commands, substitutions and pipes?

P.S. Having said all that, I could never remember all those ${x...} forms. And I do use some of them in my scripts! E.g. ,${x-} and ${x:-}. In this example I cannot even tell what's the difference without consulting the docs.

[shamrin]

Brainstorm mode on.

I wonder if this would be better:

${x | sed s/pat/replace/g}

And this:

${x | tr [a-z] [A-Z]}

In other words, ${x | ...} could be a syntax sugar over $(echo $x | ...) pattern.

[andychu]

Yes good question.. this is open to debate, but the rationale is:

• $(basename $x) implies that you're forking a process, and so does $(echo x), where echo is a builtin. That is, a command substitution is a process invocation.
  • On the other hand, something like $basename(x) or $uppercase(x) is a function call.
  • It might be possible to magically optimize things, but in general I would like to avoid magic. One of the principles of Oil is that syntax and semantics should correspond. Similar semantics should use similar syntax; different semantics should use different syntax. So I don't want to use the same syntax for two different things.
• I want to have integer, bool, array, and associative array arguments and return values, not just strings.

A canonical example is:

declare -A assoc=([k]=v [k2]=v2)

# bash
for value in "${assoc[@]}"; do echo $value; done
for key in "${!assoc[@]}"; do echo $key; done  # yes this a weird syntax!


# Oil:
for value in @values(assoc) { echo $value }
for key in @keys(assoc) { echo $key }

So basically keys and values are functions that take an assoc array and return an array. That doesn't work with command subs.

Well I guess it could if we come up with some canonical serialization... but I'm not sure that is a good idea? Hm.

Note the oil literal syntax is {k: v, k2: v2} like JS/Python.

[andychu]

There are also a bunch of string functions in Python that take integers that I think would be useful in shell:

s.split(n)  # maximum number of times to split

s.find(needle, start_pos)

s.replace(s, r, n)  # maximum number of replacements

Although I'm pretty sure Oil won't have method call syntax, so it will be more like split(s, n).

I also want the keyword argument syntax, like:

find(s, needle, start_pos=3)

[andychu]

Related thread about left-to-right syntax: https://oilshell.zulipchat.com/#narrow/stream/121540-oil-discuss/topic/left-to-right.20syntax.20ideas

(Sorry I am switching back and forth between Zulip and github. Github is for stuff I don't want to get "lost" while Zulip is sort of free-form)

[andychu]

This depends on #636 to break the Python dependency (and maybe the left-to-right syntax evaluation)

Won't get to this in 2020, can be done on demand once the infrastructure is there. We are parsing everything we need now, but the evaluator has to be done before doing any of this

[andychu]

#865 is related

[andychu]

This came up again in a conversation with Alex from Shell++

We should nail this down; it's one of his key complaints

[bar-g]

Nice. Maybe the simplifications in ruby and crystal can even provide good precedence for keeping the noise level nicely down to whats bearable for typical shell usage measures.

double 3 #=> 6
sum 3, 4, 5 #=> 12
sum sum(3, 4), 5 #=> 12

(Compact overview at: https://learnxinyminutes.com/docs/crystal (scroll to "# Functions"))

Or, might oil even be able to do without the , in the word/command language? Because there every new function would have to started with . (appended to vars or literals), $ or @ (short form) or $[ (expression ), anyway?

[andychu]

We almost certainly want/need parens, because otherwise functions aren't distinct from procs

procs return an integer status value, while functions can return structured data. It's something of an open design question

The parens / no parens distinction means "string arguments" vs. "typed arguments" right now. I think functions have only typed arguments, while procs can have both.

• Design for Python-like functions in Oil #1112

[bar-g]

(Now, I'm not sure to which issue the function-call syntax belongs.)

In ruby/crystal the parens are also only optional where the call is unambiguous.

But I get that you're reasoning to better prefer obvious (explicit) and always consistent syntax. Also, to require parens for all function calls seems consistent to "parens indicate containing expression language". So yeah, not so sure how the inconsistency vs. simplicity would work out if there were more inference done.

Maybe a better alternative to keep syntax noise down in oil could be to have idomatic oil only use short, single argument function calls, to avoid long comma and quoting chains. And always use argument vectors (splice array/dict) to pass more values.

Not sure what you mean by left-to-right syntax above, but maybe it's related to "piping" vectors?:
@args | func1() | func2() ?

[andychu]

We have a design and implementation for methods and function calls, so closing this old issue

Help wanted here -

https://www.oilshell.org/blog/2024/01/release-0.19.0.html#ideas-for-contributions-and-feedback