Backcall operator / Do notation

[nythrox]

Can we get a backcall operator like in Livescript and Glean?

do
  data <- $.get 'ajaxtest'
  $ '.result' .html data
  processed <- $.get 'ajaxprocess', data
  $ '.result' .append processed
  alert 'hi'

Gets converted into

$.get('ajaxtest', function(data){
  $('.result').html(data);
  $.get('ajaxprocess', data, function(processed){
    $('.result').append(processed);
  });
});
alert('hi');

This frees us from needing to ask the language developer to add features like await/async, try/catch, generators, etc...
And gives functional programmers and framework developers a whole new level of power without ruining the ergonomy of the framework (see: effect-ts).

I'd hope for a syntax which is not as unnatural as gleam or scala, and more to Roc and Idris which use the ! suffix (allowing an easy suffix like ! also gives us ?. and any other type of chaining without devs having to implement it)

[bbrk24]

Isn't this just util.promisify? Even if you aren't in Node, this isn't hard to implement yourself:

const promisify = fn => (...args) =>
  new Promise((resolve, reject) =>
    fn(...args, (result, error) => error == null ? resolve(result) : reject(error))
  );

[nythrox]

@bbrk24
No, backcall operator isn't a higher order function. Its a syntactic feature which moves everything after it into a callback.
For example:

const msg <- Promise.resolve(0).then;
console.log(msg);

gets turned into

Promise.resolve(0).then((msg ) -> console.log(msg))

It can't be done in userland since it rearranges the order of statements in an expression, while maintaining the appearance of a normal control flow (direct-style).

Await/async does this too, but backcalls are more general, since they move everything after the backcall into a callback, being passed into the function on the right side of the backcall operator.

Here is another example

test <- (fn -> fn("hello world"))
console.log(test)
// prints "hello world"

It turns the rest of the scope into a callback, which get passed into the right side of the backcall operator

[bbrk24]

I'm still not sure I understand the point. Your initial motivating example still feels like it could just use promisify:

get := util.promisify $@get
async do
  data := await get 'ajaxtest'
  $('.result').html data
  processed := await get 'ajaxprocess', data
  $('.result').append processed
alert 'hi'

async do isolates the asynchronicity, meaning the outside scope doesn't need to wait for it to complete. The alert 'hi' at the bottom will be hit while the await get 'ajaxtest' is waiting.

Your second example,

const msg <- Promise.resolve(0).then;
console.log(msg);

feels to me like a convoluted way to avoid saying await.

As for the third example, I really don't see the point in saying x <- (fn) -> fn(...). That seems like a really convoluted way of just assigning directly to x.

[edemaine]

I guess one interesting thing about backcalls, compared to promisify, is that in principle everything could remain synchronous in the JavaScript sense. For example, if $.get was synchronous and then called the callback, then everything would resolve synchronously. It's kind of like continuations...

That said, I'm not aware of many scenarios where callbacks are used in a synchronous fashion; they're mostly for asynchronous behavior. In that case, promisify + await seems simpler for anyone familiar with promises.

By the way, have you seen IcedCoffeeScript? I used to use it, back before ES gained promises and its own notion of await, but meanwhile it offered a pretty nice await/defer syntax for what I think is backcalls:

await $.getJSON url, defer json
console.log json
↓↓↓
$.getJSON url, (json) ->
  console.log json

One nice thing here is it doesn't assume that the callback is an appended last argument: you can put it anywhere. It also supported loops and such: [playground]

results = new Array(list.length)
await
  for item, i in list
    fetch item, defer results[i]
console.log results

I liked it at the time, and it took me a while to understand and transition to promises and await. But nowadays I'd prefer Civet's console.log await.all fetch item, which is equivalent to the above IcedCoffeeScript (in the async world). As bbrk24 points out, async do console.log await.all fetch item hides the async aspect; the only difference is if fetch was actually synchronous.

Another question: what APIs are you using that still use callbacks? I feel like most of them have transitioned to promises. One exception is node:fs, but node:fs/promises is just a few more keystrokes away.

[bbrk24]

console.log await.all fetch item

console.log await.all list.map fetch .?

[nythrox]

I'm still not sure I understand the point. Your initial motivating example still feels like it could just use promisify:

This is a contrived example to show how to do callback-oriented code without await/async

feels to me like a convoluted way to avoid saying await.

You are absolutely correct. This is exactly await/async, except that it doesn't only work for Promises, but any callback that has the structure of a promise (monadic structures). Which means it can work for exceptions, promises, nullables, iterators, generators, coroutines, promises, and many other control-flow structures that wouldn't need to be added to the language itself. This means we could add powerful features in user-land and make them usable with this simple sugar syntax, without having to ask language developers to implement custom syntax for every feature.

[nythrox]

@edemaine

It's kind of like continuations...

Yes, this lets you write code that uses continuations (callbacks) in direct-style, ie, in the sequential fashion that async/await allows.
Continuations are very powerful and a -lot- can be done on top of them, but the reason we don't see much usage now a days is due to how ugly it is to use them (callback hell).

Backcall operator solves this for all constructs that can be built with continuations (iterators, async/await, streams, even request handlers) and would be a game changer for people who are actively building these constructs from scratch, like the Effect-TS guys, reactivex, old promise libraries, coroutine libraries, http libraries and etc. to have much more control and power over their code.

[nythrox]

the do notation is also universal in functional programming languages, which don't ever need to introduce custom syntax for things like await/async, exceptions, iterators, nullables, and more, since the do notation is sufficient for solving the legibility problem of monads (which encapsules almost all control flow constructs and much more)

[nythrox]

I feel like I am repeating myself here and I want to correctly transmit the WHY of do notation. Please help me out here if my explanations aren't what you guys are looking for.

To summarize:

• Do notation is await/async, but generalized for any construct (like promises) that can be built with callbacks
• Callbacks are very powerful, can and are used to build things like promises, cancellable promises, coroutines, generators, iterators, exceptions, backtracing, deferring, nullables, streams and more.
• Working with callbacks sucks (because of callback hell), this sugar syntax fixes it for all cases above. Currently it is uncommon to work with anything but promises, but that is partially because of how hard it is to work with callbacks. Libraries like RxJs and Effect-TS are very powerful but ugly to use and read, and this would in large part be solved by allowing us to write code in direct style (like await async)
• Do notation is ubiquitous in functional languages, a very mature construct, and all the above has been a solved problem for ages. Functional languages keep their core much smaller while being more powerful, since most constructs are built in userland.