Foreign Macro Interface

[kirsle]

Something that could benefit all versions of RiveScript is to design a "Foreign Macro Interface"

This would allow bot authors to write object macros written in literally any programming language, as long as a "host script" is written for the language. The host script's responsibility would be to read JSON input over STDIN, and write the result as JSON over STDOUT. This is very similar to how the Perl support for RiveScript-Java already works (com.rivescript.lang.Perl and its Perl host script). Similarly is the perl-objects example for RiveScript-Python.

Each implementation of RiveScript would have a generic "Foreign Macro Handler" that can work with any programming language. It might be defined like this (Python example):

from rivescript import RiveScript
from rivescript.lang.foreign import ForeignMacroHandler

bot = RiveScript()
bot.set_handler("ruby", ForeignMacroHandler(bot,
    host="/path/to/rubyhost",
))

# and proceed as normal...

The things that would be needed for this to work:

• 5x Foreign Macro Handlers, one for each implementation of RiveScript. These would be general purpose handlers that can work with all programming languages. They could support both interpreted languages like Ruby as well as compiled languages (using options to control the compile pipeline, such as a gcc command to build C code).
• 1x Host Script per programming language. This would be a program that speaks the Foreign Macro API (reading and writing JSON over standard I/O), and would only need to be written once for each language and would be equally usable by all versions of RiveScript.

Full Example: Ruby

For example, you could have RiveScript source that defines a Ruby object macro (at the time of writing, there is no native RiveScript implementation available for Ruby):

> object reverse ruby
    message = @args.join(" ")
    return a.reverse!
< object

+ reverse *
- <call>reverse <star></call>

When the RiveScript interpreter (say, the Python one) reads the source file, it would know that Ruby code has a handler (the ForeignMacroHandler), which would simply store the Ruby source code as a string until it's actually <call>ed on in RiveScript code.

When the <call> tag is processed, the Python RiveScript bot would shell out to the Ruby Host Script and send it a JSON blob along the lines of:

// The STDIN sent to the Ruby Host Script
{
  "username": "localuser",
  "message": "reverse hello world",
  "vars": {  // user variables
    "topic": "random",
    "name": "Noah"
  },
  // the Ruby source of the macro
  "source": "message = @args.join(\" \")\nreturn a.reverse!"
}

The Ruby Host Script would read and parse the JSON, evaluate and run the Ruby source, and return a similar JSON blob over STDOUT:

{
  "status": "ok",
  "vars": { "topic": "random", "name": "Noah" },
  "reply": "dlrow olleh"
}

RiveScript API Inside Foreign Macros

The big obvious drawback is that object macros typically receive (rs, args) parameters, where the rs is the same instance of the parent bot, and the code can call functions like current_user() and set_uservar().

To support this for foreign macros, each Host Script can define a "shim" class for the RiveScript API. All they would need to implement are the user variable functions, like setUservars() and currentUser(). They could also implement the bot and global functions if those are useful.

What the Host Script could do is just keep a dictionary of user vars, initially populated using the Input JSON, and provide shim user var functions that update its dictionaries. And when writing the Output JSON it can just serialize those user variable dicts.

For the case when the Foreign Macro already exists as a Native Macro in one implementation or another (i.e. Python), the RiveScript shim API should match the conventions of the native version, i.e. using current_user() rather than currentUser() as the naming convention for functions. In the case that one programming language has many Native Macros (e.g. JavaScript being usable in Go, JS and Java), the "most pure" version's API should be used (e.g. the JavaScript Foreign Macro should resemble the API of rivescript-js, not the Go-style naming convention from Go, or anything the Java port does).

So for example, for Python object macros, if you're programming your bot in Python, you would just use the default built-in Python support because this would be the most efficient: the code can be evaluated and cached by the program rather than on demand. But if you're programming your bot in Go, and you want to use Python objects, you could use the Foreign Macro Interface and have the exact same RiveScript API available to use.

Code Layout

Each implementation of RiveScript would keep its ForeignMacroHandler in its own git tree, probably under the lang/ namespace.

Host Scripts would be best bundled together as one large package, all in a common git repo. Possibly named something like rivescript-host-scripts or an acronym like rsfmh (RiveScript Foreign Macro Host).

The Host Scripts repo would include all possible available host scripts (Ruby, Bash, Go, C++, whatever the community is up to the task to write...) and would be easy to install somehow, so that as a mere mortal chatbot developer, your setup steps might be like:

$ pip install rivescript
$ git clone https://github.com/aichaos/rsfmh

bot = RiveScript()
bot.set_handler("ruby", ForeignMacroHandler(bot,
    host="./rsfmh/ruby.rb",
))

[dcsan]

When the RiveScript interpreter (say, the Python one) reads the source file, it would know that Ruby code has a handler

so this is for someone to run macros in a language separate from their current server language?
it seems that would only be useful if theres a big library of these macros.
at least for us, we do want to be able to call out to code written in a native language, but it's always code that we write. so if we're running a nodeJS server, we'd write JS code. calling ruby from python from rivescript seems a bit esoteric.
currently we're having problems just getting a result from a call in the same language.
aichaos/rivescript-js#234