Hello 
,
I got a bit of a road block with Roto. Initially I wanted to ask only about that, but then I remembered the XY problem so here is a bit longer question but with extra context of what I want to build. I hope itās worth it 
Setup:
Imagine an application that sits between the user and a server. The user uses the app to execute commands that operate on the server state or just query information etc. Most parts (e.g. UI, business logic, ..) of these commands are implemented locally in the app and use either generic or custom API endpoints on the server.
Why Roto is interesting for this application:
The server that the application talks to is not universal for all app users. Instead, users run their own server, which may or may not have the typical API endpoints. This means they will have to adjust the commands to work with their setup. This would be much easier if commands were written in Roto, loaded and compiled on the fly, instead of having to recompile the app. Another reason is collaboration (and generic software maintenanceā¦), it is trivial to configure your environment to work with app version > foo and only maintain your own Roto scripts. It is much easier to check-in Roto scripts in your repo than a patch to the app. You can also easily share your Roto scripts with others.
Issue 1: Interfaces?
The commands all share the same interface to the application. This could be for example the presence of two functions with a specific signature. It would be cool to be able to ask runtime.compile(foo).unwrap().implements(my_interface) -> bool or comparable. Similarly, it would be nice to import an interface from within a Roto script and stating that this script implements it. That way future LSP integration and the runtime on compile could clearly error out if the interface is not satisfied.
Overall, this is neither important nor required, as it is not too bad to check for a bunch of functions manually and provide adequate error messages to the user yourself.
Issue 2: Enum availability
When the app does a request to the server, it gets a response back (letās pretend itās HTTP). As the commands do not care about the details of HTTP but only about the content and state logic, the app pre-parses the response. The Roto script that implements the command wants to know what the result status (200 OK, 404 NotFound, ā¦) is of the response. As Roto provides a match statement, this seems the perfect use case for that. But how do I convert my Rust enum into a variant type that is available in Roto? Just wrapping it in Val<Enum> didnāt cut it. Maybe I am holding it wrong? (Also, why are they called variants and not enums?)
Issue 3: Keeping State
Letās imagine a (Roto based) command in the app that requests a resource from the server. The app handles the most of the annoying http protocol, does the request and returns the payload of the servers response to the script by calling the process(payload) function. The script does some work and concludes that it need to do another request. Once again the app does itās thing (e.g. waiting for network delay, following redirects, what ever) and once it receives the response, calls the process(payload) function once more. Here is the problem: How does this Roto process() function know, if it has received the first response or the second? How can the Roto script itself keep a state machine?
(work around)
(A solution that would work for the current state of Roto is to make the commands synchronous, as in, there is only a single function with a loop, that calls into the app to do stuff and only exits when it is done. This is not ideal as the app would not be able to tell the user what the command is currently doing, which state it is in and it would require sophisticated synchronization efforts to allow multiple commands to run in parallel, not blocking the app when waiting for network, canceling commands when the network goes down etc. I would loose some of the advantages I hope to gain from Roto!)
Of course I have seen the Runtime<Ctx> context. Itās almost exactly what is needed here. But only almost. For one, the Roto documentation does not mention if the context is mutable from within Roto? Given the docs and the example I would assume not (didnāt test it/read source). Second, the context is defined by the Rust side. This is not an option here, as the app does not know what type of context data any given script might want to store.
Option 1:
A blunt solution could look like this: The app kindly asks the script to tell it what the static context is. In this example via the give_cookie function. This cookie of context is then passed along with every other function call. This does not work for multiple reasons (but solvable ones). First, Rustc will not like this naive type earasure. Second, the passed cookie will be passed by value to the Roto functions, making it mutable is non trivial (iirc thereās another forum post on this?).
A.roto:
record Foo {
# ...
}
fn give_cookie() -> Foo {
return Foo { .. };
}
fn default_task(cookie: Foo, marmelade: i32) -> bool {
...
if cookie.hidden_data {
cookie.hidden_data = false;
return false;
} else {
return true;
}
}
B.roto:
record Bar {
# ...
}
fn give_cookie() -> Bar {
return Bar { .. };
}
fn default_task(cookie: Bar, marmelade: i32) -> bool {
if !cookie.secret {
cookie.secret = true;
return false;
} else {
return true;
}
}
main.rs:
fn run_roto(file: &str) {
let lib = library! { ... };
let mut runtime = Runtime::from_lib(lib).unwrap();
let pkg = runtime.compile(file).unwrap();
let cookie_jar = pkg.get_function::<fn() -> Val<_>("give_cookie").unwrap();
let task = pkg.get_function::<fn(_, i32) -> bool>("default_task").unwrap();
let cookie: _ = cookie_jar.call();
loop {
if !task.call(cookie, 15) {
...
}
}
}
fn main() {
run_roto("A.roto");
run_roto("B.roto");
}
Option 2:
A more desirable solution could be the introduction of the static keyword to Roto. In that case, each script could define their own context as they like within Roto (e.g. record Foo {}). Then instanciate it on runtime.compile() with default values using the new static keyword. Similar to the Ctx, this static variable will be available on every function call and will be mutable, as to advance state machines etc.:
A.roto:
record Foo {
# ...
}
static cookie = Foo { ... };
fn default_task1(marmelade: i32) -> bool {
if cookie.hidden_data {
cookie.hidden_data = false;
return false;
} else {
return true;
}
}
B.roto:
record Bar {
# ...
}
static cookie = Bar { ... };
fn default_task(marmelade: i32) -> bool {
if !cookie.secret {
cookie.secret = true;
return false;
} else {
return true;
}
}
main.rs:
fn run_roto(file: &str) {
let lib = library! { ... };
let mut runtime = Runtime::from_lib(lib).unwrap();
let pkg = runtime.compile(file).unwrap();
let task = pkg.get_function::<fn(i32) -> bool>("default_task").unwrap();
loop {
if !task.call(15) {
...
}
}
}
fn main() {
run_roto("A.roto");
run_roto("B.roto");
}
Thatās it for now.
Overall Roto looks good and promising. The documentation that is there is of good quality and helpful. Sometimes I whished for a bit more documentation or more expressive / verbose examples but given the young age of the project I wonāt complain about it. I think more back references from the Roto language docs to the Roto library docs would help (āNice Roto feature, but how do I make it available in my runtime?ā) in addition to the already existing code examples, which I loved! I also liked the error messages, they made it easy to fix my mistakes. Thereās a bit of danger with the naming this project uses for documentation, so here is your heads up: Be sure to differentiate for the user between āAPI documentationā, ādocs.rsā, ādocsā. ālanguage referenceā, ādocumentationā and āroto.docs.nlnetlabs.nlā.
Thanks for taking a look, happy to hear the discussion,
Teufelchen
Iām still not sure whether this is the right choice. Iāve written some justification here, but feedback is welcome: