Could we port OMicroB to the Numworks CPU (an ARMv7-M Cortex-M7 on 32bits) ? Step ONE in a larger project #36
Comments
|
Hi, thank you for the detailed plan ! I'll comment on the first goal and your current issues. The lack of First, we probably need to tweak the configuration that OMicroB is using when calling arm-gcc to fit that expected for numworks binaries. Maybe you could share with me the repo of your "baby C app", so I can get up to speed on exactly what this configuration is, and test on my side ? Unfortunately I do not own a numworks calculator, but I can probably get one fairly quickly; in the meantime, I can at least see if things compile. |
|
Hi, thanks for your quick reply. On one side, I think I can easily find all the options that I should change in the compilation process (when On the other side, I know I'm not that skilled enough to write drivers and all such complicated code... |
|
From what I was able to read in the |
Pretty much yes, you can only run the program in its totality. |
|
I have created a new branch (https://github.com/stevenvar/OMicroB/tree/numworks) and started to do some tweaks. Currently, OMicroB should produce a binary compatible with numworks, but that does not even start the OCaml interpreter. Maybe you can give it a try ? You can configure omicrob with |
|
It looks like the changes I started to implement on OMicroB, only more mature and cleaner, thanks! |
|
I think the /usr/bin/ld : /home/lilian/publis/OMicroB.git/lib/libcamlrun.a(bindings.o) : dans la fonction « caml_numworks_print_string »
bindings.c:(.text+0x22b) : référence indéfinie vers « microbit_print_string »
collect2: error: ld returned 1 exit status
File "hello.ml", line 1:
Error: Error while building custom runtime system
make: *** [/home/lilian/publis/OMicroB.git/targets/numworks/tests/hello_world/Makefile:11 : hello.elf] Erreur 2
make : on quitte le répertoire « /home/lilian/publis/OMicroB.git/targets/numworks/tests/hello_world » |
|
I tweaked a few things:
|
|
With these changes, the |
Oh right, this is a mistake on my part. I will fix it.
I think all of these errors are due to things I forgot in the linking parameters. The first three should be easy enough to fix by following the default app Makefile. The other ones probably require adding some - lm,... flags. I will look into it. Is it possible for me to test this flashing website without owning a calculator? |
|
These should be added to the file defining the |
|
I've pushed a new commit which should hopefully fix all of that, let me know if that works :) |
|
Who nice, the names and icons are correctly found on the flashing website. I guess the issue could be solved by what was proposed in the // TODO: Check why __exidx_start/__exidx_end is needed
void __exidx_start() { }
void __exidx_end() { }I'll try that! |
|
It didn't work.The issue is harder. https://stackoverflow.com/questions/5764414/undefined-reference-to-sbrk seems to reference this same issue, and I'm talking to a very skilled person on the Omega (a fork of Epsilon, the OS of Numworks) Discord server. In the options given to the linker, we have |
|
I'm not sure I understand, does that mean the example app (https://github.com/numworks/epsilon-sample-app-c) does not compile as is ? If it does, what is the fundamental difference between it and our binary (when using |
|
Exactly, the example app does not compile when the linker has the option With the latest commit, the flashing website gives this error: It seems even worse 😭 |
|
No, I think we're making progress. I pushed a new commit with these three stub functions. Does it work better ? |
|
YEAS! It works! I tweaked the |
|
Ah good. Do you mean that with the version I put (which already had a while true loop at the end), the string did not stay on the screen ? How did you change the ml program exactly ? |
|
I was afraid that the version you had would be impossible to exit, so my |
|
Oh right, that makes sense. I dont know exactly how "premptive" the OS of the calculator is, so it's probably a bad idea to block forever you're right. A good Step 2 would be to add some well chosen primitives to allow for writing interactive programs (like polling for the buttons, drawing on the screen, etc). I can give you some pointers on how to implement these if you want. We could also port some of the existing programs and benchmarks to see the how much power we're dealing with compared to what we had previously. Then we can start thinking about the larger goal, writing a REPL (Read-Eval-Print-Loop) that runs on the calculator. The main difficulty will be to fit the OCaml compiler in there of course. |
|
Congrats for making it work, it's already a good step forward! |
|
For step 2, maybe we could first focus on a non interactive system. For example, there exists plenty of "tiny OCaml-like language" implemented in OCaml itself, like for instance a student project Mini Caml Interpreter. |
|
I think if we can have a tiny OCaml-like interpreter that executes a given string, it shouldn't be impossible to connect it to the local filesystem of the calculator. The main idea here is that we could leverage the power of the internal Python editor! |
|
Just to show you, because it's nice to have achieved together this first step 🎉 ! |
|
I started writing additional OCaml functions, to see a little bit how it works.
|
|
I'm struggling to understand what files to modify, between But now I'm trying to add a So the |
|
That's because this specific compilation pass is for the PC binary, which |
|
Hm could we not implement "fake" UI functions of the Numworks in the simulator? I don't really care about the simulator actually. |
|
Indeed, the prolog app and the minicaml one were just baby steps to start experimenting with all that. It's satisfactory that they work well now, but none of them were the real goal. I would prefer porting an interpreter rather than a compiler: as a user of this "dreamed OCaml-Numworks app", I think the goal should be to get the type of a function (or the value of some function evaluation), quickly written in the (Python) text editor, saved as "ocaml.py" and ran from the Numworks OCaml app. |
I've seen a flag like |
|
No not really, this flag just tells gcc not to include the runtime tooling for C++ exceptions in the final binary, it has nothing to do with OCaml. |
|
I delved a bit into the Toploop module of OCaml, and wrote this script which could ideally be compiled to the Numworks ( (* OCaml Interpreter for the Numworks *)
let long_delay = 3000
let exit (code:int) = ()
let filename = "ocaml.py"
let default_program = "
(* Example of an OCaml script to use
with the OCaml-Numworks app *)
let rec fibonacci n =
if n <= 1 then
n
else
fibonacci (n-1) + fibonacci (n-2)
;;
"
(* https://stackoverflow.com/questions/55689054/ocamlbuild-with-toploop-toplevel *)
let eval code =
(* let as_buf = Mylexing.from_string code in *) (* see below *)
let as_buf = Stdlib.Lexing.from_string code in
let parsed = !Toploop.parse_toplevel_phrase as_buf in
ignore (Toploop.execute_phrase true Format.std_formatter parsed)
let () =
Toploop.initialize_toplevel_env (); (* to initialize the toplevel env, mandatory *)
clear_screen (); (* remove if not compiling for the Numworks *)
print_endline ("Loading code from '" ^ filename ^ "' ...");
let file_content = read_any_file filename in
let file_content = if file_content = "" then default_program else file_content in
print_endline "Parsing the file...";
eval file_content;
print_newline ();
delay long_delay; (* remove if not compiling for the Numworks *)
exit 0I managed to compile this script (just without the few lines that rely on our new Numworks stdlib, so the calls to $ ocamlbuild interpreter.byte -pkgs compiler-libs,compiler-libs.toplevel
$ ./interpreter.byte
Loading code from 'ocaml.py' ...
Parsing the file...
val fibonacci : int -> int = <fun>😄 This behavior is exactly what I hope to obtain, for the "OCaml interpreter" Numworks app: read and parse the content of a (predefined) Python file from the local storage, run the toplevel loop on its content, and display the output. The compilation with 😭 The compilation with So I brutally tried to copy the entire content of the
So from this point forward, I don't really know what to do or to try. @Vertmo if you followed until here and have any idea... thanks in advance (again). |
|
Hi, |
|
Hi :) Just for the I'm kinda overwhelmed by all the files in the |
|
Well, the good news is, we have someone on this project who knows a lot about |
|
Do you think it could be worth it to open a question on https://discuss.ocaml.org/, asking how should I start in this direction of recompiling |
|
Regarding |
|
Yes, I think most of the complexity of |
|
Regarding
I've worked on this repository and on my fork, separately, so I'll have to work a bit to merge things, I'll try that this week. |
|
I've worked for a couple of hours on trying to compile the above script with I think I manually resolved the issues with the There is not even a |
|
I'm lost. |
Indeed, the OCaml compiler is a complex piece of software, it makes sense that it would require a lot of work. I'm sure we can get there.
I think that's because the |
|
Thanks for the encouragement!
Thanks! |
|
|
Regarding the version of OCaml, it's also what the badge in the README.md shows: 4.07. Thanks. |
|
It's quite heavy, every module relies on everything: to have a "simple" eval function as I hoped, the cmo and cmi and cmt format modules have to be included, and the Marshall one too. Either this will be very hard to port to the Numworks, or in fact the Toploop eval part can run without needing to generate cmi/cmo/cmt files and without Marshall-ing anything. |
|
I managed to get the The generated C file is very large (13 Mb). I guess it includes the bytecode for the all the OCaml files included in the compilation command - and there are a lot! Many errors like: interpreter.c:325820:23: error: 'caml_obj_tag' undeclared here (not in a function); did you mean 'caml_obj_dup'?
325820 | /* 32 */ (void *) &caml_obj_tag,
| ^~~~~~~~~~~~
| caml_obj_dup
interpreter.c:325824:23: error: 'caml_ml_input' undeclared here (not in a function)
325824 | /* 36 */ (void *) &caml_ml_input,
| ^~~~~~~~~~~~~
interpreter.c:325827:23: error: 'caml_ml_flush' undeclared here (not in a function); did you mean 'caml_mul_float'?
325827 | /* 39 */ (void *) &caml_ml_flush,
| ^~~~~~~~~~~~~
...Because all the primitives are not implemented in the C runtime, I guess. I have to stop working on that now, maybe I'll find time to continue next week. Sorry 😞 |
|
And due to the changes I added in |
|
I fixed back the three previously working examples, and continued to tweak on I think right now, my key issue is here: in the C code in
|
|
Great ! This is progress ! I've been trying to compile the work you pushed, but I seem to be missing a
Indeed, that's a lot. We will try to get that down :)
These errors are a bit weird I admit. I cannot reproduce them right now on your branch, because I think you commented the allocator calls ? I will try to get a look at them shortly :) |
|
Indeed I forgot about the |
|
I manage as well to compile the $ ./interpreter.byte
./interpreter.byte: symbol lookup error: ./interpreter.byte: undefined symbol: caml_static_alloc |
|
Yes, I ran in the same issue at first. |
|
Awesome! I won't have time before Sunday so I'll just read the commits you upload and the comments here. Thanks! |
|
Ok, let's do a little recap of the progress/what's left to do: First, @bvaugon has been really helpful with the M&C GC bug: he found it, resolved it, and even found another unrelated bug. I merged his work on our branch. Second, on the ocaml-interpreter front: I've made some progress in eliminating unsupported C-functions in the Stdlib/Compiler Libs. Unfortunately there are at least two functions left that I fear will be an issue: The first is The second is Ok, sorry for the long text, but as you can see these are somewhat critical issues which we need to think about carefully before we continue. Let me know what you think :) |
|
Hi, thanks for the recap, and congratulations to @bvaugon for fixing these two bugs in the GC! |
Hi there everybody 😃 !
As the title of the issue suggest, I would like to continue the work done by @Vertmo into porting the OMicroB project to Micro:bit, and port it now to another ARM32 processor, which is very close to the Micro:bit and thus it shouldn't be too hard. From what I understood, the Micro:bit is running on a Arm Cortex-M4 32 bit processor with FPU, with RAM 128KB and at 64MHz. Quite less than what the Numworks calculators offer, but in a very close setup!
About the Numworks
The Numworks calculators are open-source: their OS is Epsilon, written in C++ (and parts in C).
Its CPU is an ARMv7-M Cortex-M7 on 32bits, with its core clocked at 216 MHz and 256K of Static RAM. There exists two other open-source fork OS, Upsilon and Omega. I'm aiming only at Epsilon, to aim at the largest possible audience.
By default on the main OS, there is already a MicroPython interpreter as well as a (very good) text editor. All that is written in C++ and in C. It works amazingly well, considering the limited hardware...
My goal(s)
My first goal: I would like to port OMicroB to the Numworks devices. That is, being able to write on my laptop, in OCaml, tiny code that could be compiled to bytecode, and then to the correct assembly language (then binary) for the CPU of the Numworks, using OMicroB.
But because of the nature of the calculator environment, I don't want to flash the program to the Numworks. If I do that, I'm most certain that nothing will work. And even if it worked, that would overwrite the OS of the Numworks, that's not what I want.
Instead, such assembly language or object (
.arm_ofile) could then be used to be embedded in an application for the Numworks (see below).My dream goal: it is to have the same experience as the Python app (that is, a full text editor, interpreter + terminal) for the OCaml language.
If not possible, most likely because the RAM of the Numworks is limited, then I could focus instead on the camllight language, or even on tinier OCaml-like languages (like Mini-Caml-Interpreter).
My guess is that, if the language interpreter itself is written in pure C (like it is the case for Lua, see below, or for caml-light), this should be possible: all I would need is to have a
execute_this_phrase(const char* line_of_code)function, available in the rest of the C code.What I tried so far
It is possible to write an installable ("flashable") application (a
.nwafile) either in C++, but also in C and in Rust. Following the tutorial, I successfully wrote a tiny C application, which does nothing interesting yet, to try it out.I want to add at least one object file, compiled for the correct ARM32 bit architecture, to this C application, like what is done for the lua app (see below also).
I tried (for hours, all weekend long) to obtain on my laptop a compiler for OCaml that could produce object code for ARM 32bits, this way I hoped of being able to follow the official tutorial which explains how to include compiled OCaml code in a C app. I wanted to have a tiny C app for the Numworks, where a very simple basic function could be written in OCaml separately (on the laptop), and the called in the app.
I tried all the projects I could find online, none of them worked for me (and I tried a lot). This lead me to be interested in OMicroB.
Yesterday, I tried using the
omicrob -device microbit2compiler, to produce a.arm_oobject file, from a very tiny OCaml application, targeting an ARM32 processor.I encountered several issues, mainly that there are no
print_...functions, thus making the whole process quite hard: if the OCaml code cannot print to the screen of the Numworks, it is quite pointless. Let's address this issue later.Then when I focused on a simpler program, I managed to compile it (and obtain a bunch of compiled file, one being the
.arm_oobject file).I tried to include it in my baby C app, and got a lot of compilation issues (from what I gathered, because of float-abi=hard vs float-abi=softfp differences, and because the
main.cfile declares amain()function and the compiled OCaml code also declares amain()function).An app for Lua: a possible inspiration
And... There is also the lua application, which ports a full Lua 5.4.4 interpreter on the Numworks. It doesn't ship a text editor, and only loads and prints the result of one script, that has to be shipped with the app while installing it. It's not amazing in terms of user experience, but it's already great.
If that is not possible to ship a text editor + interpreter (+ terminal/toplevel?) on the Numworks, then at least I would be happy to be have what the Lua app provides, or even less.
Many thanks for reading all the way down. I'll be actively working on this for the next two weeks at least. Thanks if you can help!
The text was updated successfully, but these errors were encountered: