Cannot make natmod .mpy module for ARCH = armv6m / armv7m if floats are involved

[rkompass]

I cannot make natmod .mpy module for ARCH = armv6m if floats or doubles are involved.

The examples in examples/natmod are mostly made nicely, so my setup seems to be correct (had to install pyelftools, not mentioned in the docs, but no problem).

But in their Makefile ARCH = x64 is defined.

The examples without floats also are made if this setting is changed to ARCH = armv6m or ARCH = armv7m.
But examples/natmod/features2 which has floats involved throws errors:

main.c: In function 'add':
main.c:26:12: error: implicit declaration of function 'mp_obj_new_float'; did you mean 'mp_obj_new_list'? [-Werror=implicit-function-declaration]
   26 |     return mp_obj_new_float(mp_obj_get_float(x) + mp_obj_get_float(y));
      |            ^~~~~~~~~~~~~~~~
      |            mp_obj_new_list
......

The above resulted from cross-checking if I made something wrong.
My original attempt was a benchmarking natmode multiplication function:

/*
	We multiply several times to explore the speed of the FP hardware / software
*/

// Include the header file to get access to the MicroPython API
#include "py/dynruntime.h"

// The function explicitly uses single precision floats
STATIC mp_obj_t mulxy500_f(mp_obj_t x, mp_obj_t y) {
	float r = 1.0;
	float fx = mp_obj_get_float_to_f(x);
	float fy = mp_obj_get_float_to_f(y);
	for (int i = 0; i < 500; ++i) {
	    r *= fx;
	    r *= fy;
	}
    return mp_obj_new_float_from_f(r);
}
STATIC MP_DEFINE_CONST_FUN_OBJ_2(mulxy500_f_obj, mulxy500_f);

// The function explicitly uses double precision floats
STATIC mp_obj_t mulxy500_d(mp_obj_t x, mp_obj_t y) {
	double r = 1.0;
	double dx = mp_obj_get_float_to_d(x);
	double dy = mp_obj_get_float_to_d(y);
	for (int i = 0; i < 500; ++i) {
	    r *= dx;
	    r *= dy;
	}
    return mp_obj_new_float_from_d(r);
}
STATIC MP_DEFINE_CONST_FUN_OBJ_2(mulxy500_d_obj, mulxy500_d);

// This is the entry point and is called when the module is imported
mp_obj_t mpy_init(mp_obj_fun_bc_t *self, size_t n_args, size_t n_kw, mp_obj_t *args) {
    // This must be first, it sets up the globals dict and other things
    MP_DYNRUNTIME_INIT_ENTRY

    // Make the functions available in the module's namespace
    mp_store_global(MP_QSTR_mulxy500_f, MP_OBJ_FROM_PTR(&mulxy500_f_obj));
    mp_store_global(MP_QSTR_mulxy500_d, MP_OBJ_FROM_PTR(&mulxy500_d_obj));

    // This must be last, it restores the globals dict
    MP_DYNRUNTIME_INIT_EXIT
}

with Makefile:

# Location of top-level MicroPython directory
MPY_DIR = /home/raul/Micropython/w600/micropython

# Name of module
MOD = mulxy500

# Source files (.c or .py)
SRC = mulxy500.c

# Architecture to build for (x86, x64, armv7m, xtensa, xtensawin)
ARCH = armv6m

# Include to get the rules for compiling and linking the module
include $(MPY_DIR)/py/dynruntime.mk

Here the compilation succeeds, but the linker gives the error: LinkError: build/mulxy500.o: undefined symbol: __aeabi_fmul.

What am I about to do?

[dlech]

You will need to statically link libgcc.

[rkompass]

All the libgcc.a 's I found in my /usr/lib/gcc/arm-none-eabi/10.3.1/ seem to only point to __aeabi_fmul but not contain it.
I checked it with nm -gC ./libgcc.a | grep fmul in different subdirectories.

[rkompass]

Correction: I have it there:
_arm_mulsf3.o:
00000001 W __aeabi_fmul
should indicate it is in the libgcc.a right?
Now CFLAGS := $(CFLAGS) -mfloat-abi=soft -l:./libgcc.a in the Makefile does not seem to force the static linking.

[jimmo]

Soft floats are not supported in dynamic native modules. The float example only works on architectures with hardware floating point (i.e. not armv6m or armv7m). (The documentation and features2 readme should probably make this more explicit).

The problem is that your dynamic module ends up having to bring in all the soft float support code, because it has no way to link to the support already provided in the main firmware. (The MicroPython dynamic linker can only link the symbols in dynruntime.h). Hardware float is fine because there is no support code required. (e.g. what you're seeing here is the linker error because it needs the soft implementation of float multiply)

You could probably make this work if you were prepared to have a much larger .mpy file (which ends up using a lot more RAM on the target device), but it would be wasteful to duplicate all the floating point support code that is already on the device.

[rkompass]

@jimmo
Even on pyboard CP has an impressing speed advantage (partly >100%):

Timing function: time.ticks_us()
CPU frequency:  168 MHz
sys.platform: pyboard
sys.version: 3.4.0; MicroPython v1.20.0-162-gd080d427e on 2023-06-06
sys.modules: {}
os.sysname: pyboard
os.nodename: pyboard
os.release: 1.20.0
os.version: v1.20.0-162-gd080d427e on 2023-06-06
os.machine: PYBv1.1 with STM32F405RG
001AdditionA: 2.6652s,2.6652s,2.6685s,2.6686s,2.6687s (7498Hz)
002MultiplicationA: 2.6830s,2.6865s,2.6865s,2.6866s,2.6833s (7448Hz)
003DivisionA: 2.6845s,2.6875s,2.6876s,2.6877s,2.6847s (7445Hz)
004ArchimedesPiA: 1.8589s,1.8611s,1.8589s,1.8588s,1.8590s (108Hz)
005MixedBigIntA: 1.4972s,1.4972s,1.4990s,1.4974s,1.4975s (2404Hz)

Timing function: time.monotonic_ns()
CPU frequency:  168 MHz
sys.version: 3.4.0
sys.modules: {}
os.sysname: stm32f4
os.nodename: stm32f4
os.release: 8.1.0   (Circuitpython 8.1)
os.version: 8.1.0 on 2023-05-22
os.machine: PyboardV1_1 with STM32F405RG
001AdditionA: 1.1148s,1.1148s,1.1148s,1.1148s,1.1148s (17941Hz)
002MultiplicationA: 1.1270s,1.1269s,1.1270s,1.1270s,1.1269s (17747Hz)
003DivisionA: 1.1796s,1.1796s,1.1796s,1.2030s,1.1796s (16888Hz)
004ArchimedesPiA: 1.0272s,1.0271s,1.0272s,1.0271s,1.0272s (195Hz)
005MixedBigIntA: 1.4988s,1.5237s,1.5237s,1.5000s,1.5233s (2378Hz)

[rkompass]

Though it seems not to be as precise: bm_004ArchimedesPiA(1) -> (3.14158, 10) on CP instead of (3.141593, 10) on MP.

[jimmo]

@rkompass Very interesting, thanks for sharing!! I can replicate the results here.

I'm not so sure now that the rough 1:2 timing relation is due to fp math (as a thought half a year ago, when I found this).

The fact that the BigInt test is so similar whereas the float ones are so different does point towards float. One important thing about floating point in MicroPython is that they are heap allocated, so any GC/heap differences can show up significantly in floating point benchmarks.

Given my other measurements the interpreter speed is responsible for almost all speed gains/losses.

FWIW, on the main MicroPython benchmark suite (at least the tests that also run on CircuitPython), the results are more similar. Faster for MicroPython for some, faster for CircuitPython for others. Notably the more float-heavy ones are the ones that seem to favour CircuitPython.

$ ./run-perfbench.py -s ~/mpy/circuitpy-pybv11 ~/mpy/micropy-pybv11 
diff of scores (higher is better)
N=100 M=100                /home/jimmo/mpy/circuitpy-pybv11 -> /home/jimmo/mpy/micropy-pybv11         diff      diff% (error%)
bm_chaos.py                    433.67 ->     361.55 :     -72.12 = -16.630% (+/-0.01%)
bm_fannkuch.py                  57.34 ->      74.29 :     +16.95 = +29.561% (+/-6.32%)
bm_float.py                   6905.03 ->    5856.82 :   -1048.21 = -15.180% (+/-0.04%)
bm_hexiom.py                    42.26 ->      46.66 :      +4.40 = +10.412% (+/-4.08%)
bm_nqueens.py                 3519.86 ->    4230.72 :    +710.86 = +20.196% (+/-3.43%)
bm_pidigits.py                 562.22 ->     643.51 :     +81.29 = +14.459% (+/-5.80%)
core_qstr.py                    85.33 ->      64.41 :     -20.92 = -24.517% (+/-11.86%)
core_yield_from.py             364.06 ->     353.28 :     -10.78 =  -2.961% (+/-0.02%)
misc_mandel.py                3582.57 ->    3170.26 :    -412.31 = -11.509% (+/-5.97%)
misc_pystone.py               2145.89 ->    2442.35 :    +296.46 = +13.815% (+/-4.91%)
misc_raytrace.py               430.72 ->     377.61 :     -53.11 = -12.331% (+/-7.89%)

Could it be that Circuitpython has @native mode on by default on all functions?

Surely not? Native code takes typically twice as much RAM, it would be wasteful.

Running your test on MicroPython with the native emitter used for all functions (by compiling to .mpy with -X emit=native) gives:

001AdditionA: 1.9468s,1.9470s,1.9472s,1.9462s,1.9471s (10273Hz)
002MultiplicationA: 1.9646s,1.9649s,1.9647s,1.9646s,1.9650s (10179Hz)
003DivisionA: 1.7949s,1.7963s,1.7965s,1.7973s,1.7966s (11134Hz)
004ArchimedesPiA: 1.4702s,1.4693s,1.4715s,1.4706s,1.4722s (136Hz)
005MixedBigIntA: 1.4031s,1.4035s,1.4034s,1.4023s,1.4069s (2564Hz)

So that @native alone couldn't explain it (and also we'd see a difference in the main test suite).

Though it seems not to be as precise: bm_004ArchimedesPiA(1) -> (3.14158, 10) on CP instead of (3.141593, 10) on MP.

This is definitely worth investigating, there's lots of scope for improved performance at the expense of accuracy. Both are single-point though and I still would have thought the overhead of the interpreter would shadow any differences in floating point implementation performance.

[jimmo]

@rkompass There are two main differences that explain this. CircuitPython compiles with -O2 for all sources (MicroPython uses -Os and -O2 for some critical files), and it uses "object representation C" by default (MicroPython uses "D" by default).

When I compile MicroPython for PYBv11 with those settings it's now marginally faster than CircuitPython on all your benchmarks:

Timing function: time.ticks_us()
sys.platform: pyboard
sys.version: 3.4.0; MicroPython v1.20.0-187-gf01d5fb65-dirty on 2023-06-08
sys.modules: {}
os.sysname: pyboard
os.nodename: pyboard
os.release: 1.20.0
os.version: v1.20.0-187-gf01d5fb65-dirty on 2023-06-08
os.machine: PYBv1.1 with STM32F405RG
001AdditionA: 1.0563s,1.0564s,1.0563s,1.0563s,1.0563s (18933Hz)
002MultiplicationA: 1.0655s,1.0655s,1.0655s,1.0655s,1.0655s (18770Hz)
003DivisionA: 1.1159s,1.1159s,1.1159s,1.1159s,1.1159s (17923Hz)
004ArchimedesPiA: 0.9440s,0.9440s,0.9440s,0.9440s,0.9440s (212Hz)
005MixedBigIntA: 1.2035s,1.2048s,1.2048s,1.2049s,1.2049s (2989Hz)

And for interest, with the @native emitter:

001AdditionA: 0.7276s,0.7273s,0.7277s,0.7274s,0.7272s (27494Hz)
002MultiplicationA: 0.7332s,0.7333s,0.7333s,0.7332s,0.7333s (27276Hz)
003DivisionA: 0.7839s,0.7839s,0.7840s,0.7839s,0.7839s (25513Hz)
004ArchimedesPiA: 0.6829s,0.6829s,0.6829s,0.6829s,0.6829s (293Hz)
005MixedBigIntA: 1.1418s,1.1423s,1.1423s,1.1421s,1.1422s (3152Hz)

And similarly on the performance benchmarks:

$ ./run-perfbench.py -s ~/mpy/circuitpy-pybv11 ~/mpy/micropy-pybv11-C-O2 
diff of scores (higher is better)
N=100 M=100                /home/jimmo/mpy/circuitpy-pybv11 -> /home/jimmo/mpy/micropy-pybv11-C-O2         diff      diff% (error%)
bm_chaos.py                    433.67 ->     465.67 :     +32.00 =  +7.379% (+/-0.03%)
bm_fannkuch.py                  57.34 ->      80.04 :     +22.70 = +39.588% (+/-6.32%)
bm_float.py                   6905.03 ->    7345.78 :    +440.75 =  +6.383% (+/-0.04%)
bm_hexiom.py                    42.26 ->      49.58 :      +7.32 = +17.321% (+/-4.08%)
bm_nqueens.py                 3519.86 ->    4493.95 :    +974.09 = +27.674% (+/-3.43%)
bm_pidigits.py                 562.22 ->     745.21 :    +182.99 = +32.548% (+/-5.81%)
core_qstr.py                    85.33 ->      95.03 :      +9.70 = +11.368% (+/-11.86%)
core_yield_from.py             364.06 ->     365.06 :      +1.00 =  +0.275% (+/-0.02%)
misc_mandel.py                3582.57 ->    3715.56 :    +132.99 =  +3.712% (+/-5.97%)
misc_pystone.py               2145.89 ->    2427.34 :    +281.45 = +13.116% (+/-4.91%)
misc_raytrace.py               430.72 ->     513.37 :     +82.65 = +19.189% (+/-7.89%)

I didn't test rp2/pico, but I would expect exactly the same results. The explanation is fairly straightforward:

• -O2 enables some significant optimisations at the expense of making the firmware about 17% larger. (It's about +8-10% on most of our benchmarks)
• Using object repr C means that floats are now not heap allocated and use the same inline representation we use for interned strings and small integers. This is a huge win for any floating point computation (because about half the time is spent allocating float objects). However this comes at a cost (you guessed it) the accuracy goes to 30-bit (instead of 32-bit), which is why we don't enable it by default. (I note that Arduino enable this on all their boards, e.g. Portenta, Giga, Nicla Vision, etc).

I guess the takeaway here is that some documentation could probably help so if people want to build their own faster firmware, it's good to know about those two tweaks.

[rkompass]

I apprechiated your findings very much, @jimmo, just didn't express that then. Here is the opportunity to make up for it. Thanks.

[jonnor]

It is possible to use soft floating point in dynamic native modules (.mpy). I am doing it in the emlearn-micropython project.
To solve the linker errors like undefined symbol: __aeabi_fmul, one has to explicitly add all the relevant .o files from libgcc.a
Here is an example: https://github.com/emlearn/emlearn-micropython/blob/master/src/emliir/Makefile#L23
Finding the correct .o files for a given symbol is very tedious to do manually, so I made a small script for that:
https://github.com/emlearn/emlearn-micropython/blob/master/tools/find_symbols.py

It might be possible to integrate this kind of approach mpy_ld.py - since it knows all the unresolved symbols, it could attempt to resolve them using libgcc.a

[rkompass]

Thank you very much @jonnor . This is a very nice progress.

If I understand that correctly, then you add the _arm_cmpsf2.o lesf2.o _arm_fixsfsi.o . . . fp function object files to the linking process in case the architecture is armv6m or armv7m.

Several questions do I have now:

1. Where did you get these object files from? My current understanding is: You have to have them as files each. Are they generated as part of processing using find_symbols.py?
2. In case of the ususal MP binaries, e.g. for RP2 these objects are partly existent already in the binary, partly in the firmware ROM. Do you see a possibility to use these already present objects?

From @jimmo 's answer:

The problem is that your dynamic module ends up having to bring in all the soft float support code, because it has no way to link to the support already provided in the main firmware. (The MicroPython dynamic linker can only link the symbols in dynruntime.h). Hardware float is fine because there is no support code required. (e.g. what you're seeing here is the linker error because it needs the soft implementation of float multiply)

You could probably make this work if you were prepared to have a much larger .mpy file (which ends up using a lot more RAM on the target device), but it would be wasteful to duplicate all the floating point support code that is already on the device.

3. Am I right in the assumption that the method you use (link the fp function objects into the .mpy file) end up having this much larger .mpy file?

A question to @jimmo or other maintainers:
Could the symbols for the FP routines be added to dynruntime.h? My current understanding is: This would not bloat the MP binary. Is that correct?

The idea is, at leas I have the feeling, it is:
In the end such libraries like ulab and emlearn could be made as .mpy files without waste of flash space, right?
This would benefit many users, I think. Not everyone is able/interested to recompile his binaries everytime he wants to try some functions from these libraries.
It could make Micropython much more valuable as a learning platform, which it already is, of course.

Thank you very much for your answers.
I will take time and exercise and try to make the benchmarking natmod .mpy module for the RP2 (from above) now.

[jonnor]

1. Where did you get these object files from? My current understanding is: You have to have them as files each. Are they generated as part of processing using find_symbols.py?

find_symbols.py just helps me as a developer to find which files to specify for linking. It takes as input of which symbols were missing (LinkErro exception from mpy_ld). The specified files are extracted into the build directory from libgcc.a (using ar). The relevant parts of the Makefile:

LIBGCC_FILENAME = $(shell $(CROSS)gcc $(CFLAGS) -print-libgcc-file-name)

_arm_cmpsf2.o:
	$(CROSS)ar -x $(LIBGCC_FILENAME) $(SOFTFP_O)

2. In case of the ususal MP binaries, e.g. for RP2 these objects are partly existent already in the binary, partly in the firmware ROM. Do you see a possibility to use these already present objects?

That is another approach. I do not know what is required for that.

3. Am I right in the assumption that the method you use (link the fp function objects into the .mpy file) end up having this much larger .mpy file?

Yes, functions are added to the .mpy module even though they might already exist in the MicroPython image. But in the case of emlearn ML/DSP modules, I typically need 3-10 float functions. Each module is quite small (1-4 kB), even when including these in the module. So I don't really find the duplication to be problematic at all.

RP2 is a bit of a special case, in that they have an FPU which is non-standard. So on that platform this will use emulated floating point, which is substantially slower. But emlearn-micropython is typically 10-100x faster than ML libraries that generate (Micro)Python code anyway.

I think it is very important to not let perfect be the enemy of good. In my book, having support for floating point for natmod that just-works on all platforms would be great - even if it means soft floating point with some duplication (on some platforms). People can then decide if the trade-off is worth it. And we can make it better over time.

[rkompass]

Thanks, this helps (at least me :-) a lot.
Also thank you for finding and adding to this older topic.

[jonnor]

Been working more this more recently, by making fast FFT as a dynamic native module.
Hit a linking issue using cosf/sinf, filed here: #14430
But otherwise it seems to work OK, using the approach described in my previous comment.

[vshymanskyy]

One way to solve it is this PR: #15838

[rkompass]

This is huge. I hope this PR will be accepted soon.

I'm also looking forward to see ulab as natmod mpy module.