Counting RMT pulses using pulse_counter.

[c2akula]

Hello Toit-ers,

I run a small studio that works on interactive sculptures and R&D services. I came across Toit recently and been pondering using it for a project which involves using several (8x) ESP32s on a wireless network, each controlling a RGBW strip (>800) leds. A Raspberry Pi acts as a server streaming out pixel values (extract scan lines from a GIF) for each strip 120 times a second. Effectively, the strips act like pixel lines on a screen and refresh at 120fps. Part of the project involves composing LED effects which happens to be ergonomic when working with HSV (I can make a separate feature-request issue about the pixel_strip package elsewhere) before dumping to the strip as RGB (by conversion). To aid with the composition, we currently use a primitive called step(time_interval, target_value) which generates values between [0,target_value] with an increment computed from the time_interval. For example, we would use step(1000, 2) to generate a square wave with a period of one second. Currently, this is done in software and involves expensive % and / operations. If you were to use step(1000, 255) this would generate a saw-tooth wave. When I looked at the plot of the values produced reminded me of how a timer's counter works. Usually, one is able to read the timer's counter values during its operation on an interrupt and do something with the count. In my case, I could use the count to scale, say, the brightness of an LED pixel or shift it's hue value by that amount to another pixel etc.
I figured that the RMT and Pulse Counter peripherals could be used together to achieve this at hardware efficiency without any software arithmetic. I suppose at the high-level this is what I have in mind -

1. Use RMT to generate pulses (the period of the pulses could vary at runtime, for example, in response to the turning of a knob etc.)
2. Connect RMT to a GPIO
3. Connect pulse counter to the same GPIO above

I could imagine this being useful for controlling stepper motors using the STEP/DIR interface but also applying acceleration profiles (fixed at compile-time or computed at run-time) potentially in an isolated task or process with CRITICAL priority. I suppose the question is how, if, this could be achieved with what's already available in the language and standard library? If it can't then any suggestions would be helpful. If things like the one I mentioned about the stepper motor could be done, I'm considering switching shop from C++ to Toit entirely. We already made the switch from Teensy to ESP32 a while ago, although something like TeensyAudio remains a welcome addition to the Toit ecosystem, a discussion for another day.

Thanks,
CA

[erikcorry]

I think it's unrealistic for one ESP32 to control 800 pixels at 120Hz. Too much data per second. It's about 384kbytes/s. Driving the pixels with the UART blows up the memory use by about 3x, so this would require about 10Mbits/s on the serial port, but it is running at only 2.5megabaud.

Put another way the bits on the WS2812B protocol take up 1.25µs so you can send max 800kbits/s. At 32 bits per pixel that's 25000 pixels/s maximum. So at 120Hz you are limited to a chain of max 208 pixels. In practice probably only 100 is realistic.

[erikcorry]

I think for the step() calculations you can do quite a lot with lookup tables (assuming div and mod are too slow), but I don't quite understand the requirements. It's unclear how the frame rate from the gif interacts with the final frame rate in the real world, and how the step function is used to combine the two.

ESP32s are quite cheap and it might be easiest to have more of them to drive the LED strips. Probably you need an actual wire between them to get them 100% synchronized with each other. At least I don't know whether data received over wifi can be time stamped accurately enough.

[floitsch]

I'm now quoting numbers for the SK6812, but the WS2812B should be pretty similar.

The duration of a bit is 1.25us (https://www.smdledlamp.com/Uploads/image/20190618/1560850645423175.png). Given 4 bytes of data for each LED, that gives (1.25 * 32 * 800)us = ~32ms for the whole LED strip. At most this allows a framerate of ~32FPS.

Note: I think there are some pixel-strips that have a higher datarate. In that case 120Hz might be possible. I doubt you would notice it on non-moving objects, though.

[c2akula]

I think for the step() calculations you can do quite a lot with lookup tables (assuming div and mod are too slow), but I don't quite understand the requirements. It's unclear how the frame rate from the gif interacts with the final frame rate in the real world, and how the step function is used to combine the two.

I apologize, the example I used isn't helping. I just wanted to give context to the kind of work/projects where I'd like to use Toit for. As for where the step function is used, that's in composing LED effects at run time. What I mean by composition is this - on one side you have libraries that provide effects like a black-box like here or you have FastLED which gives you a bunch of primitives that you can use to make those black-box effects. Often, you'll find that you have to combine multiple effects. For instance, I'd want to make a snake effect with a fade in it's tail while glittering. The step function is a primitive that acts like a tick counter and aids in such building more complex primitives that then can be used to make full effects like a snake effect or a glitter effect (for example). I'm aware that I can optimize the implementation of the step function but would prefer to use any hardware capability that's on offer by the microcontroller to do it's job which is to count.

The combination of RMT + PulseCounter is attractive because it enables the modulation of RMT pulses dynamically acting as a carrier wave of sorts on which the PulseCounter could piggy-back on.

[c2akula]

Here's how the step is currently implemented. I could optimize it in various ways, but fundamentally, it's a counter and there's a peripheral on the ESP that can do it much more efficiently and provide the values asynchronously, so LED effects/updates can happen in interrupts.

auto step(timepoint t_start, duration interval, int value)
{
        const auto since = now() - t_start;
        const auto capped = since % interval;
        return (capped * value) / t_start;
}

And here's an example effect that uses the step. The effect shifts a chunk of pixels (bar) to the right.

auto bar_shift_right()
{
        const auto n = num_leds / 2 + 1;        // length of the bar
        const auto s = step(t_start, interval, num_leds + n);

        for (int i = 0; i < num_leds; i++)
        {
                leds[i] = (i < s && i > (s -n)) ? max_value : 0;
        }
}

[floitsch]

Here's how the step is currently implemented. I could optimize it in various ways, but fundamentally, it's a counter and there's a peripheral on the ESP that can do it much more efficiently and provide the values asynchronously, so LED effects/updates can happen in interrupts. int8_t step(timepoint t_start, duration interval, uint8_t value) { return ((now() - t_start) % interval) * value / t_start; }

This shouldn't take long to compute in Toit. Primitive calls are relatively fast, but they are still taking a bit of time, so using the hardware instead of a few instructions of Toit is generally not worth it.

[floitsch]

Nice project. One of my projects is to add ~50m of LED strips to my apartment, so I'm hopeful your project works (and if not that we can fix it).
Sidenote: when I recently tried to run the SK6812 (RGBW) I sometimes had an issue that the pixels were shifted by ~5 pixels when I tried to control more than just a few pixels. I didn't investigate much more yet, but if you see something similar, let me know and I will prioritize debugging this.

--

As Erik said, that's a lot of data. Streaming pixel values 120 times a second would be a lot of data. I guess you would only stream individual pixel values more rarely, and then adjust them (for example with the step function).

I looked into using the RMT and Pulse-counter together.
If I understand correctly, you would use the RMT to send pulses at very precise timings. The pulse counter would then count those, giving you the correct step values. In other words, you would increment a counter at very precise intervals.

In theory it should be possible to do this on a single pin, but I wasn't able to do it from Toit. As soon as I reuse the same pin in the pulse-counter and the RMT the other doesn't see the values anymore. (My guess is that the setup functions change the pin's direction to exclusive input/output).

I then connected two pins (32 and 33) and emitted the RMT output on one and counted on the other, and that worked (as expected).

I'm not sure it's a real win, though: the timing would, obviously, be extremely precise and there would be no software interaction, but the division/modulo operations shouldn't be that costly either. Compared to a primitive-invocation (crossing the Toit/C boundary) a division/modulo should still be cheaper. As such, you wouldn't win that much.

That said, it might actually be faster. After all you would switch one primitive call (the one to the monotonic_us) with another (the one to get the pulse-counter's value), but would avoid the division/modulo. I'm not sure it's worth it.

For controlling step motors, we probably want to use the actual MCPWM peripheral (https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/peripherals/mcpwm.html). I haven't looked at it yet, but I would expect that it has all the features that would be needed.

[c2akula]

I would prefer not to expose things like PIN_INPUT_ENABLE ... but if we find a nice way of exposing the functionality, we can add it. We are also trying to make it easier to have custom firmwares, in which case changing yours would be relatively straight-forward.

Perhaps, I don't understand. What would a custom firmware imply here?

I imagine something like doing a Github fork, then applying a patch, or changing the menuconfig. Then you would tell Jaguar (or directly Toit) where the firmware is, and it would use it as a base. It would install the program and run it on it. This way your firmware could build in functionality that others don't have. Things get also easy with Github, as their builders can just build the firmware for you (similar to how we build the firmware right now). Note that I use the term "firmware" here, but I'm really referring to the "base system". Programs can be installed after the base system has been built. This is similar to how you download Toit's image from our Github releases and then install the programs on top.

Again: this would only be necessary, if we don't add it to our image directly.

OK, yes, that makes sense. It'd definitely be useful to have functionality that's specific to an organization's needs. I'd signup for that even if that was a paid service similar to the cloud platform that you have.

[c2akula]

That said: I'm still not convinced you would win a lot by using the pulse-counter instead of just dividing the time. That approach would make sense if you are doing something directly in the interrupt handler, but otherwise the division/modulo should not cost too much. After all, you are only using it once per strip update, correct?

No, it happens every time a pixel needs to be interacted with. For example, we could use step to do a blink on a pixel and you can compose a pixel shift function on top of the step or you can compose a bounce effect where you have one (or several) pixels bounce between the ends of a strip, so for every increment of the leds, you'll need to invoke the step function. In short, no, not per strip update, but per pixel.

I think I'm misunderstanding something. There are only a few pulse counters on an ESP32, so at best you can get 4? values per strip update. (It wouldn't make sense to ask for different values while you update the strip, as they all latch the new value at the same time). Whether you ask for these 4 values from the pulse counter, or whether you compute them shouldn't make a difference in performance.

Sorry, I imagine the two of us have the same picture in mind. Where each step is a value that I could use to scale the brightness of a pixel (for instance). So, you could imagine, you want to take a pixel and scale it's brightness over a certain duration (x-axis) and every increment by the counter would be used in the computation of that pixel's brightness at that time instant. You could also imagine building on top of this behavior to compose more complex effects at a per-pixel or set-of-pixels basis.

[floitsch]

Sorry, I imagine the two of us have the same picture in mind. Where each step is a value that I could use to scale the brightness of a pixel (for instance). So, you could imagine, you want to take a pixel and scale it's brightness over a certain duration (x-axis) and every increment by the counter would be used in the computation of that pixel's brightness at that time instant. You could also imagine building on top of this behavior to compose more complex effects at a per-pixel or set-of-pixels basis.

I think we agree, but I don't understand the value of it. During one refresh cycle you are going to update 800 LEDs. That, by itself, takes ~30ms. The esp32 has 4? pulse counters, so you could get 4 step values from the hardware. Since the 800 LEDs latch their new values at the same time, you can thus use 4 hardware values (the ones you read at the beginning when updating the LEDs) from it. Instead of using hardware, you can just use a call to Time.monotonic_us and compute the step you should have instead. Since you only need to do this 4 times, you wouldn't see any difference in performance. It's basically noise compared to pushing all the data to the UART.

[c2akula]

Sorry, I imagine the two of us have the same picture in mind. Where each step is a value that I could use to scale the brightness of a pixel (for instance). So, you could imagine, you want to take a pixel and scale it's brightness over a certain duration (x-axis) and every increment by the counter would be used in the computation of that pixel's brightness at that time instant. You could also imagine building on top of this behavior to compose more complex effects at a per-pixel or set-of-pixels basis.

I think we agree, but I don't understand the value of it. During one refresh cycle you are going to update 800 LEDs. That, by itself, takes ~30ms. The esp32 has 4? pulse counters, so you could get 4 step values from the hardware. Since the 800 LEDs latch their new values at the same time, you can thus use 4 hardware values (the ones you read at the beginning when updating the LEDs) from it. Instead of using hardware, you can just use a call to Time.monotonic_us and compute the step you should have instead. Since you only need to do this 4 times, you wouldn't see any difference in performance. It's basically noise compared to pushing all the data to the UART.

I believe the effect of having limited no. of counters is that I won't be able to use more than that many effects, assuming that each effect would use a separate counter, so therefore quite limiting compared to a software-based counter? Do I understand correctly?

If I'm using the step function like I show here, then I only need one pulse counter to generate the counts. I could then bind the counter to an RMT and modulate the frequency of this RMT according to my needs. You already mentioned about MCPWM peripheral for motor control, so I wouldn't need the counters for this purpose.

[c2akula]

Sorry, I imagine the two of us have the same picture in mind. Where each step is a value that I could use to scale the brightness of a pixel (for instance). So, you could imagine, you want to take a pixel and scale it's brightness over a certain duration (x-axis) and every increment by the counter would be used in the computation of that pixel's brightness at that time instant. You could also imagine building on top of this behavior to compose more complex effects at a per-pixel or set-of-pixels basis.

I think we agree, but I don't understand the value of it. During one refresh cycle you are going to update 800 LEDs. That, by itself, takes ~30ms. The esp32 has 4? pulse counters, so you could get 4 step values from the hardware. Since the 800 LEDs latch their new values at the same time, you can thus use 4 hardware values (the ones you read at the beginning when updating the LEDs) from it. Instead of using hardware, you can just use a call to Time.monotonic_us and compute the step you should have instead. Since you only need to do this 4 times, you wouldn't see any difference in performance. It's basically noise compared to pushing all the data to the UART.

The current way that the step function is implemented means that if there was some piece of code that were to block then the since would be larger compared to if it were to happen on an interrupt, so updates would be precise. To work around this we compute how much progress should’ve been given the larger since value and scale the output by that amount. This, obviously, has the effect that one gets jumps. This always felt like a dirty hack to me. In an interrupt driven application updates wouldn’t ever be blocked and be asynchronous. I could imagine Toit’s Task being helpful here but that’s still cooperative, so I might need Process instead and orchestrate other stuff between updates of the effects, which depending on the frame rate would give me between 8 and 32ms of budget most of the time. Being able to isolate these kinds of tasks into asynchronous blocks driven by interrupts from peripherals is ideal.

I can see that the rmt module has provision to attach a callback on pulse edges but the pulse counter doesn’t. The pulse_counter example shows polling the counter’s value but no callback to get triggered on update. In the , I suppose I could attach a callback to the gpio pin that’s attached to the pulse counter so that when the counter sees a pulse from the rmt, I could read the counter’s value in that callback?