Understanding how Micropython interacts with ESP32 ROM, SRAM, SRAM in RTC, and PSRAM

[git-n-pissed]

I'm trying to select between several different ESP32 modules to use for upgrading a product which is currently using a ESP-12F. I'm going to ask the questions that I feel like I should know at this point in my life, but am not confident enough in what I think I know to place a DigiKey order of 10K units.

Datasheets are attached for three of the ESP32 module groups I'm considering. I say "groups" because there are several variants in each "group". Those groups are the older ESP32-WROOM-32UE and ESP32-WROVER-IE, and the newer ESP32-S3-WROOM-1U. All modules in first two groups use one of two processors (the ESP32-D0WD-V3 (no PSRAM) or ESP32-D0WDR2-V3 (has PSRAM)) and all modules in the last group use the one of the six variants ESP32-­S3 (some with PSRAM, some without).

ROM:
The older module groups have 448KB of ROM, and the newer has 384KB, but does this matter to me or Micropython? When a new image is flashed to a unit, it is installed completely on the SPI flash, correct? Does the ROM ever change? Knowing what the acronym stands for, I realize the irony of that question, but it puzzles me why ROM size is advertised so prominently if its data cannot be changed.

SRAM and SRAM in RTC:
The older module groups have 520KB of SRAM and the newer has 512KB. I understand the purpose of SRAM, but the datasheets for all module groups claims 16KB of "SRAM in RTC", and the ESP32-WROVER datasheet goes into further detail claiming that there is 8KB of SRAM in RTC called "RTC FAST" for the main CPU to use during RTC boot and the same for the co-CPU, but called "RTC SLOW". Does the size of SRAM in RTC matter to Micropython? Would it affect any code I write or is the existence of "SRAM in RTC" just an irrelevant hardware implementation detail?

PSRAM:
This is the one that I'm the most confused about. Many years ago loboris made a post in the old forum where he outlined how to enable PSRAM support in Micropython running on an ESP32:

https://forum.micropython.org/viewtopic.php?t=3553

I've glanced at the Micropython repo to see if PSRAM support had been added in master and it looks like it was added for the ESP32-S2 variants (5093d49), but I'm unclear if it was added for the ESP32-S3 or the ESP32-D0WDR2-V3 used by the older ESP32 variants I am considering. It is not clear to me if, when using a build of Micropython which supports PSRAM on an ESP32 which has PSRAM, the utilization of the PSRAM is seamless or not. Is all SRAM utilized before any PSRAM is used? Do I have to specify what data to put on PSRAM (i.e. some large file, a giant list of strings, etc.)? The ESP-IDF programming guide linked below gives 5 options on how PSRAM can be used, I'm just not sure which one Micropython uses or can use.

https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/external-ram.html#configuring-external-ram

The "default" option of "Provide External RAM via malloc()" with a threshold specified via "CONFIG_SPIRAM_MALLOC_ALWAYSINTERNAL" seems very sensible, but again, it isn't clear to me if that is the way Micropython uses PSRAM.

Additionally, from those of you who have used PSRAM, how much of a performance hit can be expected compared to SRAM?

esp32-wroom-32e_esp32-wroom-32ue_datasheet_en.pdf

esp32-wrover-e_esp32-wrover-ie_datasheet_en.pdf

esp32-s3-wroom-1_wroom-1u_datasheet_en.pdf

[jimmo]

ROM

This is invisible to MicroPython.

We loosely use the term "ROM" to refer to Flash in most cases -- i.e. "ROM is where the firmware is stored". It's read-only from the perspective of the MicroPython code.

(Except that not all flash is "ROM" by this definition of course -- MicroPython carves out most of the flash for the filesystem).

PSRAM

MicroPython supports PSRAM (SPIRAM) on ESP32, ESP32-S2, ESP32-S3, but it has to be enabled during compilation, at which point it's seamless. There are hardware limitations to the total amount of external RAM that can be used (4MiB on some, I'm not sure if that applies to all).

When using external RAM, the MicroPython heap exclusively uses the external RAM, leaving the internal RAM to the IDF (which is used for a bunch of things including SSL and socket buffers... see #8940 for some more info).

Does the size of SRAM in RTC matter to Micropython?

Only if you want to use it.

[git-n-pissed]

Good info on ROM and PSRAM. One of the posts in the link you provided also gives an idea of how slow PSRAM is, saying that a call to gc.collect() can take 100ms.

Could you expand on the SRAM in RTC aspect a bit? How would I "use it"? Are there specific real time functions that utilize it? If so, what are they? If the SRAM in RTC isn't used, can it be used (or is it already used) for anything else (i.e. combined to the normal heap)?

[jimmo]

It's RAM that keeps its content even when the chip is powered off (assuming the RTC stays powered)... i.e. it's called that because it's powered/managed by the RTC peripheral, not because it's inherently "real time".

https://github.com/micropython/micropython/blob/master/ports/esp32/machine_rtc.c#L145

I'm not sure this is documented, but the idea is that you can provide a bytes/bytearray and just stash it there and retrieve it later.

[jimmo]

One of the posts in the link you provided also gives an idea of how slow PSRAM is, saying that a call to gc.collect() can take 100ms.

Yes. This is slow partly because PSRAM is slower, but also because there's typically just a lot of it.

See the bug I linked to above about fragmentation, one thing we'd like to do is make it possible to optionally use less of the total amount (or possibly provide a way to bring the heap size down at runtime), because there's no point collecting the whole 4MiB if your app typically only uses 100kiB.