Odd behavior when adding to 'modules' directory on ESP82666 GENERIC build?

[Jibun-no-Kage]

Odd behavior when adding to 'modules' directory on ESP82666 GENERIC build?

I added a few modules to the ports/eps8266/modules directory, generated a firmware image, my modules were compiled to mpy, they show up under "help('modules')" view, all seems fine. But... the file system on the ESP8266 is MIA (missing in action). I can't see the file system in Thonny for example. Yes, I have the files view enabled. It is just gone. What happened? There is no customization other than adding a few py files to the modules directory for the given port.

[jimmo]

Sounds like _boot.py is gone?

Rather than adding files into the modules directory, you should be writing a custom board definition with your own manifest.py.

[Jibun-no-Kage]

Has the official method changed? I recall the documentation once said you could do it, and there are many examples via Google that show that method as available. Not saying it is a best practice or such, just that it was once acceptable, or at least doable.

Nor am I discounting you stating creating my own board definition was suggested.

Ah... looking at the _boot.py I must have overwritten it at some point, it only has a reference to import gc and setting the threshold. The reference to the uos.mount for bdev was missing! Opps!

[jimmo]

Has the official method changed?

I'm not sure it was ever "official" but coping files to the modules dir used to work (and does still work). You could also override the path to the modules dir on the make command line to point to your own directory, which was preferable because modifying the micropython repo was awkward and not good practice. Unfortunately overriding the directory caused problems for ports that require specific frozen modules to exist (e.g. esp32, esp8266, rp2) -- it was fine for boards where any frozen modules were optional, e.g. stm32.

So the manifest system is now the official method (for almost three years now). The documentation could use some improving true (although again we can't control stuff that you find on Google unless it's micropython.org or github.com/micropython).

You can write a manifest in two ways -- either pass the path directly on the make command line or write a (out-of-tree) board definition with its own manifest and then specify the board on the make command line.

For the former, see this documentation from @peterhinch https://github.com/peterhinch/micropython-samples/blob/master/fastbuild/README.md#frozen-modules-and-manifests and for the latter see examples at https://github.com/micropython/micropython-example-boards/tree/main/boards

When #8914 and #9037 are merged I promise I will update https://docs.micropython.org/en/latest/reference/manifest.html with the Complete Guide To Writing Board Definitions And Manifests 😁

[Jibun-no-Kage]

As always thanks for the links and such. Looking forward to the updated docs.

The mean time, I will figure it out. :) Since I am likely doing some deep stuff, I think I am still going to end up tweaking some of the 'mpconfig' files, to get the leanest image possible for a 1M board. However, I will try to stick to the manifest based approach. Right now I have an asyncio based solution I have been testing, for the MQTT<->ESP01<->UART<->Pico solution. But since the 1M board definition, does not include asyncio, that is the sticky point right now. Fortunately, the code I have when frozen is not to big or complex.

[jimmo]

This is one of the reasons why writing a board definition is the way to go -- you can do anything you like in mpconfigboard.h, mpconfigboard.{mk,cmake} and manifest.py to configure exactly the build options you like. So you can start with a copy of the 1M config and then modify it as you see fit.

[Jibun-no-Kage]

I keep this up, I am going to know how this firmware generation works inside out... THEN you will all be TROUBLE. LOL

[Jibun-no-Kage]

Yup and in the long run.... easier to maintain.