Generate latest toolchain for ESP8266 - why was support dropped as of 2020r3?

[davydnorris]

I want to build the latest toolchain for use with both the ESP32 chips and ESP8266 RTOS and NonOS SDKs under Windows and Eclipse. The current builds are all created under MSYS and Mingw, and are not compatible with a full Cygwin stack.

I noticed that the last support for ESP8266 chips was 2020r3. Is there a technological reason support was dropped for later versions? I would really like to be able to have cross compilers all using the same versions of GCC, binutils and newlib because I'm starting to migrate my ESP8266 NonOS code over to use the RTOS SDK, and then to ESP32 based chips.

I'm happy to help to add these if required if it allows me to make a recent toolchain for both RTOS and NonOS

[davydnorris]

So I have got the latest toolchain building OK for esp8266 (I had to pull up zlib latest) and compiling OK. I have a couple of issues:

• there were a lot of unresolved symbols during linking, which were almost all fixed by adding -lnosys to the linker command line, but I still have a missing symbol "end". I get warnings about reentrant versions of functions but that looks to be what's expected when you link with libnosys
• I tried adding --specs=nosys.specs to the command line but it doesn't appear to do anything. Should this automatically add the nosys and nano versions of libraries?
• the iram section is about 30k larger than before, which I need to investigate further as this isn't going to fit in an ESP8266. I'm currently using iron versions of libc and libm so will try changing those and see if that makes a difference but there may be further changes needed to the build (possibly removing the floating point formatting options)

By far the biggest changes I had to make were in print formatting, due to the change of definitions of integer sizes. See:
espressif/esp-idf#9511 and
https://docs.espressif.com/projects/esp-idf/en/v5.0.1/esp32/migration-guides/release-5.x/gcc.html#int32-t-and-uint32-t-for-xtensa-compiler

This was annoying for my NonOS code, but it's a deal breaker for the RTOS, where it affects just about every library. What's worse is that I tried using the 'PRIxxx' macros instead and, while that worked fine with the new toolchain, the old toolchain breaks because these macros get switched over based on the sizeof long, not sizeof integer! This means that moving to the new toolchain will be a breaking change unless we work out these macros.

The other thing I am wondering is that every gcc build previously required some local patches to force l32 (Use l32i to access 1- and 2-byte quantities in memory instead of l8ui/l16ui), and I don't know if these are still needed or not. I guess I'll find out once I get a working build that I can flash. If Max Filippov ever reads this, please throw your input into the thread!!

[HamzaHajeir]

Hi there,

Have you reached to a good result for ESP8266 compiler? I'm looking for C++20 support to introduce to the Arduino core.

Thanks

[davydnorris]

Hi @HamzaHajeir - I did get the toolchain built successfully, matching the existing toolchains for the various ESP32 chips.

I then tried to compile some of my firmware and ran into a whole lot of pain:

• the new toolchain definition has changed the definitions of integer and long types. This is described in depth in the repo and in various places. What it means is that the underlying sizes of all the types are unchanged, and the maths is all fine, but any format strings using %x %u %d etc. break.
• I went through my test code and took hours to change the format to %lx %lu %ld and then recompiled and found a few missing functions during link stage. I finally managed to fix those up with a bunch of new linker options and got a clean build, but the code is significantly larger than before and is now too big to fit.
• I then switched back to my old toolchain in order to get a map file so I could compare sizes of functions, and all the format statements broke again! I tried moving them to the new ANSI formats such as PRId32 but they broke too as it appears they haven't been set up properly to work with the new types
• don't even get me started on the ESP8266 RTOS implementation. Every single module has to be edited for format strings so I just gave up on trying to get an RTOS program working

So at this point I have what I think is a working toolchain, but the binaries are too big, and I have to work out how to fix the format strings so they will work with the changes in integer types before I can start to figure out how to slim things down

[HamzaHajeir]

I see, it's a common issue for new espressif toolchains. I think you just can bypass those checks by a compile flag, disable warnings as errors I guess. But what's so strange is the large binaries.

…

On Thu, Sep 28, 2023, 04:43 davydnorris ***@***.***> wrote: Hi @HamzaHajeir <https://github.com/HamzaHajeir> - I did get the toolchain built successfully, matching the existing toolchains for the various ESP32 chips. I then tried to compile some of my firmware and ran into a whole lot of pain: - the new toolchain definition has changed the definitions of integer and long types. This is described in depth in the repo and in various places. What it means is that the underlying sizes of all the types are unchanged, and the maths is all fine, but any format strings using %x %u %d etc. break. - I went through my test code and took hours to change the format to %lx %lu %ld and then recompiled and found a few missing functions during link stage. I finally managed to fix those up with a bunch of new linker options and got a clean build, but the code is significantly larger than before and is now too big to fit. - I then switched back to my old toolchain in order to get a map file so I could compare sizes of functions, and all the format statements broke again! I tried moving them to the new ANSI formats such as PRId32 but they broke too as it appears they haven't been set up properly to work with the new types - don't even get me started on the ESP8266 RTOS implementation. Every single module has to be edited for format strings so I just gave up on trying to get an RTOS program working So at this point I have what I think is a working toolchain, but the binaries are too big, and I have to work out how to fix the format strings so they will work with the changes in integer types before I can start to figure out how to slim things down — Reply to this email directly, view it on GitHub <#40 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AH3O7J5IB2MQHD4HIEDI3WLX4TI4DANCNFSM6AAAAAAU5N43YY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[davydnorris]

@HamzaHajeir - I have found this is a generic GNU C issue, as I use a component tester that is based on the Atmel AVR code and I faced the same issue compiling it when I used a later toolchain.

I actually solved the AVR issue by looking at what Arduino was doing with the Atmel code and the later GNU toolchains and added a bunch of flags, but I already use quite a few of these in my ESP projects.

It may be worthwhile looking at what version of the toolchain is used under Arduino for the ESP8266 and checking the compile flags there - there may be something I have missed

[Lapshin]

@davydnorris , @HamzaHajeir , maybe my message is too late for you. But I will post it here because it may be useful for somebody in future.

Basically, the latest Espressif toolchain release supports ESP8266 assembler compilation but does not provide libc, libstdc++, ...

To build the working toolchain for esp8266 you can do these steps:

1. Define esp8266 in GCC project:

diff --git a/gcc/config/xtensa/t-esp-multilib b/gcc/config/xtensa/t-esp-multilib
index 416338909e1..8d27b2ab224 100644
--- a/gcc/config/xtensa/t-esp-multilib
+++ b/gcc/config/xtensa/t-esp-multilib
@@ -16,9 +16,9 @@
 # along with GCC; see the file COPYING3.  If not see
 # <http://www.gnu.org/licenses/>.
 
-MULTILIB_OPTIONS = mdynconfig=xtensa_esp32.so/mdynconfig=xtensa_esp32s2.so/mdynconfig=xtensa_esp32s3.so mfix-esp32-psram-cache-issue fno-rtti
-MULTILIB_DIRNAMES = esp32 esp32s2 esp32s3 psram no-rtti
-MULTILIB_REQUIRED = mdynconfig=xtensa_esp32.so mdynconfig=xtensa_esp32.so/fno-rtti \
+MULTILIB_OPTIONS = mdynconfig=xtensa_esp8266.so/mdynconfig=xtensa_esp32.so/mdynconfig=xtensa_esp32s2.so/mdynconfig=xtensa_esp32s3.so mfix-esp32-psram-cache-issue fno-rtti
+MULTILIB_DIRNAMES = esp8266 esp32 esp32s2 esp32s3 psram no-rtti
+MULTILIB_REQUIRED = mdynconfig=xtensa_esp8266.so mdynconfig=xtensa_esp8266.so/fno-rtti mdynconfig=xtensa_esp32.so mdynconfig=xtensa_esp32.so/fno-rtti \
 mdynconfig=xtensa_esp32.so/mfix-esp32-psram-cache-issue mdynconfig=xtensa_esp32.so/mfix-esp32-psram-cache-issue/fno-rtti \
 mdynconfig=xtensa_esp32s2.so mdynconfig=xtensa_esp32s2.so/fno-rtti \
 mdynconfig=xtensa_esp32s3.so mdynconfig=xtensa_esp32s3.so/fno-rtti

2. Remove picolibc from the crosstool-NG config. (File samples/xtensa-esp-elf/crosstool.config):
   2.1 Should be done to avoid build error: tls.c:46: Error: invalid register 'THREADPTR' for 'wur' instruction

3. Build crosstool-NG as usual, e.g:

./bootstrap
./configure --enable-local
make
./ct-ng xtensa-esp-elf
./ct-ng build

After this you can use toolchain to compile esp8266:

1. Set export XTENSA_GNU_CONFIG=$PWD/builds/xtensa-esp-elf/lib/
2. Add option -mdynconfig=xtensa_esp8266.so for toolchain calls.

Or instead of using XTENSA_GNU_CONFIG you can use wrappers. You need to create xtensa-esp8266-elf.. files. Look at the comment #65 (comment)

I hope this should work, but I did not make tests if compiled code works on a real chip