BUILD.sh script failing

[pak-man]

BUILD.sh fails when compiling kendrite SDK:
===[ 'mklfs' created ]===
===[ BUILDING MicroPython FIRMWARE ]===
.
.
-----[ source k210_env ... ]-----
-----[ k210_env export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/username/micropython/MAIX/MicroPython_K210_LoBo/k210-freertos/../kendryte-toolchain/bin/ ... ]---

=====[ PLATFORM_MK ./mpy_support/k210.mk ./platform/drivers/k210.mk ./third_party/k210.mk ./third_party/spiffs/k210.mk ./k210.mk ]=====
=====[ Compiling mpy_support docs platform third_party ... ]=====
=====[ compile mpy_support docs platform third_party . ]=====
=====[ AllDirs ... ]======
=====[ drivers/ ]======
=====[ sdk/ ]======
w25qxx.c:44:10: fatal error: devices.h: No such file or directory
#include <devices.h>
^~~~~~~~~~~
compilation terminated.
make[2]: *** [Makefile:74: w25qxx.o] Error 1
make[1]: *** [Makefile:59: compile] Error 2
GEN build/genhdr/qstr.i.last
../../micropython/py/mpstate.c:27:10: fatal error: py/mpstate.h: No such file or directory
#include "py/mpstate.h"
^~~~~~~~~~~~~~
compilation terminated.
../../micropython/py/malloc.c:31:10: fatal error: py/mpconfig.h: No such file or directory
#include "py/mpconfig.h"
^~~~~~~~~~~~~~~
compilation terminated.
../../micropython/py/gc.c:33:10: fatal error: py/gc.h: No such file or directory
#include "py/gc.h"
^~~~~~~~~
compilation terminated.
../../micropython/py/pystack.c:30:10: fatal error: py/runtime.h: No such file or directory
#include "py/runtime.h"
^~~~~~~~~~~~~~
compilation terminated.
../../micropython/py/qstr.c:31:10: fatal error: py/mpstate.h: No such file or directory
#include "py/mpstate.h"

[loboris]

Maybe something went wrong when cloning the repository. Try to clone again, after finiched check your local directory, it should have the same structure and files as in the repository.

I've tested with fresh repository clone, and no errors were reported:

boris@UbuntuMate:~$ git clone https://github.com/loboris/MicroPython_K210_LoBo.git
Kloniranje u 'MicroPython_K210_LoBo'...
remote: Enumerating objects: 112, done.
remote: Counting objects: 100% (112/112), done.
remote: Compressing objects: 100% (103/103), done.
remote: Total 2232 (delta 9), reused 53 (delta 6), pack-reused 2120
Primanje objekata: 100% (2232/2232), 36.72 MiB | 7.63 MiB/s, završeno.
Razrješavanja delta: 100% (812/812), završeno.
boris@UbuntuMate:~$ cd MicroPython_K210_LoBo/k210-freertos
boris@UbuntuMate:~/MicroPython_K210_LoBo/k210-freertos$ ./BUILD.sh
Downloading kendryte-toolchain, please wait ...
Unpacking kendryte-toolchain
'kendryte-toolchain' prepared, ready to build
===[ 'mklfs' created ]===
===[ BUILDING MicroPython FIRMWARE ]===
.
.
===[ Creating 'MaixPy.kfpkg' ]===
===[ kfpkg created ]===

------------------------------------------------
   text	   data	    bss	    dec	    hex	filename
1400486	  45745	4007272	5453503	 5336bf	MaixPy
------------------------------------------------

============================
====== Build finished ======
            version: 1.11.5
 Firmware file size: 1454080
 Flash FS starts at: 4194304
============================
boris@UbuntuMate:~/MicroPython_K210_LoBo/k210-freertos$ 

[pak-man]

I tried again:
git clone https://github.com/loboris/MicroPython_K210_LoBo.git -> all fine
./BUILD.sh -> downloads and unpacks your modified version of kendrite toolchain,
but when it starts compiling, same pb occurs.
I checked my env and i found no environment variable related to micropython or kendrite
I use gcc 9.1.1 from Fedora 30 workstation 64 bits.
I have no problem compiling MAIX with its own version of kendrite toolchain.

There might be something in your env that i don´t have and would explain why it fails. Otherwise I have no clue why those micropython headers are not properly included.

[loboris]

I've tried the build on the fresh install of Fedora 30 Workstation in virtual machine and I've been able to reproduce he issue.
This is quite strange as the build on the fresh Ubuntu 18.04 passes without issues.
I've checked, and it looks like all the makefiles are creeated corectly and all the include paths are right.
For now, I have no idea what could be different in the development environment between Ubuntu and Fedora, but I'll try to find the reason for the issue.

[pak-man]

Here is a copy of my env:
myenv.txt

Would you post yours from Ubuntu 18.04 ?
There might be a clue in it. Otherwise, which version of gcc are you running ?

[Qutix]

Hello,

I was able to build the repo on my very first try (and ever since) using Ubuntu 19.04. here are my details:
ubuntu_printenv.txt

uname -a
Linux Qutix 5.0.0-20-generic #21-Ubuntu SMP Mon Jun 24 09:32:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/8/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 8.3.0-6ubuntu1' --with-bugurl=file:///usr/share/doc/gcc-8/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --prefix=/usr --with-gcc-major-version-only --program-suffix=-8 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 8.3.0 (Ubuntu 8.3.0-6ubuntu1)

[loboris]

I think the issue has nothing to do with the local environment.
I've had some success by explicity exporting all the include paths, it compiles ok, but does not link the final binary.

If you are familiar with Docker, I've created a Docker image which includes everything needed to build the MicroPython image and works well on Fedora. The image runs as user mpy and starts in /home/mpy/MicroPython_K210_LoBo/k210-freertos. You can get it executing:

sudo docker pull loboris/ubuntu_micropython
sudo docker run -it loboris/ubuntu_micropython

[Qutix]

Tried to build the latest updated repo on my Ubuntu 19.04 where I was able to build before.
Now I get "Segmentation fault" error messages:

Have you seen this one before? Tried multiple times, always the same error.

[loboris]

My working machine is Ubuntu 18.04, but I've tried also on Ubuntu 19.04 and couldn't get any errors:

boris@UbuntuMate-VirtualBox:~/MicroPython_K210_LoBo/k210-freertos$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 19.04
Release:	19.04
Codename:	disco
boris@UbuntuMate-VirtualBox:~/MicroPython_K210_LoBo/k210-freertos$ 
boris@UbuntuMate-VirtualBox:~/MicroPython_K210_LoBo/k210-freertos$ uname -a
Linux UbuntuMate-VirtualBox 5.0.0-21-generic #22-Ubuntu SMP Tue Jul 2 13:27:33 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
boris@UbuntuMate-VirtualBox:~/MicroPython_K210_LoBo/k210-freertos$ 
boris@UbuntuMate-VirtualBox:~$ git clone https://github.com/loboris/MicroPython_K210_LoBo.git
Kloniranje u 'MicroPython_K210_LoBo'...
remote: Enumerating objects: 112, done.
remote: Counting objects: 100% (112/112), done.
remote: Compressing objects: 100% (103/103), done.
remote: Total 2232 (delta 9), reused 53 (delta 6), pack-reused 2120
Primanje objekata: 100% (2232/2232), 36.72 MiB | 7.66 MiB/s, završeno.
Razrješavanja delta: 100% (812/812), završeno.
boris@UbuntuMate-VirtualBox:~$ cd MicroPython_K210_LoBo/k210-freertos
boris@UbuntuMate-VirtualBox:~/MicroPython_K210_LoBo/k210-freertos$ ./BUILD.sh
Downloading kendryte-toolchain, please wait ...
Unpacking kendryte-toolchain
'kendryte-toolchain' prepared, ready to build
===[ 'mklfs' created ]===
===[ BUILDING MicroPython FIRMWARE ]===
.
.
===[ Creating 'MaixPy.kfpkg' ]===
===[ kfpkg created ]===

------------------------------------------------
   text	   data	    bss	    dec	    hex	filename
1400494	  45745	4007272	5453511	 5336c7	MaixPy
------------------------------------------------

============================
====== Build finished ======
            version: 1.11.5
 Firmware file size: 1454080
 Flash FS starts at: 4194304
============================
boris@UbuntuMate-VirtualBox:~/MicroPython_K210_LoBo/k210-freertos$ 

There is a make executable in MicroPython_K210_LoBo/kendryte-toolchain/bin left there by mistake. I may couse the "segmentation fault" error if this directory is somehow included in your PATH before /usr/bin.
Try to delete it and try to build again.

I'll fix it tommorow.

[Qutix]

It did not help. I'll reinstall the linux virtual machine and check again.

Do you have any guides on how to setup Eclipse? It complains about the kendryte toolchain not found in the PATH, that I could solve by adding the toolchain/bin folder to the PATH environment variable. But there tons of error regarding the MP_QSTR_chdir and the other strings. As far as I know there would be a build step that would generate a file that I should include. I just can't start a build from inside Eclipse.

The Eclipse run and debug profiles are all empty. When I go to Project/Build I get an error message: "No rule to make target 'all'. Stop".

I had to install the PyDev Eclipse module from the marketplace. Are there other prerequisites?

[loboris]

Eclipse project files are included in the repository and contains all needed configuration.
Several Build targets are also configured.

To include The MicroPython project into Eclipse just goto (in Eclipse):
File->Import->General->Existing Projects into Workspace
Select root directory [select MicroPython_K210_LoBo directory]
Finish.
Rebuild index
Execute Index->Freshen All Files.

Tested and works.

[loboris]

QSTRINGS will often be unrecognized by Eclipse after clean and fresh build. In that case just execute Index->Freshen All Files. You only need to do this again after clean or after adding some new Qstrings...

[Qutix]

On a brand new Ubuntu 19.04 with the same commands, same setup I was able to build this repo. The problem must have been in my environment, but I am unsure what that was. Good now.

Thanks for the Eclipse tips. I read them in the readme.md as well... the index rebuild and including the project. But Eclipse somehow could not recognize stuff and index rebuilding did nothing. I'll try it again later. Might have been an installation fault that caused the seg faults as well or I don't know.

[loboris]

You must use Index -> Freshen All Files, not index rebuild.
All paths are relative, so there should be no problem...

[domschl]

As for the initial build failure (basically no include files are found on non-Ubuntu-18.04 platforms), this seems to be caused by incompatiblities between make versions: The include path is composed by Makefile by sinclude several sub-projects include-paths from *.mk file. Those files include \n and \r escapes which are not resolved when using different (probably newer) versions of make. A possible solution would be to change the *.mk file generation, replacing \n with actual newlines and removing the \rs. I do not know if that would break older make versions.

[loboris]

@domschl
Thanks for this hint.
I've made some changes to the Makefiles, the part which generates mk files (the include paths) and now building on Fedora 30 Workstation, for example, works without issues.
It should now work in all distros and all make versions.

I've basically only replaced the $(shell echo ... >mk) with $(shell printf ... >mk) and removed all \n in variables.
As it was, it shouldn't acctually work on Ubuntu to, I'm not shure why it was working...

The update will be commited later today or tommorow.