Run dcurl on armhf based Linux distribution for DE10-Nano Kit

[marktwtn]

Base on the discussion of #174 (comment),
the old image file from LampaLab/iota_fpga should not be used since it is out-of-date and does not support armhf.

On de10-nano board, the dcurl should be run with the compiler flag -mfloat-abi=hard to avoid performance impact and hence we need an armhf image file.
An armhf image is offer by @tylerw1369 in this github repository.
After the dcurl is successfully running with the new image file, we should merge the pull request #174 and change the image file of the other de10-nano boards as well.

[marktwtn]

I cross compiled the dcurl and transfer the executables to the de10-nano board.
They ran well, hence the modification of pull request #174 can be merged if we use the armhf image file.

However, there are something I need to mention:

• The time for burning the image into the micro SD card is quite long. Is it reasonable?
• I can not use apt command to install the tool like GNU Make. It shows the message like

Package make is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'make' has no installation candidate

• For some reason, the FPGA-related executables can not be executed unless we use sudo.
  Output message:

cpow-ctrl open fail: Permission denied

I have not investigated these weird behaviour yet.
I wrote it down for reminding that there are somethings we need to overcome if we are going to use the armhf image file.

[jserv]

@tylerw1369, can you comment for the customized image?

[jserv]

By the way, there is another armhf based Debian GNU/Linux image for de10-nano: https://github.com/ikwzm/FPGA-SoC-Linux/blob/master/doc/install/de10-nano.md

[jserv]

I followed the instructions of meta-de10-nano to build Angstrom-v2018.06 derived minimal image.

Tested on node0:

$ ls /tmp/de10-nano-build/deploy/glibc/images/de10-nano
Angstrom-de10-nano-image-minimal-glibc-ipk-v2018.06-de10-nano.rootfs.cpio
Angstrom-de10-nano-image-minimal-glibc-ipk-v2018.06-de10-nano.rootfs.ext3
Angstrom-de10-nano-image-minimal-glibc-ipk-v2018.06-de10-nano.rootfs.manifest
Angstrom-de10-nano-image-minimal-glibc-ipk-v2018.06-de10-nano.rootfs.tar.gz
Angstrom-de10-nano-image-minimal-glibc-ipk-v2018.06-de10-nano.rootfs.tar.xz
Angstrom-de10-nano-image-minimal-glibc-ipk-v2018.06-de10-nano.rootfs.wic
...

rootfs compressed with gzip is of size 191MB.

[tylerw1369]

Ya I believe the difference is logging in as root on lampas image file vs a usr with sudo privileges on mine. I am going to try and make an image with a standard root login and that should take care of having to always sudo. I'll get on it and let you know when finished.

[tylerw1369]

I'm also going to see if I can get the kernel mod to load automatically at boot as well

[marktwtn]

Ya I believe the difference is logging in as root on lampas image file vs a usr with sudo privileges on mine. I am going to try and make an image with a standard root login and that should take care of having to always sudo. I'll get on it and let you know when finished.

@tylerw1369 I think maybe we should change the file privilege of /dev/cpow-ctrl instead of logging as root.

---

I can not use apt command to install the tool like GNU Make. It shows the message like

Package make is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'make' has no installation candidate

I forgot to use $ sudo apt update first, my bad.
Now it has no problem at all.

[jserv]

Once @marktwtn repackages armhf-based Linux image file (into node0 or similar servers for hosting), we can close this issue first and then think about further improvements and FPGA-specific CI plans.

[marktwtn]

I have tried three different image files and only the one from @tylerw1369 can work.

• tyler1369 repository
  However, the image file is around 8.6 GB, which is too large for the 8GB SD card.
• FPGA SoC Linux
  The steps is unusual. It makes the kernel image as a debian package and uses it later.
  However, I can not even login into the de10-nano board to do the further installation.
  I powered up the board and it did not show anything.
  Shouldn't the kernel be installed before trying to login?
• meta de10 nano
  The repository is forked from Intel.
  Somehow I had build failure on my desktop due to the compiler segmentation fault.
  Hence I decided to use the image file previously built by @jserv on node0.
  And de10 nano board keeps rebooting and print the message like

  [    9.923416] of_fixed_clk: probe of soc:base-fpga-region:clk_0 failed with error -5
  [    9.931347] of_fixed_clk: probe of soc:base-fpga-region:fft_sub_clk_0 failed with error -5
  [    9.940037] of_fixed_clk: probe of soc:base-fpga-region:pll_stream failed with error -5
  [    9.948459] of_fixed_clk: probe of soc:base-fpga-region:custom_reset_synchronizer failed with error -5
  

I have not tried the method offered by Linux on ARM DE10-Nano Kit.
I'll give it a try.

[tylerw1369]

I will lower the size of my image file to suite your needs. It will be easy and then you can fit it on your cards. It doesnt need to be that big in the first place as i only am running dcurl on it as a pow node. Im at work today but ill get it done tomorrow when I am off.

[tylerw1369]

Here is the smaller image file. Its 2.1 gigs.
https://drive.google.com/open?id=11QwliFc9okIU7HgwvQam2UTkUfVfClPe
Let me know if you have issues with it.
@marktwtn
Also im sure changing the file permissions for the /dev/cpow-ctrl would work. I will have to check it out.

[jserv]

Here is the smaller image file. Its 2.1 gigs.
https://drive.google.com/open?id=11QwliFc9okIU7HgwvQam2UTkUfVfClPe
Let me know if you have issues with it.

Thank @tylerw1369 for this work. Is it based on Ubuntu Minimal?

[tylerw1369]

It's been a while since I made it but yes if I remember correctly it was based off ubuntu minimal. It pretty much has everything to run a dcurl pow node via node js and your dcurl software and nothing more. I didn't add anything to it to bog it down. I believe it runs 4.14.73 ltsi for the kernel. Also going with armhf really helped me get rid of the errors and problems I had with the image from lampa. It runs 100 percent better.

[marktwtn]

@tylerw1369 the partition seems weird.
The whole root file system is allocated with 1.6 GB and the available space has only 307 MB remained.
And there is a 5.9 GB space which is not partitioned at all.

I ran the dcurl testing files with the cross compiler on de10-nano, and everything went well.
However, it might have problem when we try to use remote worker since RabbitMQ uses OpenSSL.
Native compiler: Normal Linux distribution comes with OpenSSL and Ubuntu Minimal does not. It needs to be installed.
Cross compiler: It prints out the message like

 In file included from .../dcurl/deps/rabbitmq-c/librabbitmq/amqp_openssl.c:31:0:
.../dcurl/deps/rabbitmq-c/librabbitmq/amqp_openssl_bio.h:26:10:
fatal error： openssl/bio.h：No such file or directory
 #include <openssl/bio.h>
          ^~~~~~~~~~~~~~~

I have not figured out how to solve the problem.

[tylerw1369]

@marktwtn Ok I hope it works for you this time. This image should be perfect to work for an 8gb card.
https://opacity.io/share#handle=8d2d66d77bde24f15adfb565168706522e5febec91e64d20ab49bc54c056f85b53d03adc0207a9965140fb883b0fc0b73846593e920d8721f671c97490962627

[tylerw1369]

@marktwtn Also I got dcurl to install and compile correctly even with the BUILD_REMOTE=1 on my image file. I cloned openssl github and compiled it on my nano then ran dcurl and it finished making with no errors.
[100%] Built target test_sasl_mechanism
make[2]: Leaving directory '/home/tyler/dcurl-update/dcurl/deps/rabbitmq-c/build'
make[1]: Leaving directory '/home/tyler/dcurl-update/dcurl/deps/rabbitmq-c/build'
CC build/curl.o
CC build/constants.o
CC build/trinary.o
CC build/dcurl.o
CC build/implcontext.o
CC build/common.o
CC build/pow_fpga_accel.o
CC build/remote_common.o
CC build/remote_interface.o
CC build/test-trinary.o
LD build/test-trinary
CC build/test-curl.o
LD build/test-curl
CC build/test-dcurl.o
LD build/test-dcurl
CC build/test-pow.o
LD build/test-pow
CC build/test-multi-pow.o
LD build/test-multi-pow
LD build/libdcurl.so
CC build/remote_worker.o
CC build/worker-curl.o
CC build/worker-constants.o
CC build/worker-trinary.o
CC build/worker-dcurl.o
CC build/worker-implcontext.o
CC build/worker-common.o
CC build/worker-pow_fpga_accel.o
CC build/worker-remote_common.o
LD build/remote-worker
rm build/test-trinary.o build/test-curl.o build/test-pow.o build/test-multi-pow.o build/test-dcurl.o
tyler@arm:~/dcurl-update/dcurl$

I havent followed to closely the remote worker part so Im not sure how I would test it to see if it indeed functions correctly.

[marktwtn]

@marktwtn Also I got dcurl to install and compile correctly even with the BUILD_REMOTE=1 on my image file. I cloned openssl github and compiled it on my nano then ran dcurl and it finished making with no errors.

Thanks for the testing. I'll keep checking the functionality of the remote worker.

---

I tried the new image and I can not boot correctly.
It kept showing the message like:

spl: fat register err - -1
### ERROR ### Please RESET the board ###

Did boot loader have something wrong?

I burned the image with the command
$ sudo dd if=./XXX.img of=/dev/sdX bs=1M

[tylerw1369]

Odd I'll check when I get home from work and see if something is wrong with the image

[tylerw1369]

Ok I re-imaged the file. Try this
https://opacity.io/share#handle=a2da86d7c72ac6496ae454601365cf1442d568b3daa78a3eaa9ba8637a2b4222ebc3d62468a9ff9fd72c27371f6fbdec0a7d0eac3d4753f7277fff6ffb52ffa5

[marktwtn]

Ok I re-imaged the file. Try this
https://opacity.io/share#handle=a2da86d7c72ac6496ae454601365cf1442d568b3daa78a3eaa9ba8637a2b4222ebc3d62468a9ff9fd72c27371f6fbdec0a7d0eac3d4753f7277fff6ffb52ffa5

It still has the same error.
Does this image file work well on your SD card?
If so, I will find another SD card to give it a try.

[tylerw1369]

yes i can boot from it on my sd card. let me try uploading it via google
https://drive.google.com/open?id=1MFkvts_zdfZuN8N_HnOimkK0NSsayABG

[tylerw1369]

Its not your card for some reason the image file wont load please hold

[tylerw1369]

@marktwtn Ok for some reason i cant get partition 1 to copy correctly. Download this partition and restore this image to partition 1 and it will work. I dont know why it wont work work with the 8gb image file but this should fix it so it boots correctly.
https://drive.google.com/open?id=12yGYPtH6dotN1z9f56BGcO3CdISQuPjU

[marktwtn]

@marktwtn Ok for some reason i cant get partition 1 to copy correctly. Download this partition and restore this image to partition 1 and it will work. I dont know why it wont work work with the 8gb image file but this should fix it so it boots correctly.
https://drive.google.com/open?id=12yGYPtH6dotN1z9f56BGcO3CdISQuPjU

@tylerw1369 it works now. Thanks~

[marktwtn]

The dcurl, including the remote worker works well on the armhf based Linux distribution.

Thanks for the help of @tylerw1369 again.
By the way, you can use the command $ sudo apt-get install libssl-dev to get the header files and libraries of OpenSSL instead of building them from the source code.

---

Once we decide what software to be included in the Linux distribution, I will create another image file for deploying.