From 6fc1ef393c0461969d799294c195ddd1a0c2fdd0 Mon Sep 17 00:00:00 2001 From: Kit Klein Date: Thu, 3 Sep 2020 19:44:12 -0400 Subject: [PATCH] Initial integration of 3.0 release and integrated builder --- .gitignore | 5 +- README.md | 34 +++-- .../app/include/user_config.h | 119 +++++++++++++----- .../app/include/user_mbedtls.h | 29 +++-- .../app/include/user_modules.h | 5 +- .../app/include/user_version.h | 7 +- scripts/build-firmware | 91 ++++++++++---- scripts/spiffs.lst | 2 +- 8 files changed, 204 insertions(+), 88 deletions(-) diff --git a/.gitignore b/.gitignore index 533bf9c..6c56811 100644 --- a/.gitignore +++ b/.gitignore @@ -42,6 +42,9 @@ src/lfs/lfs.img *.x86_64 *.hex +# Build Artifacts +build +firmware/builds + # Private files private - diff --git a/README.md b/README.md index 1e3304e..aa4beb6 100644 --- a/README.md +++ b/README.md @@ -5,11 +5,11 @@ # Konnected **Konnected** integrates wired alarm system sensors and sirens to SmartThings, Home Assistant, OpenHAB, or Hubitat using a NodeMCU based ESP8266 development board and (optional) relay. This project consists of a few components: - + 1. [NodeMCU](http://nodemcu.com/index_en.html) based firmware for an ESP8266 development board in `firmware` 1. Lua and HTML source code for the NodeMCU in `src`. All these files are built into a SPIFFS file system which runs on NodeMCU 1. [SmartThings](https://www.smartthings.com/) platform code in `smartapps` and `devicetypes` - + ![](http://docs.konnected.io/assets/images/konnected-alarm-panel.jpg) ## Skip this Installation! @@ -26,7 +26,7 @@ #### Note on Device Drivers Windows and Mac users will need to download drivers so your computer can talk to the ESP8266 chip over USB. Depending -on which board you have, there are different drivers: +on which board you have, there are different drivers: **[WeMos CH340 drivers](https://wiki.wemos.cc/downloads)** for boards that: * have the name _LoLin_ on the back or front @@ -41,23 +41,19 @@ on which board you have, there are different drivers: Konnected leverages the [NodeMCU](https://github.com/nodemcu/nodemcu-firmware) codebase and [Docker builder](https://hub.docker.com/r/marcelstoer/nodemcu-build/) to create a base nodeMCU firmware image and a filesystem containing the Konnected application. Building only requires a few steps. 1. Download and install [Docker](https://www.docker.com/products/docker-desktop) -1. Clone the Konnected and nodeMCU repos to the same level in your working directory +1. Clone the Konnected repo git clone https://github.com/konnected-io/konnected-security.git - git clone https://github.com/nodemcu/nodemcu-firmware.git - -1. Check out the 2.2.1 release of nodemcu - pushd nodemcu-firmware && git checkout 2.2.1-master_20190405 && popd - -1. Use the build-firmware script to kick off the build - providing a semantic version command line argument as shown below. The build script will automatically pull down the nodeMCU docker builder, build the base firmware, create an LFS image, and build a SPIFFS file system containing the entire Konnected application. +1. Use the build-firmware script to kick off the build - providing a semantic version command line argument as shown below. The build script will automatically pull down the correct nodeMCU image, and use this nodeMCU docker builder to create base firmware, an LFS image, and a SPIFFS file system containing the entire Konnected application. cd konnected-security ./scripts/build-firmware 2-2-99 -1. Once the build completes a folder will be created in `firmware/builds` named after the version specified in the previous step. This folder will contain two files also reflecting the version. +1. Once the build completes a folder will be created in `firmware/builds` named after the version specified in the previous step. This folder will contain three files also reflecting the version. 1. konnected-filesystem-0x100000-2-2-99.img 1. konnected-firmware-2-2-99.bin + 1. konnected-esp8266-2-2-99.bin *Note: Each time you build it will remove any prior build outputs corresponding to the same version.* *Note: Versions in this project should always be formatted `--`.* @@ -67,26 +63,26 @@ Flashing a build is simple with the [Konnected Flashing Tool](https://help.konne Mac and Linux users can also easily flash from the command line using [scripts/flash](scripts/flash). - 1. You must have Python installed with `pip` or `pip3`. - * **Mac users**: I recommend using [Homebrew](https://brew.sh/) and `brew install python` - + 1. You must have Python installed with `pip` or `pip3`. + * **Mac users**: I recommend using [Homebrew](https://brew.sh/) and `brew install python` + 1. Open up a terminal and install `esptool` packages: - + pip3 install esptool - + 1. Run the script in `scripts/flash` to flash the firmware and software to the device. You must pass in version and serial port args. The flash script will always attempt to flash a matching version in `firmware/builds` before falling back to `firmware/releases` ./scripts/flash 2-2-99 /dev/ttyS3 - + *Note: You may also need to make the script executable by running `chmod 755 scripts/flash`.* - + ### Donations We work hard on this project because we're passionate about making home automation accessible to everybody. Millions of homes in North America and worldwide are already wired with sensors and have the potential to become smart homes. We want to make this a reality. - + If you've used Konnected Security and it's improved your life and your home security, please consider [donating](http://docs.konnected.io/donate) to help us achieve that goal. diff --git a/firmware/nodemcu-firmware-overlay/app/include/user_config.h b/firmware/nodemcu-firmware-overlay/app/include/user_config.h index 1bde68d..e852092 100644 --- a/firmware/nodemcu-firmware-overlay/app/include/user_config.h +++ b/firmware/nodemcu-firmware-overlay/app/include/user_config.h @@ -22,6 +22,26 @@ //#define BIT_RATE_AUTOBAUD +// At start-up firmware details like: +// +// NodeMCU 3.0.1.0 +// branch: +// commit: +// release: +// release DTS: +// SSL: false +// build type: integer +// LFS: 0x0 +// modules: file,gpio,net,node,rtctime,sntp,tmr,uart,wifi +// build 2020-01-27 17:39 powered by Lua 5.1.4 on SDK 3.0.2(824dc80) +// +// will be printed to serial console. While it's mandatory for bug reports +// and good for development, it may be unwanted for non-interactive serial +// devices. + +//#define DISABLE_STARTUP_BANNER + + // Three separate build variants are now supported. The main difference is in the // processing of numeric data types. If LUA_NUMBER_INTEGRAL is defined, then // all numeric calculations are done in integer, with divide being an integer @@ -41,11 +61,12 @@ // The Lua Flash Store (LFS) allows you to store Lua code in Flash memory and // the Lua VMS will execute this code directly from flash without needing any -// RAM overhead. If you want to enable LFS then set the following define to -// the size of the store that you need. This can be any multiple of 4kB up to -// a maximum 256Kb. +// RAM overhead. You can now configure LFS directly in the System Partition +// Table insted of at compile time. However for backwards compatibility setting +// LUA_FLASH_STORE defines the default partition size if the NodeMCU partition +// tool is not used. -#define LUA_FLASH_STORE 0x10000 +//#define LUA_FLASH_STORE 0x10000 //#define LUA_FLASH_STORE 0x40000 max LFS size // By default Lua executes the file init.lua at start up. The following @@ -59,7 +80,7 @@ // NodeMCU supports two file systems: SPIFFS and FATFS, the first is available -// on all ESP8266 modules. The latter requires extra H/W so is less common. +// on all ESP8266 modules. The latter requires extra H/W so it is less common. // If you use SPIFFS then there are a number of options which impact the // RAM overhead and performance of the file system. @@ -71,11 +92,14 @@ // general, limiting the size of the FS only to what your application needs // gives the fastest start-up and imaging times. +// You can now configure SPIFFS size and position directly in the System +// Partition Table. However backwards compatibility SPIFFS_MAX_FILESYSTEM_SIZE +// can be set and this defines the default SPIFFS partition size if the NodeMCU +// partition tool is not used. The value (~0x0) means the maximum size remaining. + #define BUILD_SPIFFS -#define SPIFFS_FIXED_LOCATION 0x100000 -#define SPIFFS_MAX_FILESYSTEM_SIZE 128000 -//#define SPIFFS_SIZE_1M_BOUNDARY #define SPIFFS_CACHE 1 // Enable if you use you SPIFFS in R/W mode +#define SPIFFS_MAX_FILESYSTEM_SIZE 128000 #define SPIFFS_MAX_OPEN_FILES 4 // maximum number of open files for SPIFFS #define FS_OBJ_NAME_LEN 31 // maximum length of a filename @@ -84,22 +108,22 @@ // The HTTPS stack requires client SSL to be enabled. The SSL buffer size is // used only for espconn-layer secure connections, and is ignored otherwise. -// Some HTTPS applications require a larger buffer size to work. See +// Some HTTPS applications require a larger buffer size to work. See // https://github.com/nodemcu/nodemcu-firmware/issues/1457 for details. // The SHA2 and MD2 libraries are also optionally used by the crypto functions. // The SHA1 and MD5 function are implemented in the ROM BIOS. The MD2 and SHA2 // are by firmware code, and can be enabled if you need this functionality. #define CLIENT_SSL_ENABLE -//#define MD2_ENABLE #define SHA2_ENABLE #define SSL_BUFFER_SIZE 5376 +#define SSL_MAX_FRAGMENT_LENGTH_CODE MBEDTLS_SSL_MAX_FRAG_LEN_4096 // GPIO_INTERRUPT_ENABLE needs to be defined if your application uses the // gpio.trig() or related GPIO interrupt service routine code. Likewise the -// GPIO interrupt hook is requited for a few modules such as rotary. If you -// don't require this functionality, then commenting out these options out +// GPIO interrupt hook is required for a few modules such as rotary. If you +// don't require this functionality, then commenting out these options // will remove any associated runtime overhead. #define GPIO_INTERRUPT_ENABLE @@ -113,6 +137,9 @@ //#define TIMER_SUSPEND_ENABLE //#define PMSLEEP_ENABLE +// The net module optionally offers net info functionnality. Uncomment the following +// to enable the functionnality. +#define NET_PING_ENABLE // The WiFi module optionally offers an enhanced level of WiFi connection // management, using internal timer callbacks. Whilst many Lua developers @@ -130,10 +157,6 @@ // Enable creation on the wifi.eventmon.reason table #define WIFI_EVENT_MONITOR_DISCONNECT_REASON_LIST_ENABLE -// Enable use of the WiFi.monitor sub-module -//#define LUA_USE_MODULES_WIFI_MONITOR - - // Whilst the DNS client details can be configured through the WiFi API, // the defaults can be exposed temporarily during start-up. The following // WIFI_STA options allow you to configure this in the firmware. If the @@ -155,7 +178,23 @@ #define ENDUSER_SETUP_AP_SSID "konnected" -// The following sections are only relevent for those developers who are +// I2C software driver partially supports use of GPIO16 (D0) pin for SCL line. +// GPIO16 does not support open-drain mode and works in push-pull mode, +// so clock stretching will not be possible, because circuit in slave device that +// supposed to drive SCL low during stretching will not be capable to hold SCL low. +// Also I2C speed will be limited to no more than 400000 Hz (FAST mode). +// This define is does not have an effect on an old driver (see I2C_MASTER_OLD_VERSION). + +//#define I2C_MASTER_GPIO16_ENABLE + +// For compatibility reasons you can switch to old version of I2C software driver. +// It does not support changing speed, have only one bus id = 0, does not support GPIO16 +// and works only in Standard(slow) mode with clock speed around 50kHz. + +#define I2C_MASTER_OLD_VERSION + + +// The following sections are only relevant for those developers who are // developing modules or core Lua changes and configure how extra diagnostics // are enabled in the firmware. These should only be configured if you are // building your own custom firmware and have full access to the firmware @@ -189,6 +228,24 @@ // change this if you have tracked the implications through the Firmware sources // and understand the these. +#define NODEMCU_EAGLEROM_PARTITION 1 +#define NODEMCU_IROM0TEXT_PARTITION 2 +#define NODEMCU_LFS0_PARTITION 3 +#define NODEMCU_LFS1_PARTITION 4 +#define NODEMCU_TLSCERT_PARTITION 5 +#define NODEMCU_SPIFFS0_PARTITION 6 +#define NODEMCU_SPIFFS1_PARTITION 7 + +#ifndef LUA_FLASH_STORE +# define LUA_FLASH_STORE 0x0 +#endif + +#define SPIFFS_FIXED_LOCATION 0x100000 +#ifndef SPIFFS_MAX_FILESYSTEM_SIZE +# define SPIFFS_MAX_FILESYSTEM_SIZE 0xFFFFFFFF +#endif +//#define SPIFFS_SIZE_1M_BOUNDARY + #define LUA_TASK_PRIO USER_TASK_PRIO_0 #define LUA_PROCESS_LINE_SIG 2 #define LUA_OPTIMIZE_DEBUG 2 @@ -196,17 +253,17 @@ #define STRBUF_DEFAULT_INCREMENT 3 #define LUA_USE_BUILTIN_DEBUG_MINIMAL // for debug.getregistry() and debug.traceback() -#ifdef DEVELOPMENT_TOOLS -#if defined(LUA_CROSS_COMPILER) || !defined(DEVELOPMENT_USE_GDB) +#if defined(DEVELOPMENT_TOOLS) && defined(DEVELOPMENT_USE_GDB) +extern void LUA_DEBUG_HOOK (void); +#define lua_assert(x) ((x) ? (void) 0 : LUA_DEBUG_HOOK ()) +#elif defined(DEVELOPMENT_TOOLS) && defined(LUA_CROSS_COMPILER) extern void luaL_assertfail(const char *file, int line, const char *message); #define lua_assert(x) ((x) ? (void) 0 : luaL_assertfail(__FILE__, __LINE__, #x)) #else -extern void luaL_dbgbreak(void); -#define lua_assert(x) ((x) ? (void) 0 : luaL_dbgbreak()) -#endif +#define lua_assert(x) ((void) (x)) #endif -#if !defined(LUA_NUMBER_INTEGRAL) && defined (LUA_DWORD_ALIGNED_TVALUES) +#if !defined(LUA_NUMBER_INTEGRAL) && !defined (LUA_DWORD_ALIGNED_TVALUES) #define LUA_PACK_TVALUES #else #undef LUA_PACK_TVALUES @@ -217,17 +274,22 @@ extern void luaL_dbgbreak(void); #define COAP_DEBUG #endif /* DEVELOP_VERSION */ + +#if !defined(LUA_CROSS_COMPILER) && !defined(dbg_printf) +extern void dbg_printf(const char *fmt, ...); +#endif + #ifdef NODE_DEBUG #define NODE_DBG dbg_printf #else -#define NODE_DBG +#define NODE_DBG( ... ) #endif /* NODE_DEBUG */ #define NODE_ERROR #ifdef NODE_ERROR #define NODE_ERR dbg_printf #else -#define NODE_ERR +#define NODE_ERR( ... ) #endif /* NODE_ERROR */ // #define GPIO_SAFE_NO_INTR_ENABLE @@ -235,12 +297,7 @@ extern void luaL_dbgbreak(void); #define ICACHE_STORE_ATTR __attribute__((aligned(4))) #define ICACHE_STRING(x) ICACHE_STRING2(x) #define ICACHE_STRING2(x) #x -#define ICACHE_RAM_ATTR \ - __attribute__((section(".iram0.text." __FILE__ "." ICACHE_STRING(__LINE__)))) -#define ICACHE_FLASH_RESERVED_ATTR \ - __attribute__((section(".irom.reserved." __FILE__ "." ICACHE_STRING(__LINE__)),\ - used,unused,aligned(INTERNAL_FLASH_SECTOR_SIZE))) - +#define ICACHE_RAM_ATTR __attribute__((section(".iram0.text." __FILE__ "." ICACHE_STRING(__LINE__)))) #ifdef GPIO_SAFE_NO_INTR_ENABLE #define NO_INTR_CODE ICACHE_RAM_ATTR __attribute__ ((noinline)) #else diff --git a/firmware/nodemcu-firmware-overlay/app/include/user_mbedtls.h b/firmware/nodemcu-firmware-overlay/app/include/user_mbedtls.h index a269c5d..a61ba73 100644 --- a/firmware/nodemcu-firmware-overlay/app/include/user_mbedtls.h +++ b/firmware/nodemcu-firmware-overlay/app/include/user_mbedtls.h @@ -1,4 +1,5 @@ -#include "c_types.h" +#include +#include #include "user_config.h" @@ -78,9 +79,11 @@ #define MBEDTLS_ECP_DP_SECP256R1_ENABLED #define MBEDTLS_ECP_DP_SECP384R1_ENABLED #define MBEDTLS_ECP_DP_SECP521R1_ENABLED + #define MBEDTLS_ECP_DP_SECP192K1_ENABLED #define MBEDTLS_ECP_DP_SECP224K1_ENABLED #define MBEDTLS_ECP_DP_SECP256K1_ENABLED + #define MBEDTLS_ECP_DP_BP256R1_ENABLED #define MBEDTLS_ECP_DP_BP384R1_ENABLED #define MBEDTLS_ECP_DP_BP512R1_ENABLED @@ -190,11 +193,17 @@ #define MBEDTLS_CIPHER_C #define MBEDTLS_CMAC_C #define MBEDTLS_CTR_DRBG_C -#undef MBEDTLS_DEBUG_C + +#ifdef DEVELOP_VERSION +# define MBEDTLS_DEBUG_C +#else +# undef MBEDTLS_DEBUG_C +#endif + #define MBEDTLS_DES_C #define MBEDTLS_DHM_C #define MBEDTLS_ECDH_C -#undef MBEDTLS_ECDSA_C +#define MBEDTLS_ECDSA_C #undef MBEDTLS_ECJPAKE_C #define MBEDTLS_ECP_C #define MBEDTLS_ENTROPY_C @@ -266,8 +275,10 @@ //#define MBEDTLS_MEMORY_ALIGN_MULTIPLE 4 /**< Align on multiples of this value */ //#define MBEDTLS_PLATFORM_STD_MEM_HDR /**< Header to include if MBEDTLS_PLATFORM_NO_STD_FUNCTIONS is defined. Don't define if no header is needed. */ -#define MBEDTLS_PLATFORM_STD_CALLOC pvPortCalloc /**< Default allocator to use, can be undefined */ -#define MBEDTLS_PLATFORM_STD_FREE vPortFree /**< Default free to use, can be undefined */ +extern void *mbedtls_calloc_wrap(size_t n, size_t sz); +#define MBEDTLS_PLATFORM_STD_CALLOC mbedtls_calloc_wrap /**< Default allocator to use, can be undefined */ +extern void mbedtls_free_wrap(void *p); +#define MBEDTLS_PLATFORM_STD_FREE mbedtls_free_wrap /**< Default free to use, can be undefined */ //#define MBEDTLS_PLATFORM_STD_EXIT exit /**< Default exit to use, can be undefined */ //#define MBEDTLS_PLATFORM_STD_TIME time /**< Default time to use, can be undefined. MBEDTLS_HAVE_TIME must be enabled */ //#define MBEDTLS_PLATFORM_STD_FPRINTF fprintf /**< Default fprintf to use, can be undefined */ @@ -292,17 +303,11 @@ //#define MBEDTLS_SSL_CACHE_DEFAULT_TIMEOUT 86400 /**< 1 day */ //#define MBEDTLS_SSL_CACHE_DEFAULT_MAX_ENTRIES 50 /**< Maximum entries in cache */ -#if 0 -// dynamic buffer sizing with espconn_secure_set_size() -extern unsigned int max_content_len; -#define MBEDTLS_SSL_MAX_CONTENT_LEN max_content_len; -#else // the current mbedtls integration doesn't allow to set the buffer size dynamically: // MBEDTLS_SSL_MAX_FRAGMENT_LENGTH feature and dynamic sizing are mutually exclusive -// due to non-constant initializer element in app/mbedtls/library/ssl_tls.c:150 +// due to non-constant initializer element in app/mbedtls/library/ssl_tls.c:150 // the buffer size is hardcoded here and value is taken from SSL_BUFFER_SIZE (user_config.h) #define MBEDTLS_SSL_MAX_CONTENT_LEN SSL_BUFFER_SIZE /**< Maxium fragment length in bytes, determines the size of each of the two internal I/O buffers */ -#endif //#define MBEDTLS_SSL_DEFAULT_TICKET_LIFETIME 86400 /**< Lifetime of session tickets (if enabled) */ //#define MBEDTLS_PSK_MAX_LEN 32 /**< Max size of TLS pre-shared keys, in bytes (default 256 bits) */ diff --git a/firmware/nodemcu-firmware-overlay/app/include/user_modules.h b/firmware/nodemcu-firmware-overlay/app/include/user_modules.h index 3e9f757..8877624 100644 --- a/firmware/nodemcu-firmware-overlay/app/include/user_modules.h +++ b/firmware/nodemcu-firmware-overlay/app/include/user_modules.h @@ -21,8 +21,8 @@ //#define LUA_USE_MODULES_COLOR_UTILS //#define LUA_USE_MODULES_CRON #define LUA_USE_MODULES_CRYPTO +//#define LUA_USE_MODULES_DCC #define LUA_USE_MODULES_DHT -#define LUA_USE_MODULES_DS18B20 //#define LUA_USE_MODULES_ENCODER #define LUA_USE_MODULES_ENDUSER_SETUP // USE_DNS in dhcpserver.h needs to be enabled for this module to work. #define LUA_USE_MODULES_FILE @@ -43,7 +43,9 @@ //#define LUA_USE_MODULES_OW //#define LUA_USE_MODULES_PCM //#define LUA_USE_MODULES_PERF +//#define LUA_USE_MODULES_PIPE //#define LUA_USE_MODULES_PWM +//#define LUA_USE_MODULES_PWM2 //#define LUA_USE_MODULES_RC //#define LUA_USE_MODULES_RFSWITCH //#define LUA_USE_MODULES_ROTARY @@ -54,6 +56,7 @@ //#define LUA_USE_MODULES_SIGMA_DELTA #define LUA_USE_MODULES_SJSON #define LUA_USE_MODULES_SNTP +//#define LUA_USE_MODULES_SOFTUART //#define LUA_USE_MODULES_SOMFY //#define LUA_USE_MODULES_SPI //#define LUA_USE_MODULES_SQLITE3 diff --git a/firmware/nodemcu-firmware-overlay/app/include/user_version.h b/firmware/nodemcu-firmware-overlay/app/include/user_version.h index a33e207..7be96c4 100644 --- a/firmware/nodemcu-firmware-overlay/app/include/user_version.h +++ b/firmware/nodemcu-firmware-overlay/app/include/user_version.h @@ -2,6 +2,7 @@ #define __USER_VERSION_H__ #include "version.h" /* ESP firmware header */ +#include #define NODE_VERSION_MAJOR ESP_SDK_VERSION_MAJOR #define NODE_VERSION_MINOR ESP_SDK_VERSION_MINOR @@ -11,10 +12,12 @@ #define NODE_VERSION_STR(x) #x #define NODE_VERSION_XSTR(x) NODE_VERSION_STR(x) -#define NODE_VERSION "Konnected firmware 2.3.0\r\nNodeMCU " ESP_SDK_VERSION_STRING "." NODE_VERSION_XSTR(NODE_VERSION_INTERNAL) +# define NODE_VERSION "Konnected firmware 2.3.0\r\nNodeMCU " ESP_SDK_VERSION_STRING "." NODE_VERSION_XSTR(NODE_VERSION_INTERNAL) +// Leave the space after # in the line above. It busts replacement of NODE_VERSION in the docker build which is not needed anymore with this PR. +// Can be removed when the script is adapted #ifndef BUILD_DATE -#define BUILD_DATE "20190805" +#define BUILD_DATE BUILDINFO_BUILD_DATE #endif extern char SDK_VERSION[]; diff --git a/scripts/build-firmware b/scripts/build-firmware index 3e958d8..67aa66f 100755 --- a/scripts/build-firmware +++ b/scripts/build-firmware @@ -1,10 +1,11 @@ -#!/bin/bash +#!/usr/bin/env bash function usage() { echo echo "usage: $0 " echo - echo ' version should be formatted --' + echo ' -v version should be formatted --' + echo ' -c optional abs path to custom nodemcu-firmware repo' echo echo 'Execute from the root dir of the repo. Upon successful completion the release will be created in firmware/releases/' echo " ex: '$0 2-3-0'" @@ -14,28 +15,67 @@ if [ "$#" == 0 ]; then usage fi -VERSION="$1" -if ! [[ "${VERSION}" =~ ^[0-9]+-[0-9]+-[0-9]+ ]] ; then - echo "Version not formatted correctly..." - usage -fi - set -e -FIRMWARE_PATH="${PWD}/../nodemcu-firmware" FIRMWARE_OVERLAY_PATH="firmware/nodemcu-firmware-overlay" BUILD_PATH="firmware/builds/${VERSION}" IMAGE_NAME="konnected-firmware-${VERSION}" + LFS_BASE="${PWD}/src/lfs" +OUTPUT_BASE="${PWD}/build" +OUTPUT_APP="${OUTPUT_BASE}/app" +FW_BASE="${OUTPUT_BASE}/nodemcu-firmware" +FW_TAG="3.0-master_20200610" +VERSION="99-99-99" + +# process cmd args +while getopts "v:c:h" opt; do + case ${opt} in + h ) + usage + ;; + c ) + FW_BASE="$OPTARG" + FW_TAG='' + echo "using custom nodemcu-repo ${FW_BASE}" + ;; + v ) + VERSION="$OPTARG" + if ! [[ "${VERSION}" =~ ^[0-9]+-[0-9]+-[0-9]+ ]] ; then + echo "Version not formatted correctly..." + usage + fi + echo "tagging as version ${VERSION}" + ;; + \? ) + echo "Invalid Option: -$OPTARG" 1>&2 + exit 1 + ;; + esac +done + +rm -rf ${OUTPUT_BASE} +mkdir -p "${OUTPUT_BASE}"; +chown -R $USER "${OUTPUT_BASE}" + +# pull and update the FW repo if none is supplied +if [ ! -z "$FW_TAG" ] ; then + echo "Checking out nodemcu-firmware:${FW_TAG}" + git clone https://github.com/nodemcu/nodemcu-firmware.git \ + "${FW_BASE}" + cd "${FW_BASE}" + git checkout "${FW_TAG}" + git submodule update --recursive + cd - +fi # Copy firmware configuration from this repository to the nodemcu-firmware repo -cp "${FIRMWARE_OVERLAY_PATH}"/app/include/* "${FIRMWARE_PATH}/app/include/" +cp "${FIRMWARE_OVERLAY_PATH}"/app/include/* "${FW_BASE}/app/include/" -rm -f "${FIRMWARE_PATH}/local/fs/*" +rm -f "${FW_BASE}/local/fs/*" # Build NodeMCU firmware image -docker run -e "INTEGER_ONLY=1" \ - -e "IMAGE_NAME=${IMAGE_NAME}" \ - --rm -ti -v "${FIRMWARE_PATH}:/opt/nodemcu-firmware" marcelstoer/nodemcu-build build +docker run -e "IMAGE_NAME=${IMAGE_NAME}" \ + --rm -ti -v "${FW_BASE}:/opt/nodemcu-firmware" marcelstoer/nodemcu-build build # Build LFS image with application files LUA_FILES=$(find ${LFS_BASE} -iname "*.lua" | sed "s|${LFS_BASE}|/lfs|g") @@ -43,22 +83,31 @@ LUA_FILES=$(find ${LFS_BASE} -iname "*.lua" | sed "s|${LFS_BASE}|/lfs|g") echo Adding files to LFS: echo ${LUA_FILES} docker run --rm -ti \ - -v ${FIRMWARE_PATH}:/tools \ + -v ${FW_BASE}:/nodemcu-firmware \ + -v ${OUTPUT_APP}:/build \ -v ${LFS_BASE}:/lfs \ - marcelstoer/nodemcu-build /tools/luac.cross.int -m 0x10000 -f -o "/lfs/lfs.img" ${LUA_FILES} + marcelstoer/nodemcu-build /nodemcu-firmware/luac.cross.int -m 0x10000 -f -o "/build/lfs.img" ${LUA_FILES} echo -# Build SPIFFS image +# transfer source files not in the lfs to the build output (ignore the status) +cp -f ${LFS_BASE}/../* "${OUTPUT_APP}" 2>/dev/null || true; echo "Building SPIFFS..." + +# Build spiffsimg tool (manual for now because of https://github.com/nodemcu/nodemcu-firmware/issues/2943) +docker run --rm -ti \ + -v ${FW_BASE}:/nodemcu-firmware \ + marcelstoer/nodemcu-build make -C /nodemcu-firmware/tools/spiffsimg + +# Create SPIFFS image docker run \ - --rm -ti -v "${FIRMWARE_PATH}:/opt/nodemcu-firmware" \ + --rm -ti -v "${FW_BASE}:/opt/nodemcu-firmware" \ -v "${PWD}/scripts:/scripts" \ - -v "${PWD}/src:/opt/lua" marcelstoer/nodemcu-build bash /scripts/build-spiffs + -v "${OUTPUT_APP}:/opt/lua" marcelstoer/nodemcu-build bash /scripts/build-spiffs # Create or clear the output directory and transfer files mkdir -p "${BUILD_PATH}" rm -rf "${BUILD_PATH}/*" -cp "${FIRMWARE_PATH}/bin/nodemcu_integer_${IMAGE_NAME}.bin" "${BUILD_PATH}/${IMAGE_NAME}.bin" -cp "${FIRMWARE_PATH}/bin/konnected-filesystem-0x100000.img" "${BUILD_PATH}/konnected-filesystem-0x100000-${VERSION}.img" +cp "${FW_BASE}/bin/nodemcu_integer_${IMAGE_NAME}.bin" "${BUILD_PATH}/${IMAGE_NAME}.bin" +cp "${FW_BASE}/bin/konnected-filesystem-0x100000.img" "${BUILD_PATH}/konnected-filesystem-0x100000-${VERSION}.img" srec_cat -output "${BUILD_PATH}/konnected-esp8266-${VERSION}.bin" -binary "${BUILD_PATH}/${IMAGE_NAME}.bin" -binary -fill 0xff 0x0000 0x100000 "${BUILD_PATH}/konnected-filesystem-0x100000-${VERSION}.img" -binary -offset 0x100000 diff --git a/scripts/spiffs.lst b/scripts/spiffs.lst index 6a78aa4..c993a11 100644 --- a/scripts/spiffs.lst +++ b/scripts/spiffs.lst @@ -7,4 +7,4 @@ import /opt/lua/http_index.html.gz http_index.html.gz import /opt/lua/init.lua init.lua import /opt/lua/sensors.lua sensors.lua import /opt/lua/settings.lua settings.lua -import /opt/lua/lfs/lfs.img lfs.img \ No newline at end of file +import /opt/lua/lfs.img lfs.img \ No newline at end of file