Initial integration of 3.0 release and integrated builder

konnected-io · Sep 3, 2020 · 6fc1ef3 · 6fc1ef3
1 parent 9693e64
commit 6fc1ef3
Show file tree

Hide file tree

Showing 8 changed files with 204 additions and 88 deletions.
diff --git a/.gitignore b/.gitignore
@@ -42,6 +42,9 @@ src/lfs/lfs.img
 *.x86_64
 *.hex
 
+# Build Artifacts
+build
+firmware/builds
+
 # Private files
 private
-
diff --git a/README.md b/README.md
@@ -5,11 +5,11 @@
 # Konnected
 
 **Konnected** integrates wired alarm system sensors and sirens to SmartThings, Home Assistant, OpenHAB, or Hubitat using a NodeMCU based ESP8266 development board and (optional) relay. This project consists of a few components:
- 
+
  1. [NodeMCU](http://nodemcu.com/index_en.html) based firmware for an ESP8266 development board in `firmware`
  1. Lua and HTML source code for the NodeMCU in `src`. All these files are built into a SPIFFS file system which runs on NodeMCU
  1. [SmartThings](https://www.smartthings.com/) platform code in `smartapps` and `devicetypes`
- 
+
 ![](http://docs.konnected.io/assets/images/konnected-alarm-panel.jpg)
 
 ## Skip this Installation!
@@ -26,7 +26,7 @@
 #### Note on Device Drivers
 
 Windows and Mac users will need to download drivers so your computer can talk to the ESP8266 chip over USB. Depending
-on which board you have, there are different drivers: 
+on which board you have, there are different drivers:
 
 **[WeMos CH340 drivers](https://wiki.wemos.cc/downloads)** for boards that:
 * have the name _LoLin_ on the back or front
@@ -41,23 +41,19 @@ on which board you have, there are different drivers:
 Konnected leverages the [NodeMCU](https://github.com/nodemcu/nodemcu-firmware) codebase and [Docker builder](https://hub.docker.com/r/marcelstoer/nodemcu-build/) to create a base nodeMCU firmware image and a filesystem containing the Konnected application. Building only requires a few steps.
 
 1. Download and install [Docker](https://www.docker.com/products/docker-desktop)
-1. Clone the Konnected and nodeMCU repos to the same level in your working directory
+1. Clone the Konnected repo
 
         git clone https://github.com/konnected-io/konnected-security.git
-        git clone https://github.com/nodemcu/nodemcu-firmware.git
-
-1. Check out the 2.2.1 release of nodemcu
 
-        pushd nodemcu-firmware && git checkout 2.2.1-master_20190405 && popd
-
-1. Use the build-firmware script to kick off the build - providing a semantic version command line argument as shown below. The build script will automatically pull down the nodeMCU docker builder, build the base firmware, create an LFS image, and build a SPIFFS file system containing the entire Konnected application.
+1. Use the build-firmware script to kick off the build - providing a semantic version command line argument as shown below. The build script will automatically pull down the correct nodeMCU image, and use this nodeMCU docker builder to create base firmware, an LFS image, and a SPIFFS file system containing the entire Konnected application.
 
         cd konnected-security
         ./scripts/build-firmware 2-2-99
 
-1. Once the build completes a folder will be created in `firmware/builds` named after the version specified in the previous step. This folder will contain two files also reflecting the version.
+1. Once the build completes a folder will be created in `firmware/builds` named after the version specified in the previous step. This folder will contain three files also reflecting the version.
    1. konnected-filesystem-0x100000-2-2-99.img
    1. konnected-firmware-2-2-99.bin
+   1. konnected-esp8266-2-2-99.bin
 
 *Note: Each time you build it will remove any prior build outputs corresponding to the same version.*
 *Note: Versions in this project should always be formatted `<major>-<minor>-<patch>`.*
@@ -67,26 +63,26 @@ Flashing a build is simple with the [Konnected Flashing Tool](https://help.konne
 
 Mac and Linux users can also easily flash from the command line using [scripts/flash](scripts/flash).
 
- 1. You must have Python installed with `pip` or `pip3`. 
-    * **Mac users**: I recommend using [Homebrew](https://brew.sh/) and `brew install python`  
- 
+ 1. You must have Python installed with `pip` or `pip3`.
+    * **Mac users**: I recommend using [Homebrew](https://brew.sh/) and `brew install python`
+
  1. Open up a terminal and install `esptool` packages:
-     
+
         pip3 install esptool
-        
+
  1. Run the script in `scripts/flash` to flash the firmware and software to the device. You must pass in version and serial port args. The flash script will always attempt to flash a matching version in `firmware/builds` before falling back to `firmware/releases`
 
          ./scripts/flash 2-2-99 /dev/ttyS3
- 
+
  *Note: You may also need to make the script executable by running `chmod 755 scripts/flash`.*
- 
+
 
 ### Donations
 
 We work hard on this project because we're passionate about making home automation accessible to everybody. Millions of
  homes in North America and worldwide are already wired with sensors and have the potential to become smart homes. We
  want to make this a reality.
- 
+
 If you've used Konnected Security and it's improved your life and your home security, please consider [donating](http://docs.konnected.io/donate) to help us
 achieve that goal.
 

diff --git a/firmware/nodemcu-firmware-overlay/app/include/user_config.h b/firmware/nodemcu-firmware-overlay/app/include/user_config.h
@@ -22,6 +22,26 @@
 //#define BIT_RATE_AUTOBAUD
 
 
+// At start-up firmware details like:
+//
+// NodeMCU 3.0.1.0
+//         branch:
+//         commit:
+//         release:
+//         release DTS:
+//         SSL: false
+//         build type: integer
+//         LFS: 0x0
+//         modules: file,gpio,net,node,rtctime,sntp,tmr,uart,wifi
+//  build 2020-01-27 17:39 powered by Lua 5.1.4 on SDK 3.0.2(824dc80)
+//
+// will be printed to serial console.  While it's mandatory for bug reports
+// and good for development, it may be unwanted for non-interactive serial
+// devices.
+
+//#define DISABLE_STARTUP_BANNER
+
+
 // Three separate build variants are now supported. The main difference is in the
 // processing of numeric data types.  If LUA_NUMBER_INTEGRAL is defined, then
 // all numeric calculations are done in integer, with divide being an integer
@@ -41,11 +61,12 @@
 
 // The Lua Flash Store (LFS) allows you to store Lua code in Flash memory and
 // the Lua VMS will execute this code directly from flash without needing any
-// RAM overhead.  If you want to enable LFS then set the following define to
-// the size of the store that you need.  This can be any multiple of 4kB up to
-// a maximum 256Kb.
+// RAM overhead.  You can now configure LFS directly in the System Partition
+// Table insted of at compile time. However for backwards compatibility setting
+// LUA_FLASH_STORE defines the default partition size if the NodeMCU partition
+// tool is not used.
 
-#define LUA_FLASH_STORE 0x10000
+//#define LUA_FLASH_STORE                   0x10000
 //#define LUA_FLASH_STORE 0x40000 max LFS size
 
 // By default Lua executes the file init.lua at start up.  The following
@@ -59,7 +80,7 @@
 
 
 // NodeMCU supports two file systems: SPIFFS and FATFS, the first is available
-// on all ESP8266 modules.  The latter requires extra H/W so is less common.
+// on all ESP8266 modules.  The latter requires extra H/W so it is less common.
 // If you use SPIFFS then there are a number of options which impact the
 // RAM overhead and performance of the file system.
 
@@ -71,11 +92,14 @@
 // general, limiting the size of the FS only to what your application needs
 // gives the fastest start-up and imaging times.
 
+// You can now configure SPIFFS size and position directly in the System
+// Partition Table.  However backwards compatibility SPIFFS_MAX_FILESYSTEM_SIZE
+// can be set and this defines the default SPIFFS partition size if the NodeMCU
+// partition tool is not used. The value (~0x0) means the maximum size remaining.
+
 #define BUILD_SPIFFS
-#define SPIFFS_FIXED_LOCATION        0x100000
-#define SPIFFS_MAX_FILESYSTEM_SIZE    128000
-//#define SPIFFS_SIZE_1M_BOUNDARY
 #define SPIFFS_CACHE 1          // Enable if you use you SPIFFS in R/W mode
+#define SPIFFS_MAX_FILESYSTEM_SIZE    128000
 #define SPIFFS_MAX_OPEN_FILES 4 // maximum number of open files for SPIFFS
 #define FS_OBJ_NAME_LEN 31      // maximum length of a filename
 
@@ -84,22 +108,22 @@
 
 // The HTTPS stack requires client SSL to be enabled.  The SSL buffer size is
 // used only for espconn-layer secure connections, and is ignored otherwise.
-// Some HTTPS  applications require a larger buffer size to work.  See
+// Some HTTPS applications require a larger buffer size to work.  See
 // https://github.com/nodemcu/nodemcu-firmware/issues/1457 for details.
 // The SHA2 and MD2 libraries are also optionally used by the crypto functions.
 // The SHA1 and MD5 function are implemented in the ROM BIOS. The MD2 and SHA2
 // are by firmware code, and can be enabled if you need this functionality.
 
 #define CLIENT_SSL_ENABLE
-//#define MD2_ENABLE
 #define SHA2_ENABLE
 #define SSL_BUFFER_SIZE 5376
+#define SSL_MAX_FRAGMENT_LENGTH_CODE	MBEDTLS_SSL_MAX_FRAG_LEN_4096
 
 
 // GPIO_INTERRUPT_ENABLE needs to be defined if your application uses the
 // gpio.trig() or related GPIO interrupt service routine code.  Likewise the
-// GPIO interrupt hook is requited for a few modules such as rotary.  If you
-// don't require this functionality, then commenting out these options out
+// GPIO interrupt hook is required for a few modules such as rotary.  If you
+// don't require this functionality, then commenting out these options
 // will remove any associated runtime overhead.
 
 #define GPIO_INTERRUPT_ENABLE
@@ -113,6 +137,9 @@
 //#define TIMER_SUSPEND_ENABLE
 //#define PMSLEEP_ENABLE
 
+// The net module optionally offers net info functionnality. Uncomment the following
+// to enable the functionnality.
+#define NET_PING_ENABLE
 
 // The WiFi module optionally offers an enhanced level of WiFi connection
 // management, using internal timer callbacks.  Whilst many Lua developers
@@ -130,10 +157,6 @@
 //  Enable creation on the wifi.eventmon.reason table
 #define WIFI_EVENT_MONITOR_DISCONNECT_REASON_LIST_ENABLE
 
-//  Enable use of the WiFi.monitor sub-module
-//#define LUA_USE_MODULES_WIFI_MONITOR
-
-
 // Whilst the DNS client details can be configured through the WiFi API,
 // the defaults can be exposed temporarily during start-up.  The following
 // WIFI_STA options allow you to configure this in the firmware.  If the
@@ -155,7 +178,23 @@
 #define ENDUSER_SETUP_AP_SSID "konnected"
 
 
-// The following sections are only relevent for those developers who are
+// I2C software driver partially supports use of GPIO16 (D0) pin for SCL line.
+// GPIO16 does not support open-drain mode and works in push-pull mode,
+// so clock stretching will not be possible, because circuit in slave device that
+// supposed to drive SCL low during stretching will not be capable to hold SCL low.
+// Also I2C speed will be limited to no more than 400000 Hz (FAST mode).
+// This define is does not have an effect on an old driver (see I2C_MASTER_OLD_VERSION).
+
+//#define I2C_MASTER_GPIO16_ENABLE
+
+// For compatibility reasons you can switch to old version of I2C software driver.
+// It does not support changing speed, have only one bus id = 0, does not support GPIO16
+// and works only in Standard(slow) mode with clock speed around 50kHz.
+
+#define I2C_MASTER_OLD_VERSION
+
+
+// The following sections are only relevant for those developers who are
 // developing modules or core Lua changes and configure how extra diagnostics
 // are enabled in the firmware. These should only be configured if you are
 // building your own custom firmware and have full access to the firmware
@@ -189,24 +228,42 @@
 // change this if you have tracked the implications through the Firmware sources
 // and understand the these.
 
+#define NODEMCU_EAGLEROM_PARTITION        1
+#define NODEMCU_IROM0TEXT_PARTITION       2
+#define NODEMCU_LFS0_PARTITION            3
+#define NODEMCU_LFS1_PARTITION            4
+#define NODEMCU_TLSCERT_PARTITION         5
+#define NODEMCU_SPIFFS0_PARTITION         6
+#define NODEMCU_SPIFFS1_PARTITION         7
+
+#ifndef LUA_FLASH_STORE
+#  define LUA_FLASH_STORE                 0x0
+#endif
+
+#define SPIFFS_FIXED_LOCATION             0x100000
+#ifndef SPIFFS_MAX_FILESYSTEM_SIZE
+#  define SPIFFS_MAX_FILESYSTEM_SIZE      0xFFFFFFFF
+#endif
+//#define SPIFFS_SIZE_1M_BOUNDARY
+
 #define LUA_TASK_PRIO             USER_TASK_PRIO_0
 #define LUA_PROCESS_LINE_SIG      2
 #define LUA_OPTIMIZE_DEBUG        2
 #define READLINE_INTERVAL        80
 #define STRBUF_DEFAULT_INCREMENT  3
 #define LUA_USE_BUILTIN_DEBUG_MINIMAL // for debug.getregistry() and debug.traceback()
 
-#ifdef DEVELOPMENT_TOOLS
-#if defined(LUA_CROSS_COMPILER) || !defined(DEVELOPMENT_USE_GDB)
+#if defined(DEVELOPMENT_TOOLS) && defined(DEVELOPMENT_USE_GDB)
+extern void LUA_DEBUG_HOOK (void);
+#define lua_assert(x)    ((x) ? (void) 0 : LUA_DEBUG_HOOK ())
+#elif defined(DEVELOPMENT_TOOLS) && defined(LUA_CROSS_COMPILER)
 extern void luaL_assertfail(const char *file, int line, const char *message);
 #define lua_assert(x)    ((x) ? (void) 0 : luaL_assertfail(__FILE__, __LINE__, #x))
 #else
-extern void luaL_dbgbreak(void);
-#define lua_assert(x)    ((x) ? (void) 0 : luaL_dbgbreak())
-#endif
+#define lua_assert(x)    ((void) (x))
 #endif
 
-#if !defined(LUA_NUMBER_INTEGRAL) && defined (LUA_DWORD_ALIGNED_TVALUES)
+#if !defined(LUA_NUMBER_INTEGRAL) && !defined (LUA_DWORD_ALIGNED_TVALUES)
   #define LUA_PACK_TVALUES
 #else
   #undef LUA_PACK_TVALUES
@@ -217,30 +274,30 @@ extern void luaL_dbgbreak(void);
 #define COAP_DEBUG
 #endif /* DEVELOP_VERSION */
 
+
+#if !defined(LUA_CROSS_COMPILER) && !defined(dbg_printf)
+extern void dbg_printf(const char *fmt, ...);
+#endif
+
 #ifdef NODE_DEBUG
 #define NODE_DBG dbg_printf
 #else
-#define NODE_DBG
+#define NODE_DBG( ... )
 #endif	/* NODE_DEBUG */
 
 #define NODE_ERROR
 #ifdef NODE_ERROR
 #define NODE_ERR dbg_printf
 #else
-#define NODE_ERR
+#define NODE_ERR( ... )
 #endif	/* NODE_ERROR */
 
 // #define GPIO_SAFE_NO_INTR_ENABLE
 #define ICACHE_STORE_TYPEDEF_ATTR __attribute__((aligned(4),packed))
 #define ICACHE_STORE_ATTR __attribute__((aligned(4)))
 #define ICACHE_STRING(x) ICACHE_STRING2(x)
 #define ICACHE_STRING2(x) #x
-#define ICACHE_RAM_ATTR \
-  __attribute__((section(".iram0.text." __FILE__ "." ICACHE_STRING(__LINE__))))
-#define ICACHE_FLASH_RESERVED_ATTR \
-  __attribute__((section(".irom.reserved." __FILE__ "." ICACHE_STRING(__LINE__)),\
-                 used,unused,aligned(INTERNAL_FLASH_SECTOR_SIZE)))
-
+#define ICACHE_RAM_ATTR __attribute__((section(".iram0.text." __FILE__ "." ICACHE_STRING(__LINE__))))
 #ifdef  GPIO_SAFE_NO_INTR_ENABLE
 #define NO_INTR_CODE ICACHE_RAM_ATTR __attribute__ ((noinline))
 #else

diff --git a/firmware/nodemcu-firmware-overlay/app/include/user_mbedtls.h b/firmware/nodemcu-firmware-overlay/app/include/user_mbedtls.h
@@ -1,4 +1,5 @@
-#include "c_types.h"
+#include <stdint.h>
+#include <stddef.h>
 
 #include "user_config.h"
 
@@ -78,9 +79,11 @@
 #define MBEDTLS_ECP_DP_SECP256R1_ENABLED
 #define MBEDTLS_ECP_DP_SECP384R1_ENABLED
 #define MBEDTLS_ECP_DP_SECP521R1_ENABLED
+
 #define MBEDTLS_ECP_DP_SECP192K1_ENABLED
 #define MBEDTLS_ECP_DP_SECP224K1_ENABLED
 #define MBEDTLS_ECP_DP_SECP256K1_ENABLED
+
 #define MBEDTLS_ECP_DP_BP256R1_ENABLED
 #define MBEDTLS_ECP_DP_BP384R1_ENABLED
 #define MBEDTLS_ECP_DP_BP512R1_ENABLED
@@ -190,11 +193,17 @@
 #define MBEDTLS_CIPHER_C
 #define MBEDTLS_CMAC_C
 #define MBEDTLS_CTR_DRBG_C
-#undef MBEDTLS_DEBUG_C
+
+#ifdef DEVELOP_VERSION
+# define MBEDTLS_DEBUG_C
+#else
+# undef MBEDTLS_DEBUG_C
+#endif
+
 #define MBEDTLS_DES_C
 #define MBEDTLS_DHM_C
 #define MBEDTLS_ECDH_C
-#undef MBEDTLS_ECDSA_C
+#define MBEDTLS_ECDSA_C
 #undef MBEDTLS_ECJPAKE_C
 #define MBEDTLS_ECP_C
 #define MBEDTLS_ENTROPY_C
@@ -266,8 +275,10 @@
 //#define MBEDTLS_MEMORY_ALIGN_MULTIPLE      4 /**< Align on multiples of this value */
 
 //#define MBEDTLS_PLATFORM_STD_MEM_HDR   <stdlib.h> /**< Header to include if MBEDTLS_PLATFORM_NO_STD_FUNCTIONS is defined. Don't define if no header is needed. */
-#define MBEDTLS_PLATFORM_STD_CALLOC        pvPortCalloc /**< Default allocator to use, can be undefined */
-#define MBEDTLS_PLATFORM_STD_FREE            vPortFree /**< Default free to use, can be undefined */
+extern void *mbedtls_calloc_wrap(size_t n, size_t sz);
+#define MBEDTLS_PLATFORM_STD_CALLOC        mbedtls_calloc_wrap /**< Default allocator to use, can be undefined */
+extern void mbedtls_free_wrap(void *p);
+#define MBEDTLS_PLATFORM_STD_FREE            mbedtls_free_wrap /**< Default free to use, can be undefined */
 //#define MBEDTLS_PLATFORM_STD_EXIT            exit /**< Default exit to use, can be undefined */
 //#define MBEDTLS_PLATFORM_STD_TIME            time /**< Default time to use, can be undefined. MBEDTLS_HAVE_TIME must be enabled */
 //#define MBEDTLS_PLATFORM_STD_FPRINTF      fprintf /**< Default fprintf to use, can be undefined */
@@ -292,17 +303,11 @@
 //#define MBEDTLS_SSL_CACHE_DEFAULT_TIMEOUT       86400 /**< 1 day  */
 //#define MBEDTLS_SSL_CACHE_DEFAULT_MAX_ENTRIES      50 /**< Maximum entries in cache */
 
-#if 0
-// dynamic buffer sizing with espconn_secure_set_size()
-extern unsigned int max_content_len;
-#define MBEDTLS_SSL_MAX_CONTENT_LEN             max_content_len;
-#else
 // the current mbedtls integration doesn't allow to set the buffer size dynamically:
 //   MBEDTLS_SSL_MAX_FRAGMENT_LENGTH feature and dynamic sizing are mutually exclusive
-//   due to non-constant initializer element in app/mbedtls/library/ssl_tls.c:150 
+//   due to non-constant initializer element in app/mbedtls/library/ssl_tls.c:150
 // the buffer size is hardcoded here and value is taken from SSL_BUFFER_SIZE (user_config.h)
 #define MBEDTLS_SSL_MAX_CONTENT_LEN             SSL_BUFFER_SIZE /**< Maxium fragment length in bytes, determines the size of each of the two internal I/O buffers */
-#endif
 
 //#define MBEDTLS_SSL_DEFAULT_TICKET_LIFETIME     86400 /**< Lifetime of session tickets (if enabled) */
 //#define MBEDTLS_PSK_MAX_LEN               32 /**< Max size of TLS pre-shared keys, in bytes (default 256 bits) */