Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement mesh upgrade package #1109

Draft
wants to merge 122 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 109 commits
Commits
Show all changes
122 commits
Select commit Hold shift + click to select a range
a5f59c2
Merge branch 'master' of github.com:javierbrk/lime-packages into babe…
javierbrk Oct 6, 2023
288f202
hook test
javierajorge Oct 17, 2023
1f96c5d
added hook execution permision
javierajorge Oct 17, 2023
75fb96a
Create lime-mesh-upgrade package
selankon Oct 18, 2023
863c076
removed_files_from_other_commit
javierbrk Oct 18, 2023
5757011
Merge remote-tracking branch 'origin/meshwideupdates' into feature/me…
javierbrk Oct 18, 2023
f259854
nuevo modelo de datos
javierbrk Oct 18, 2023
e9aadbd
wip ...
javierbrk Oct 20, 2023
49d4e23
add uci interface
javierbrk Oct 23, 2023
0d82acf
removed dependencies
javierbrk Oct 23, 2023
7895145
testing uci files
javierbrk Oct 23, 2023
f98ac08
fixed underscore typo
javierbrk Oct 23, 2023
7605733
test
javierbrk Oct 23, 2023
62b6f12
Implement set_up_firmware_repository
selankon Oct 25, 2023
d7d3d30
mesh upgrade lib: improves tests, creates states as constants
javierbrk Oct 31, 2023
2504b60
save configurations to uci and make tests pass
javierajorge Oct 31, 2023
076fdd5
add inform_download_location and trigger_sheredstate_upgrade
javierajorge Nov 13, 2023
b7dd972
incorporates eupgrade and improves testing
javierbrk Nov 13, 2023
ac7c595
enables testing with real repo information
javierbrk Nov 27, 2023
e373d28
ads safeupgrade
javierajorge Nov 28, 2023
bd9e791
Merge branch 'feature/mesh-upgrade' of github.com:javierbrk/lime-pack…
javierajorge Nov 28, 2023
e86639d
Implement create local repository json data
selankon Nov 29, 2023
12c1ad2
Implement share firmware images
selankon Nov 29, 2023
4bbba91
Merge branch 'libremesh:master' into feature/mesh-upgrade
javierbrk Nov 30, 2023
d285f99
adds real unit test and fix start_node_download
javierajorge Nov 30, 2023
b905783
Add test
selankon Nov 30, 2023
308f67b
Merge branch 'feature/mesh-upgrade' of github.com:selankon/lime-packa…
javierajorge Nov 30, 2023
f061115
Merge branch 'feature/mesh-upgrade' of github.com:selankon/lime-packa…
javierajorge Nov 30, 2023
85f7f95
merge commits and use alternative eupgrade infra
javierajorge Nov 30, 2023
306cb7b
Merge branch 'feature/mesh-upgrade' of github.com:javierbrk/lime-pack…
javierajorge Nov 30, 2023
d220532
Ensure local latest json sharing
selankon Dec 1, 2023
d2f00aa
Implement become_master_node
selankon Dec 1, 2023
c58fdad
Fix eup to eupgrade
selankon Dec 1, 2023
e04af02
updates the downloaded list of files
javierajorge Dec 1, 2023
532ad42
fixes paths and urls to work with eupgrade
javierajorge Dec 5, 2023
e0458b7
helps network module return the write ip format
javierajorge Dec 5, 2023
e1fafc8
discards signature check on meshupgrade
javierajorge Jan 4, 2024
46fe6f2
Create shared state acl.d
selankon Jan 9, 2024
84ae8eb
removes transaction concept
javierajorge Jan 11, 2024
ab08fee
Merge branch 'feature/mesh-upgrade' of github.com:selankon/lime-packa…
javierajorge Jan 11, 2024
cfb0306
fixes ready for upgrade simulation for x86
javierajorge Jan 16, 2024
a6467f7
Refactor set_up_local_repository
selankon Jan 22, 2024
5f2938c
Eliminate divisive language
selankon Jan 22, 2024
95e3e77
Split become_main_node
selankon Jan 23, 2024
ba73b05
Fix main node status function
selankon Jan 23, 2024
4aa81e8
Fix get_main_node_status and improve redeability
selankon Jan 23, 2024
ce16017
improves state changes
javierajorge Jan 23, 2024
ccbab49
Merge branch 'feature/mesh-upgrade' of github.com:selankon/lime-packa…
javierajorge Jan 23, 2024
3800135
fixes start upgrade script and adds some gaurds
javierajorge Jan 23, 2024
eae0613
Try to fix mesh upgrade rpcd
selankon Jan 24, 2024
72fdde1
lime-mesh-upgrade: enable ubus list and methods call
javierajorge Jan 24, 2024
775633a
lime_mesh_upgrade: fixes test failure
javierajorge Jan 26, 2024
e3ac277
mesh-upgrade: delete unnecessary logs
selankon Jan 29, 2024
724abf8
mesh-upgrade: add test
selankon Jan 29, 2024
08402bd
lime-mesh-upgrade: fiexes json file name
javierbrk Jan 29, 2024
3b67b7f
Merge branch 'feature/mesh-upgrade' from kon
javierbrk Jan 29, 2024
426db52
lime-mesh-upgrade: fixes "invalid argument" and file permissions
javierbrk Jan 30, 2024
088c26f
lime-mesh-upgrade: fixes tests bug related to file permission
javierajorge Jan 30, 2024
2f942aa
lime-mesh-upgrade: improves testing and fixes small issues
javierajorge Jan 30, 2024
3d8205f
lime-mesh-upgrade: removes unused requirements
javierajorge Jan 30, 2024
084b24c
mesh-upgrade: refactor to implement get_mesh_upgrade_node_status
selankon Feb 1, 2024
f920940
lime_mesh_upgrade: adds fwpath
javierbrk Feb 1, 2024
39c25da
Merges get node status
javierbrk Feb 1, 2024
16c9567
Merge branch 'feature/mesh-upgrade' of github.com:javierbrk/lime-pack…
javierajorge Feb 8, 2024
8394178
Merge branch 'libremesh:master' into feature/mesh-upgrade
javierbrk Feb 22, 2024
b27caa5
lime-mesh-upgrade: add eupgrade status and more rpc
javierbrk Feb 1, 2024
4be266e
mesh-upgrade: refactor for name consistency
selankon Feb 2, 2024
e66d071
mesh-upgrade: return eupgrade status on check_eupgrade_download_failed
selankon Feb 2, 2024
d7ea20f
mesh-upgrade: prevent mesh_upgrade.state nill state
selankon Feb 2, 2024
49c7449
mesh-upgrade: deprecate STARTING state
selankon Feb 2, 2024
e9e5dd9
mesh-upgrade: improve code
selankon Feb 2, 2024
d984674
mesh-upgrade: upgrade mesh_upgrade main node initial state
selankon Feb 2, 2024
889b46d
mesh-upgrade: fix wrong uci cursor
selankon Feb 2, 2024
0a9e851
mesh-upgrade: prevent start mesh wide transaction
selankon Feb 2, 2024
183a131
mesh-upgrade: fixes tests and incorporates config persistence
javierbrk Feb 5, 2024
d65a96a
mesh-upgrade: prserve mesh config and set su_timeouts
javierbrk Feb 6, 2024
85b4bbb
mesh-upgrade: adds retry on error and abort
javierbrk Feb 22, 2024
6e3adea
Merge branch 'feature/mesh-upgrade' of github.com:javierbrk/lime-pack…
javierajorge Feb 27, 2024
d85b950
mesh_upgrade: fixes typos and add tests for confirm
javierajorge Mar 7, 2024
f45d888
Merge branch 'libremesh:master' into feature/mesh-upgrade
javierbrk Mar 21, 2024
50242d2
lime-mesh-upgrade: add ipv4 in node status
javierbrk Mar 1, 2024
5e60e13
lime-mesh-upgrade: adds rpc parameters to start_s_u
javierbrk Mar 12, 2024
88c500f
mesh-upgrade: error if filenot found
javierbrk Mar 21, 2024
a306df7
added test case, removed luci dependency
javierajorge Sep 28, 2023
d99480d
added ubus dependency
javierajorge Sep 28, 2023
a00e9e5
use random-numgen
javierajorge Sep 28, 2023
88e61e8
shared-state-async: add babel links information support
javierbrk Mar 1, 2024
0bedc8a
mesh-upgrade: adds makefile dependencies
javierbrk Mar 25, 2024
d644536
mesh-upgrade: abort
javierbrk Mar 26, 2024
2e88248
mesh-upgrade: reports time and abort scheduled upgrade
javierbrk Mar 26, 2024
79e6411
mesh-upgrade: fixes became main node error
javierbrk Mar 27, 2024
9734b3b
mesh-upgrade: check image is valid and add timing
javierbrk Apr 3, 2024
14323cb
mesh-upgrade: fix ipv4 address, add support for sharedstate async
javierbrk Apr 15, 2024
e9f334f
mesh-upgrade: fix makefile dependencies
javierbrk Apr 15, 2024
fad984b
mesh-upgrade: fixes ip test failure
javierbrk Apr 15, 2024
7c62efd
mesh-upgrade: fixes makefile error
javierbrk Apr 15, 2024
0db9965
mesh-upgrade:deleted unnecesary file
javierbrk Apr 15, 2024
ef16fbb
Merge branch 'libremesh:master' into feature/mesh-upgrade
javierbrk Apr 15, 2024
c39be29
mesh_upgrade: implements delayed insert
javierajorge Apr 16, 2024
98ba399
mesh-upgrade: fixes deadlock in publish
javierbrk Apr 19, 2024
58a3fea
mesh-upgrade: cleans log comments
javierbrk Apr 19, 2024
d2ae110
mesh-upgrade: adds readme message
javierbrk Apr 19, 2024
d14626f
mesh-upgrade: fix ability to start a new upgrade after confirmation
javierajorge Apr 19, 2024
9dcbed9
Merge branch 'libremesh:master' into feature/mesh-upgrade
javierbrk Apr 24, 2024
7210e05
shared-state: remove cached possibly corrupted files from eupgrade
javierbrk Apr 22, 2024
9b761d8
mesh-upgrade: create link of publish_mesh_upgrade
selankon May 6, 2024
2828bbd
shared-state-async: avoid removing eupgrade staff
javierajorge May 7, 2024
1f5cc43
shared-state-asymc: add simple general check
javierajorge May 10, 2024
ba6e26c
fixes verify_network_consistency apend bug
javierajorge May 16, 2024
d4332af
Merge branch 'libremesh:master' into feature/mesh-upgrade
javierbrk Aug 27, 2024
8a9efc8
Merge branch 'libremesh:master' into feature/mesh-upgrade
javierbrk Aug 28, 2024
7d366fc
f_mesh_up: move publisher to new location
javierbrk Sep 9, 2024
a03cc22
f_mesh_up: rename uci default file
javierbrk Sep 9, 2024
32b40a1
f_mu: fixes #1125 by checking that safe upgrade is ready
javierbrk Sep 13, 2024
937c645
f_mu: fixes publisher relocation
javierbrk Sep 13, 2024
26fc404
f_mu: fix tests after publisher relocation
javierbrk Sep 13, 2024
2f9794a
f_mu: fix ubus print error in get status
javierbrk Sep 13, 2024
c338195
f_mu: removes wget dependency
javierbrk Sep 23, 2024
6e0e715
Merge branch 'libremesh:master' into feature/mesh-upgrade
javierbrk Nov 25, 2024
7fd8d38
f_mu: comments cleanup
javierbrk Nov 25, 2024
160a2a4
f_mu: fix spelling errors
javierbrk Nov 25, 2024
f54b79a
f_mu: fix makefile description
javierbrk Nov 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 27 additions & 7 deletions packages/eupgrade/files/usr/lib/lua/eupgrade.lua
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ local utils = require "lime.utils"
local json = require "luci.jsonc"
local libuci = require "uci"
local fs = require("nixio.fs")
local config = require "lime.config"

local eup = {}

Expand Down Expand Up @@ -32,8 +33,30 @@ function eup.is_enabled()
return uci:get('eupgrade', 'main', 'enabled') == '1'
end

function eup.is_meshupgrade_enabled()
if uci:get('eupgrade', 'main', 'custom_api_url') == nil then
return false
else
return true
end
end

function eup.get_upgrade_api_url()
return uci:get('eupgrade', 'main', 'api_url') or ''
return uci:get('eupgrade', 'main', 'custom_api_url') or uci:get('eupgrade', 'main', 'api_url') or ''
end

function eup.set_custom_api_url(url)
status = uci:set('eupgrade', 'main', 'custom_api_url',url)
uci:save('eupgrade')
uci:commit('eupgrade')
return status
end

function eup.remove_custom_api_url()
status = uci:delete('eupgrade', 'main', 'custom_api_url')
uci:save('eupgrade')
uci:commit('eupgrade')
return status
end

function eup._check_signature(file_path, signature_path)
Expand Down Expand Up @@ -61,7 +84,7 @@ end
function eup.is_new_version_available(cached_only)
--! if 'latest' files are present is because there is a new version
if utils.file_exists(eup.FIRMWARE_LATEST_JSON) and utils.file_exists(eup.FIRMWARE_LATEST_JSON_SIGNATURE) then
if eup._check_signature(eup.FIRMWARE_LATEST_JSON, eup.FIRMWARE_LATEST_JSON_SIGNATURE) then
if eup._check_signature(eup.FIRMWARE_LATEST_JSON, eup.FIRMWARE_LATEST_JSON_SIGNATURE) or eup.is_meshupgrade_enabled() then
return json.parse(utils.read_file(eup.FIRMWARE_LATEST_JSON))
end
end
Expand All @@ -84,15 +107,12 @@ function eup.is_new_version_available(cached_only)
local sig_url = url .. ".sig"
if not utils.http_client_get(sig_url, 10, eup.FIRMWARE_LATEST_JSON_SIGNATURE) then
message = "Can't download signature " .. sig_url
utils.log(message)
end

if eup._check_signature(eup.FIRMWARE_LATEST_JSON, eup.FIRMWARE_LATEST_JSON_SIGNATURE) then
utils.log("Good signature of firmware_latest.json")
-- this will skip json signature verification when altenative url is set
if eup._check_signature(eup.FIRMWARE_LATEST_JSON, eup.FIRMWARE_LATEST_JSON_SIGNATURE) or eup.is_meshupgrade_enabled() then
return latest_data
else
message = "Bad signature of firmware_latest.json"
utils.log(message)
end
end
end
Expand Down
17 changes: 16 additions & 1 deletion packages/eupgrade/tests/test_eupgrade.lua
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ describe('eupgrade tests #eupgrade', function()
assert.is.equal('LibreMesh 20.10', eup.is_new_version_available()['version'])
end)

it('test is_new_version_available latest version is not the same as current version', function()
it('test is_new_version_available but signature fails', function()
stub(eup, '_get_board_name', function () return 'test-board' end)
stub(eup, '_get_current_fw_version', function () return 'LibreMesh 19.05' end)
stub(eup, '_check_signature', function () return false end)
Expand All @@ -58,6 +58,21 @@ describe('eupgrade tests #eupgrade', function()
assert.is.equal("Bad signature of firmware_latest.json", message)
end)

-- it('test is_new_version_available signature fails, but it does not mather since an alternative url has been set', function()
-- stub(eup, '_get_board_name', function () return 'test-board' end)
-- stub(eup, '_get_current_fw_version', function () return 'LibreMesh 19.05' end)
-- stub(eup, '_check_signature', function () return false end)
-- stub(utils, 'http_client_get', function () return latest_release_data end)
-- print("1",tostring(eup.get_upgrade_api_url()))
-- eup.set_custom_api_url("http://localhost")
-- assert.is_true(eup.is_meshupgrade_enabled())
-- print("2",eup.get_upgrade_api_url())
-- assert.is.equal('LibreMesh 20.10', eup.is_new_version_available()['version'])
-- eup.remove_custom_api_url()
-- print("3",eup.get_upgrade_api_url())

-- end)

it('test is_new_version_available unable to download info', function()
stub(eup, '_get_board_name', function () return 'test-board' end)
stub(eup, '_get_current_fw_version', function () return 'LibreMesh 19.05' end)
Expand Down
28 changes: 28 additions & 0 deletions packages/lime-mesh-upgrade/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
include $(TOPDIR)/rules.mk

PKG_NAME:=lime-mesh-upgrade
PKG_VERSION=$(GIT_COMMIT_DATE)-$(GIT_COMMIT_TSTAMP)
GIT_COMMIT_DATE:=$(shell git log -n 1 --pretty=%ad --date=short . )
GIT_COMMIT_TSTAMP:=$(shell git log -n 1 --pretty=%at . )

include $(INCLUDE_DIR)/package.mk

define Package/$(PKG_NAME)
CATEGORY:=LibreMesh
MAINTAINER:=selankon <[email protected]>
TITLE:=LibreMesh mesh wide firmware upgrade
DEPENDS:= +lua +libubus-lua +wget +safe-upgrade \
javierbrk marked this conversation as resolved.
Show resolved Hide resolved
+shared-state-mesh_wide_upgrade

PKGARCH:=all
endef

define Build/Compile
endef

define Package/$(PKG_NAME)/install
$(INSTALL_DIR) $(1)/
$(CP) ./files/* $(1)/
endef

$(eval $(call BuildPackage,$(PKG_NAME)))
13 changes: 13 additions & 0 deletions packages/lime-mesh-upgrade/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Mesh upgrade
This package can help you upgrade the firmware of all the routers in a network from a single node.

## Description and steps
1- A node must become main node, the main node will fetch the firmware for all the others. And expose it in the local network.
2- The main node announces the new firmware over shared-state-async
3- Other nodes with this package will get the news and try to download the firmware.
4- Once all the nodes have the firmware in their tmp folder the main node user will be able to schedule the safe upgrade of all the nodes (this last step is done synchronously).
5- After the specified time (60s default) all the nodes will start the safe upgrade process and the nodes will reboot.
6- The nodes will report that the new firmware has to be confirmed.
7- The main node user will verify that everything is in place an press the confirm button.
8- If the firmware is not confirmed after 600 seconds the routers will go back to the previous firmware.

13 changes: 13 additions & 0 deletions packages/lime-mesh-upgrade/files/etc/config/mesh-upgrade
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
config mesh-upgrade 'main'
option firmware_ver ''
option candidate_fw ''
option fw_path ''
option repo_url ''
option upgrade_state ''
option error '0'
option timestamp '0'
option main_node 'NO'
option retry_count '0'
option safeupgrade_start_mark '0'
option su_start_time_out '0'

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing new line.
I am not sure if we should use a config file per se, or this should end up into /etc/config/lime-* with the other configurations.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this config file main reason to exist is to keep the settings and status of the current upgrade. This is to be keep across, operating system process, reboots and sysupgrades. It is a backend mesh-upgrade state and config persistence.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
/etc/config/mesh-upgrade
Loading
Loading