Fix build and runtime failures of snap #3172

dilyn-corner · 2024-10-08T21:43:12Z

BTW these patches would be more useful as pull requests if you're able to create them.

Done! 🫡

Closes #3171
Closes #3167

WebThings Gateway definitely usually runs under Node.js 10 ... This is really a long way of saying that it shouldn't be necessary to upgrade Node.js from version 10 to version 12 to get the snap to build, because the snap was building locally with Node.js 10 before without a problem.

That's interesting. I'm not familiar enough with Node or JS to explain how that's the case.

Might there be a reason that local builds of the snap with Node.js 10 were working fine, but remote builds on snapcraft.io are somehow different?

They shouldn't be different. How are you building the snap locally? I'm building with snapcraft 8.x (various versions) with --use-lxd, I'm trying now with snapcraft 7.5.5 in a Jammy container with --destructive-mode to compare.

(Just ran a build in a Jammy LXD container using snapcraft 7.5.5 with snapcraft --destructive-mode and installed the snap with --dangerous, I get the same error message of 2024-10-08T21:39:41Z webthings-gateway.webthings-gateway[4792]: ReferenceError: globalThis is not defined).

The build environment shouldn't really impact this, as this is a runtime problem; indeed, this problem should have always existed, as snapcraft will always fetch the node version you specify (as part of the npm plugin).

I'm assuming this is intended to fix an error which I also #3167 (comment) with local builds of the snap.

Indeed it is!

In that case I found that when I started from a fresh git checkout the error went away.

Does this mean that you are building with --destructive-mode?

Whilst I am OK with the proposed fix, I would prefer to understand the cause of this error. It could be something as simple as the build directory already existing.

The real reason the failure happens is if you build, don't clean, and attempt a rebuild; the file already exists in the target location, and for some reason snapcraft is rerunning the build step for that part again. But cp will refuse to overwrite the same file, and returns with exit 1; because of that nonzero exit code, snapcraft considers a step to have failed and stops the build.

In this specific instance, overwriting is safe. Generally speaking, overwriting is usually the behavior you would want (for instance, many Makefiles will use cp -f, rm -f, ln -f, etc.).

Signed-off-by: Dilyn Corner <[email protected]>

automake requires autoconf, and most systems using autotools require both Signed-off-by: Dilyn Corner <[email protected]>

cp will bail just in case source=dest, and in that case we do not care if dest is overwritten. Signed-off-by: Dilyn Corner <[email protected]>

dilyn-corner · 2024-10-08T21:59:41Z

That's interesting. I'm not familiar enough with Node or JS to explain how that's the case.

For instance, there are a lot of these types of messages when building:

:: npm WARN notsup Unsupported engine for [email protected]: wanted: {"node":">=12.0.0"} (current: {"node":"10.24.1","npm":"6.14.12"})                  
:: npm WARN notsup Not compatible with your version of node/npm: [email protected]                                                                      
:: npm WARN notsup Unsupported engine for [email protected]: wanted: {"node":">=14.14"} (current: {"node":"10.24.1","npm":"6.14.12"})                                  
:: npm WARN notsup Not compatible with your version of node/npm: [email protected]                                                                                     
:: npm WARN notsup Unsupported engine for [email protected]: wanted: {"node":">= 16"} (current: {"node":"10.24.1","npm":"6.14.12"})                               
:: npm WARN notsup Not compatible with your version of node/npm: [email protected]                                                                                
:: npm WARN notsup Unsupported engine for [email protected]: wanted: {"node":">= 12.0.0"} (current: {"node":"10.24.1","npm":"6.14.12"})                           
:: npm WARN notsup Not compatible with your version of node/npm: [email protected]                                                                                
:: npm WARN notsup Unsupported engine for [email protected]: wanted: {"node":"^12.13.0 || ^14.15.0 || >=16.0.0"} (current: {"node":"10.24.1","npm":"6.14.12"})      
:: npm WARN notsup Not compatible with your version of node/npm: [email protected]                                                                                  
:: npm WARN notsup Unsupported engine for [email protected]: wanted: {"node":">= 12"} (current: {"node":"10.24.1","npm":"6.14.12"})                             
:: npm WARN notsup Not compatible with your version of node/npm: [email protected]                                                                              
:: npm WARN notsup Unsupported engine for [email protected]: wanted: {"node":"^12.13.0 || ^14.15.0 || >=16.0.0"} (current: {"node":"10.24.1","npm":"6.14.12"})                                                                                                                                                            
:: npm WARN notsup Not compatible with your version of node/npm: [email protected]                                                                        
:: npm WARN notsup Unsupported engine for [email protected]: wanted: {"node":"^12.13.0 || ^14.15.0 || >=16.0.0"} (current: {"node":"10.24.1","npm":"6.14.12"})       
:: npm WARN notsup Not compatible with your version of node/npm: [email protected]                                                                                   
:: npm WARN notsup Unsupported engine for [email protected]: wanted: {"node":">= 14"} (current: {"node":"10.24.1","npm":"6.14.12"})                      
:: npm WARN notsup Not compatible with your version of node/npm: [email protected]                                                                       
:: npm WARN notsup Unsupported engine for [email protected]: wanted: {"node":">=16"} (current: {"node":"10.24.1","npm":"6.14.12"})                                
:: npm WARN notsup Not compatible with your version of node/npm: [email protected]                                                                                
:: npm WARN notsup Unsupported engine for [email protected]: wanted: {"node":">= 14"} (current: {"node":"10.24.1","npm":"6.14.12"})                             
:: npm WARN notsup Not compatible with your version of node/npm: [email protected]                                                                              
:: npm WARN notsup Unsupported engine for [email protected]: wanted: {"node":">= 14.0.0"} (current: {"node":"10.24.1","npm":"6.14.12"})                        
:: npm WARN notsup Not compatible with your version of node/npm: [email protected]                                                                             
:: npm WARN notsup Unsupported engine for [email protected]: wanted: {"node":">= 12.0.0"} (current: {"node":"10.24.1","npm":"6.14.12"})                  
:: npm WARN notsup Not compatible with your version of node/npm: [email protected]                                                                       
:: npm WARN notsup Unsupported engine for [email protected]: wanted: {"node":">= 12.0.0"} (current: {"node":"10.24.1","npm":"6.14.12"})                            
:: npm WARN notsup Not compatible with your version of node/npm: [email protected]                                                                                 
:: npm WARN notsup Unsupported engine for [email protected]: wanted: {"node":"^12.22.0 || ^14.17.0 || >=16.0.0"} (current: {"node":"10.24.1","npm":"6.14.12"})                                                                                                                                                             
:: npm WARN notsup Not compatible with your version of node/npm: [email protected]

benfrancis · 2024-10-09T19:20:24Z

BTW these patches would be more useful as pull requests if you're able to create them.

Done! 🫡

Thank you very much for these patches.

Might there be a reason that local builds of the snap with Node.js 10 were working fine, but remote builds on snapcraft.io are somehow different?

They shouldn't be different. How are you building the snap locally?

I'm just running snapcraft on Ubuntu Desktop from a clone of the git repo, and sometimes snapcraft clean && snapcraft. Current version is snapcraft 8.4.2.

I'm building with snapcraft 8.x (various versions) with --use-lxd, I'm trying now with snapcraft 7.5.5 in a Jammy container with --destructive-mode to compare.

(Just ran a build in a Jammy LXD container using snapcraft 7.5.5 with snapcraft --destructive-mode and installed the snap with --dangerous, I get the same error message of 2024-10-08T21:39:41Z webthings-gateway.webthings-gateway[4792]: ReferenceError: globalThis is not defined).

The build environment shouldn't really impact this, as this is a runtime problem; indeed, this problem should have always existed, as snapcraft will always fetch the node version you specify (as part of the npm plugin).

Curiously I've not seen this error. If you want to examine a successfully built snap using Node.js 10.24.1 as per the initial snapcraft.yaml provided by @ogra1 you can install it using $ snap install webthings-gateway --revision 1 (on an AMD64 machine). I built that snap locally and uploaded it to the snap store because automated builds were broken.

Apart from this I run WebThings Gateway locally with Node.js 10 all the time in development and that's what the current stable 1.1 release is using, so it should run fine.

I'm assuming this is intended to fix an error which I also #3167 (comment) with local builds of the snap.

Indeed it is!

In that case I found that when I started from a fresh git checkout the error went away.

Does this mean that you are building with --destructive-mode?

No I mean I literally created a new local clone of the GitHub repo and ran snapcraft and the error went away.

Whilst I am OK with the proposed fix, I would prefer to understand the cause of this error. It could be something as simple as the build directory already existing.

The real reason the failure happens is if you build, don't clean, and attempt a rebuild; the file already exists in the target location, and for some reason snapcraft is rerunning the build step for that part again. But cp will refuse to overwrite the same file, and returns with exit 1; because of that nonzero exit code, snapcraft considers a step to have failed and stops the build.

In this specific instance, overwriting is safe. Generally speaking, overwriting is usually the behavior you would want (for instance, many Makefiles will use cp -f, rm -f, ln -f, etc.).

Hmm, OK.

For instance, there are a lot of these types of messages when building:

Yes these are just warnings and this is normal. The upgrade to Node.js 20 should eventually get rid of all these warnings.

benfrancis · 2024-10-09T19:20:27Z

With the patches in this PR I have successfully been able to locally build the snap and can confirm that it runs.

I can also confirm that without the commit which bumps the Node.js version the snap fails to start.

$ sudo journalctl -fu snap.webthings-gateway.webthings-gateway.service
Oct 09 20:10:15 frell webthings-gateway.webthings-gateway[82082]: npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
Oct 09 20:10:15 frell webthings-gateway.webthings-gateway[82082]: npm ERR! A complete log of this run can be found in:
Oct 09 20:10:15 frell webthings-gateway.webthings-gateway[82082]: npm ERR!     /root/snap/webthings-gateway/x2/.npm/_logs/2024-10-09T19_10_15_457Z-debug.log
Oct 09 20:10:15 frell systemd[1]: snap.webthings-gateway.webthings-gateway.service: Main process exited, code=exited, status=1/FAILURE
Oct 09 20:10:15 frell systemd[1]: snap.webthings-gateway.webthings-gateway.service: Failed with result 'exit-code'.
Oct 09 20:10:15 frell systemd[1]: snap.webthings-gateway.webthings-gateway.service: Scheduled restart job, restart counter is at 5.
Oct 09 20:10:15 frell systemd[1]: Stopped Service for snap application webthings-gateway.webthings-gateway.
Oct 09 20:10:15 frell systemd[1]: snap.webthings-gateway.webthings-gateway.service: Start request repeated too quickly.
Oct 09 20:10:15 frell systemd[1]: snap.webthings-gateway.webthings-gateway.service: Failed with result 'exit-code'.
Oct 09 20:10:15 frell systemd[1]: Failed to start Service for snap application webthings-gateway.webthings-gateway.

$ snap logs -f webthings-gateway
2024-10-09T20:10:15+01:00 webthings-gateway.webthings-gateway[82082]: npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
2024-10-09T20:10:15+01:00 webthings-gateway.webthings-gateway[82082]: npm ERR! A complete log of this run can be found in:
2024-10-09T20:10:15+01:00 webthings-gateway.webthings-gateway[82082]: npm ERR!     /root/snap/webthings-gateway/x2/.npm/_logs/2024-10-09T19_10_15_457Z-debug.log
2024-10-09T20:10:15+01:00 systemd[1]: snap.webthings-gateway.webthings-gateway.service: Main process exited, code=exited, status=1/FAILURE
2024-10-09T20:10:15+01:00 systemd[1]: snap.webthings-gateway.webthings-gateway.service: Failed with result 'exit-code'.
2024-10-09T20:10:15+01:00 systemd[1]: snap.webthings-gateway.webthings-gateway.service: Scheduled restart job, restart counter is at 5.
2024-10-09T20:10:15+01:00 systemd[1]: Stopped Service for snap application webthings-gateway.webthings-gateway.
2024-10-09T20:10:15+01:00 systemd[1]: snap.webthings-gateway.webthings-gateway.service: Start request repeated too quickly.
2024-10-09T20:10:15+01:00 systemd[1]: snap.webthings-gateway.webthings-gateway.service: Failed with result 'exit-code'.
2024-10-09T20:10:15+01:00 systemd[1]: Failed to start Service for snap application webthings-gateway.webthings-gateway.

$ sudo less /root/snap/webthings-gateway/x2/.npm/_logs/2024-10-09T19_10_15_457Z-debug.log
0 info it worked if it ends with ok
1 verbose cli [ '/snap/webthings-gateway/x2/bin/node',
1 verbose cli   '/snap/webthings-gateway/x2/bin/npm',
1 verbose cli   '--prefix',
1 verbose cli   '/snap/webthings-gateway/x2/lib/node_modules/webthings-gateway',
1 verbose cli   'run',
1 verbose cli   'run-only' ]
2 info using [email protected]
3 info using [email protected]
4 verbose run-script [ 'prerun-only', 'run-only', 'postrun-only' ]
5 info lifecycle [email protected]~prerun-only: [email protected]
6 info lifecycle [email protected]~run-only: [email protected]
7 verbose lifecycle [email protected]~run-only: unsafe-perm in lifecycle true
8 verbose lifecycle [email protected]~run-only: PATH: /snap/webthings-gateway/x2/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/snap/webthings-gateway/x2/lib/node_modules/webthings-gateway/node_modules/.bin:/snap/webthings-gateway/x2/lib/node_modules/.bin:/snap/webthings-gateway/x2/usr/sbin:/snap/webthings-gateway/x2/usr/bin:/snap/webthings-gateway/x2/sbin:/snap/webthings-gateway/x2/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
9 verbose lifecycle [email protected]~run-only: CWD: /snap/webthings-gateway/x2/lib/node_modules/webthings-gateway
10 silly lifecycle [email protected]~run-only: Args: [ '-c', 'node build/app.js' ]
11 silly lifecycle [email protected]~run-only: Returned: code: 1  signal: null
12 info lifecycle [email protected]~run-only: Failed to exec run-only script
13 verbose stack Error: [email protected] run-only: `node build/app.js`
13 verbose stack Exit status 1
13 verbose stack     at EventEmitter.<anonymous> (/snap/webthings-gateway/x2/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:332:16)
13 verbose stack     at EventEmitter.emit (events.js:198:13)
13 verbose stack     at ChildProcess.<anonymous> (/snap/webthings-gateway/x2/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
13 verbose stack     at ChildProcess.emit (events.js:198:13)
13 verbose stack     at maybeClose (internal/child_process.js:982:16)
13 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:259:5)
14 verbose pkgid [email protected]
15 verbose cwd /var/snap/webthings-gateway/x2
16 verbose Linux 6.8.0-45-generic
17 verbose argv "/snap/webthings-gateway/x2/bin/node" "/snap/webthings-gateway/x2/bin/npm" "--prefix" "/snap/webthings-gateway/x2/lib/node_modules/webthings-gateway" "run" "run-only"
18 verbose node v10.24.1
19 verbose npm  v6.14.12
20 error code ELIFECYCLE
21 error errno 1
22 error [email protected] run-only: `node build/app.js`
22 error Exit status 1
23 error Failed at the [email protected] run-only script.
23 error This is probably not a problem with npm. There is likely additional logging output above.
24 verbose exit [ 1, true ]
/root/snap/webthings-gateway/x2/.npm/_logs/2024-10-09T19_10_15_457Z-debug.log (END)

In which log did you find the error ReferenceError: globalThis is not defined? I haven't been able to find it and I would like to better understand what's going on.

In the meantime I'm going to land these patches because they do seem to fix the snap and I don't particularly mind upgrading the snap to Node.js 12 given I'm trying to upgrade to Node.js 20 anyway. However, that does diverge from the main build target of current master and I find it disconcerting that a snapcraft.yaml which was previously working for both me and @ogra1 has suddenly stopped working.

Let's see if landing this PR gets the remote builds working on other platforms.

Thanks

dilyn-corner added 3 commits October 8, 2024 16:20

snapcraft.yaml: bump node version to support globalThis

b9c0d88

Signed-off-by: Dilyn Corner <[email protected]>

snapcraft.yaml: switch autoconf to automake

5a80418

automake requires autoconf, and most systems using autotools require both Signed-off-by: Dilyn Corner <[email protected]>

snapcraft.yaml: identical files should not fail cp

2926333

cp will bail just in case source=dest, and in that case we do not care if dest is overwritten. Signed-off-by: Dilyn Corner <[email protected]>

benfrancis approved these changes Oct 9, 2024

View reviewed changes

benfrancis merged commit bc9d82e into WebThingsIO:master Oct 9, 2024
3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix build and runtime failures of snap #3172

Fix build and runtime failures of snap #3172

dilyn-corner commented Oct 8, 2024

dilyn-corner commented Oct 8, 2024

benfrancis commented Oct 9, 2024

benfrancis commented Oct 9, 2024

Fix build and runtime failures of snap #3172

Fix build and runtime failures of snap #3172

Conversation

dilyn-corner commented Oct 8, 2024

dilyn-corner commented Oct 8, 2024

benfrancis commented Oct 9, 2024

benfrancis commented Oct 9, 2024