CI Reliability 2023-09-22 #673

github-actions · 2023-09-22T00:12:59Z

Failures in node-test-pull-request/54045 to node-test-pull-request/54143 that failed 2 or more PRs
(Generated with ncu-ci walk pr --stats=true --markdown /home/runner/work/reliability/reliability/results.md)

UTC Time	RUNNING	SUCCESS	UNSTABLE	ABORTED	FAILURE	Green Rate
2023-09-22 00:12	0	12	8	1	79	12.12%

JSTest Failure

Reason	`es-module/test-vm-contextified-script-leak`
Type	JS_TEST_FAILURE
Failed PR	3 (nodejs/node#49684, nodejs/node#49523, nodejs/node#48701)
Appeared	test-ibm-rhel8-s390x-4, test-ibm-rhel8-s390x-3
First CI	https://ci.nodejs.org/job/node-test-pull-request/54048/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/54069/

Example

not ok 3491 es-module/test-vm-contextified-script-leak
  ---
  duration_ms: 1227.31500
  severity: crashed
  exitcode: -6
  stack: |-
    [469538:0x983d0e0]       49 ms: Scavenge 4.4 (4.4) -> 4.0 (5.4) MB, 0.85 / 0.00 ms  (average mu = 1.000, current mu = 1.000) allocation failure; 
    [469538:0x983d0e0]       86 ms: Scavenge 4.4 (5.7) -> 4.3 (6.4) MB, 34.11 / 0.00 ms  (average mu = 1.000, current mu = 1.000) task; 
    [469538:0x983d0e0]       90 ms: Scavenge 5.4 (6.7) -> 5.3 (8.9) MB, 0.36 / 0.00 ms  (average mu = 1.000, current mu = 1.000) task; 
    Created 128 objects
    [469538:0x983d0e0]       99 ms: Scavenge 7.4 (9.9) -> 7.3 (10.7) MB, 1.19 / 0.00 ms  (average mu = 1.000, current mu = 1.000) task; 
    [469538:0x983d0e0]      105 ms: Scavenge 8.7 (11.2) -> 8.5 (16.2) MB, 1.28 / 0.00 ms  (average mu = 1.000, current mu = 1.000) task; 
    Created 256 objects
    [469538:0x983d0e0]      289 ms: Mark-Compact 11.2 (17.4) -> 10.3 (19.7) MB, 1.74 / 0.12 ms  (+ 155.7 ms in 83 steps since start of marking, ...

Reason	`parallel/test-runner-output`
Type	JS_TEST_FAILURE
Failed PR	3 (nodejs/node#49714, nodejs/node#49654, nodejs/node#48655)
Appeared	test-digitalocean-freebsd12-x64-2, test-ibm-alpine315_container-x64-1, test-rackspace-debian10-x64-1, test-digitalocean-fedora37-x64-1, test-digitalocean-fedora38-x64-1, test-digitalocean-rhel8-x64-1, test-orka-macos11-x64-1, test-digitalocean-ubuntu1804_sharedlibs_container-x64-3, test-digitalocean-ubuntu1804_sharedlibs_container-x64-2, test-digitalocean-ubuntu1804_sharedlibs_container-x64-8, test-equinix-ubuntu1804_sharedlibs_container-arm64-1, test-equinix_mnx-smartos20-x64-3, test-equinix-ubuntu2004_sharedlibs_container-arm64-3, test-digitalocean-alpine315_container-x64-2, test-digitalocean-alpine318_container-x64-1, test-digitalocean-fedora32-x64-1, test-digitalocean-fedora38-x64-2, test-ibm-rhel8-x64-1, test-equinix_mnx-ubuntu1804-x64-1, test-equinix-ubuntu2004_container-armv7l-2, test-osuosl-rhel8-ppc64_le-2, test-equinix-rhel8_container-arm64-2, test-equinix-ubuntu1804_container-arm64-2, test-equinix-ubuntu2004_container-arm64-2, test-osuosl-ubuntu2004_container-armv7l-1, test-ibm-ubi81_container-x64-1, test-digitalocean-ubuntu1804_sharedlibs_container-x64-9, test-digitalocean-ubuntu1804_sharedlibs_container-x64-6, test-digitalocean-ubuntu1804_sharedlibs_container-x64-1, test-digitalocean-ubuntu1804_sharedlibs_container-x64-10, test-digitalocean-ubuntu1804_sharedlibs_container-x64-4, test-osuosl-aix72-ppc64_be-3, test-ibm-rhel8-s390x-4, test-nearform-macos11.0-arm64-1, test-equinix-ubuntu2004_sharedlibs_container-arm64-5, test-azure_msft-win11_vs2022-arm64-3, test-azure_msft-win11_vs2022-arm64-1, test-rackspace-win2012r2_vs2015-x64-1, test-azure_msft-win2016_vs2017-x64-2, test-rackspace-win2019_vs2019-x64-3, test-azure_msft-win2022_vs2019-x64-3, test-rackspace-win2022_vs2022-x64-2, undefined
First CI	https://ci.nodejs.org/job/node-test-pull-request/54054/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/54130/

Example

not ok 2407 parallel/test-runner-output
  ---
  duration_ms: 23943.52600
  severity: fail
  exitcode: 1
  stack: |-
    TAP version 13
    # Subtest: test runner output
        # Subtest: test-runner/output/abort.js
        ok 1 - test-runner/output/abort.js
          ---
          duration_ms: 1641.007304
          ...

Reason	`parallel/test-tls-socket-close`
Type	JS_TEST_FAILURE
Failed PR	3 (nodejs/node#49684, nodejs/node#49730, nodejs/node#49731)
Appeared	test-ibm-rhel8-s390x-4, test-ibm-rhel8-s390x-1, test-nearform-macos11.0-arm64-1
First CI	https://ci.nodejs.org/job/node-test-pull-request/54055/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/54076/

Example

not ok 2879 parallel/test-tls-socket-close
  ---
  duration_ms: 938.11100
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:125
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
    
    false !== true
    
        at Immediate._onImmediate (/home/iojs/build/workspace/node-test-commit-linuxone/test/parallel/test-tls-socket-close.js:49:18)
        at process.processImmediate (node:internal/timers:478:21) {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: false,
      expected: true,
      operator: 'strictEqual'
    }
    
    Node.js v21.0.0-pre
  ...

Reason	`wpt/test-structured-clone`
Type	JS_TEST_FAILURE
Failed PR	3 (nodejs/node#49559, nodejs/node#49745, nodejs/node#48434)
Appeared	test-equinix-ubuntu2004_container-armv7l-1, test-equinix-ubuntu2004_container-armv7l-2, test-digitalocean-ubuntu1804_sharedlibs_container-x64-3
First CI	https://ci.nodejs.org/job/node-test-pull-request/54106/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/54142/

Example

not ok 3925 wpt/test-structured-clone
  ---
  duration_ms: 413.23400
  severity: fail
  exitcode: 1
  stack: |-
    [PASS] primitive undefined
    [PASS] primitive null
    [PASS] primitive true
    [PASS] primitive false
    [PASS] primitive string, empty string
    [PASS] primitive string, lone high surrogate
    [PASS] primitive string, lone low surrogate
    [PASS] primitive string, NUL
    [PASS] primitive string, astral character
    [PASS] primitive number, 0.2
    [PASS] primitive number, 0
    [PASS] primitive number, -0
    [PASS] primitive number, NaN
    [PASS] primitive number, Infinity
    [PASS] primitive number, -Infinity
    [PASS] primitive number, 9007199254740992
    [PASS] primitive number, -9007199254740992
    [PASS] primitive number, 9007199254740994
    [PASS] primitive number, -9007199254740994
    [PASS] primitive BigInt, 0n
    [PASS] primitive BigInt, -0n
    [PASS] primitive BigInt, -9007199254740994000n
    [PASS] primitive BigInt, -9007199254740994000900719925474099400090071992...

Reason	`parallel/test-fs-watch-recursive-update-file`
Type	JS_TEST_FAILURE
Failed PR	2 (nodejs/node#49710, nodejs/node#49684)
Appeared	test-ibm-rhel8-s390x-4
First CI	https://ci.nodejs.org/job/node-test-pull-request/54045/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/54055/

Example

not ok 1370 parallel/test-fs-watch-recursive-update-file
  ---
  duration_ms: 120048.50200
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...

Reason	`parallel/test-inspector-async-stack-traces-promise-then`
Type	JS_TEST_FAILURE
Failed PR	2 (nodejs/node#49696, nodejs/node#49712)
Appeared	test-ibm-rhel8-s390x-4
First CI	https://ci.nodejs.org/job/node-test-pull-request/54096/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/54127/

Example

not ok 1829 parallel/test-inspector-async-stack-traces-promise-then
  ---
  duration_ms: 15352.92800
  severity: fail
  exitcode: 1
  stack: |-
    [test] Connecting to a child Node process
    [test] Testing /json/list
    [err] Debugger listening on ws://127.0.0.1:35977/f6b3d782-11a4-4f95-9d28-a650ca5c8a28
    [err] For help, see: https://nodejs.org/en/docs/inspector
    [err] 
    [err] Debugger attached.
    [err] 
    Timed out waiting for matching notification (break on [eval]:0)
    1
  ...

Reason	`parallel/test-performance-function`
Type	JS_TEST_FAILURE
Failed PR	2 (nodejs/node#38771, nodejs/node#49745)
Appeared	test-ibm-rhel8-x64-1, test-ibm-ubuntu1804-x64-2
First CI	https://ci.nodejs.org/job/node-test-pull-request/54080/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/54115/

Example

not ok 2289 parallel/test-performance-function
  ---
  duration_ms: 223.18900
  severity: fail
  exitcode: 1
  stack: |-
    node:internal/errors:497
        ErrorCaptureStackTrace(err);
        ^
    
    RangeError [ERR_OUT_OF_RANGE]: The value of "val" is out of range. It must be >= 1 && <= 9007199254740991. Received 0
        at RecordableHistogram.record (node:internal/histogram:283:5)
        at processComplete (node:internal/perf/timerify:40:15)
        at timerified m (node:internal/perf/timerify:87:5)
        at /home/iojs/build/workspace/node-test-commit-linux/test/parallel/test-performance-function.js:98:5
        at runNextTicks (node:internal/process/task_queues:60:5)
        at listOnTimeout (node:internal/timers:540:9)
        at process.processTimers (node:internal/timers:514:7) {
      code: 'ERR_OUT_OF_RANGE'
    }
    
    Node.js v21.0.0-pre
  ...

Reason	`parallel/test-worker-memory`
Type	JS_TEST_FAILURE
Failed PR	2 (nodejs/node#49340, nodejs/node#49713)
Appeared	test-rackspace-win2019_vs2019-x64-4, test-rackspace-win2012r2_vs2015-x64-2
First CI	https://ci.nodejs.org/job/node-test-pull-request/54070/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/54110/

Example

not ok 860 parallel/test-worker-memory
  ---
  duration_ms: 2045.05000
  severity: fail
  exitcode: 3221226505
  stack: |-
    run() called with n=7.5 (numWorkers=8)
    run() called with n=7.5 (numWorkers=8)
    run() called with n=7.5 (numWorkers=8)
    run() called with n=7.5 (numWorkers=8)
    run() called with n=7.5 (numWorkers=8)
    run() called with n=7.5 (numWorkers=8)
    run() called with n=7.5 (numWorkers=8)
    run() called with n=7.5 (numWorkers=8)
    run() called with n=6.5 (numWorkers=8)
    run() called with n=6.5 (numWorkers=8)
    run() called with n=6.5 (numWorkers=8)
    run() called with n=6.5 (numWorkers=8)
    run() called with n=6.5 (numWorkers=8)
    run() called with n=6.5 (numWorkers=8)
    run() called with n=6.5 (numWorkers=8)
    run() called with n=6.5 (numWorkers=8)
    run() called with n=5.5 (numWorkers=8)
    run() called with n=5.5 (numWorkers=8)
    run() called with n=5.5 (numWorkers=8)
    run() called with n=5.5 (numWorkers=8)
  ...

Reason	`sequential/test-vm-timeout-escape-promise-module-2`
Type	JS_TEST_FAILURE
Failed PR	2 (nodejs/node#48655, nodejs/node#49750)
Appeared	test-ibm-rhel8-s390x-1, test-osuosl-aix72-ppc64_be-2, test-ibm-rhel8-s390x-3
First CI	https://ci.nodejs.org/job/node-test-pull-request/54130/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/54136/

Example

not ok 4020 sequential/test-vm-timeout-escape-promise-module-2
  ---
  duration_ms: 425.47700
  severity: fail
  exitcode: 1
  stack: |-
    (node:3225288) ExperimentalWarning: VM Modules is an experimental feature and might change at any time
    (Use `node --trace-warnings ...` to show where the warning was created)
    /home/iojs/build/workspace/node-test-commit-linuxone/test/sequential/test-vm-timeout-escape-promise-module-2.js:22
          throw new Error(
                ^
    
    Error: escaped timeout at 100 milliseconds!
        at loop (/home/iojs/build/workspace/node-test-commit-linuxone/test/sequential/test-vm-timeout-escape-promise-module-2.js:22:13)
        at vm:module(0):1:30
        at SourceTextModule.evaluate (node:internal/vm/module:228:23)
        at assert.rejects.code (/home/iojs/build/workspace/node-test-commit-linuxone/test/sequential/test-vm-timeout-escape-promise-module-2.js:38:16)
        at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    
    Node.j...

Jenkins Failure

Reason	`Build timed out (after 60 minutes). Marking the build as failed.`
Type	JENKINS_FAILURE
Failed PR	9 (nodejs/node#49523, nodejs/node#49714, nodejs/node#49713, nodejs/node#49720, nodejs/node#49654, nodejs/node#38771, nodejs/node#49745, nodejs/node#48655, nodejs/node#49743)
Appeared	test-rackspace-win2022_vs2022-x64-6, test-rackspace-win2019_vs2019-x64-2, test-rackspace-win2022_vs2022-x64-1
First CI	https://ci.nodejs.org/job/node-test-pull-request/54049/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/54143/

Example

Build timed out (after 60 minutes). Marking the build as failed.

Git Failure

Reason	`ERROR: Error fetching remote repo 'origin'`
Type	GIT_FAILURE
Failed PR	21 (nodejs/node#48701, nodejs/node#49730, nodejs/node#49728, nodejs/node#49731, nodejs/node#49720, nodejs/node#49340, nodejs/node#49709, nodejs/node#45427, nodejs/node#38733, nodejs/node#49654, nodejs/node#49692, nodejs/node#49696, nodejs/node#49721, nodejs/node#49646, nodejs/node#49559, nodejs/node#49740, nodejs/node#38771, nodejs/node#49744, nodejs/node#46018, nodejs/node#49745, nodejs/node#49743)
Appeared	test-equinix-ubuntu1804_sharedlibs_container-arm64-1, test-equinix-ubuntu2004_container-armv7l-1, test-equinix-ubuntu2004_sharedlibs_container-arm64-3, test-equinix-ubuntu2004_sharedlibs_container-arm64-1, test-equinix-ubuntu2004_sharedlibs_container-arm64-2
First CI	https://ci.nodejs.org/job/node-test-pull-request/54069/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/54117/

Example

ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from [email protected]:nodejs/node.git
	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:999)
	at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1241)
	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1305)
	at hudson.scm.SCM.checkout(SCM.java:540)

Reason	`Command "git fetch --no-tags --progress [email protected]:nodejs/node.git +refs/heads/:refs/remotes/origin/ +refs/heads/v16.x-staging:refs/remotes/origin/_jenkins_local_branch" returned status code 128:`
Type	GIT_FAILURE
Failed PR	14 (nodejs/node#49340, nodejs/node#49709, nodejs/node#45427, nodejs/node#38733, nodejs/node#49654, nodejs/node#49692, nodejs/node#49696, nodejs/node#49721, nodejs/node#49646, nodejs/node#49559, nodejs/node#49740, nodejs/node#38771, nodejs/node#49744, nodejs/node#46018)
Appeared	test-equinix-centos7_container-arm64-1
First CI	https://ci.nodejs.org/job/node-test-pull-request/54081/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/54114/

Example

hudson.plugins.git.GitException: Command "git fetch --no-tags --progress [email protected]:nodejs/node.git +refs/heads/*:refs/remotes/origin/* +refs/heads/v16.x-staging:refs/remotes/origin/_jenkins_local_branch" returned status code 128:
stdout: 
stderr: Warning: the ECDSA host key for 'github.com' differs from the key for the IP address '140.82.114.3'
Offending key for IP in /home/iojs/.ssh/known_hosts:4
Matching host key in /home/iojs/.ssh/known_hosts:9

Build Failure

Reason	`ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?`
Type	BUILD_FAILURE
Failed PR	9 (nodejs/node#49340, nodejs/node#45427, nodejs/node#49745, nodejs/node#49743, nodejs/node#49237, nodejs/node#46018, nodejs/node#49744, nodejs/node#49741, nodejs/node#38771)
Appeared	test-equinix-ubuntu2004_container-arm64-2, test-equinix-ubuntu2004_container-armv7l-1, test-osuosl-ubuntu2004_container-armv7l-1, test-equinix-ubuntu2004_sharedlibs_container-arm64-5, test-equinix-ubuntu1804_sharedlibs_container-arm64-1
First CI	https://ci.nodejs.org/job/node-test-pull-request/54050/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/54125/

Example

ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?
Collecting metadata...
Metadata collection done.
Notifying upstream projects of job completion
Finished: FAILURE

Reason	`error: The last gc run reported the following. Please correct the root cause`
Type	BUILD_FAILURE
Failed PR	4 (nodejs/node#49698, nodejs/node#49340, nodejs/node#45427, nodejs/node#48434)
Appeared	test-ibm-ubuntu1804-x64-1, test-equinix_mnx-ubuntu1804-x64-1, test-digitalocean-ubuntu1804_sharedlibs_container-x64-1, test-digitalocean-ubuntu1804_sharedlibs_container-x64-2, test-digitalocean-ubuntu1804_sharedlibs_container-x64-9, test-digitalocean-ubuntu1804_sharedlibs_container-x64-8, test-digitalocean-ubuntu1804_sharedlibs_container-x64-10, test-digitalocean-ubuntu1804_sharedlibs_container-x64-6, undefined, test-digitalocean-ubuntu1804_sharedlibs_container-x64-3
First CI	https://ci.nodejs.org/job/node-test-pull-request/54052/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/54142/

Example

error: The last gc run reported the following. Please correct the root cause
and remove .git/gc.log.
Automatic cleanup will not be performed until the file is removed.

warning: There are too many unreachable loose objects; run 'git prune' to remove them.

Reason	`fatal: '/home/iojs/.ccache/node.shared.reference' does not appear to be a git repository`
Type	BUILD_FAILURE
Failed PR	4 (nodejs/node#49684, nodejs/node#49340, nodejs/node#49714, nodejs/node#49654)
Appeared	test-equinix-ubuntu2004_container-armv7l-1
First CI	https://ci.nodejs.org/job/node-test-pull-request/54048/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/54087/

Example

fatal: '/home/iojs/.ccache/node.shared.reference' does not appear to be a git repository
fatal: Could not read from remote repository.
fatal: failed to run repack

Reason	`fatal: No rebase in progress?`
Type	BUILD_FAILURE
Failed PR	2 (nodejs/node#49340, nodejs/node#45427)
Appeared	test-rackspace-debian10-x64-1, test-ibm-ubi81_container-x64-1, test-softlayer-debian10-x64-1, test-digitalocean-ubi81_container-x64-1
First CI	https://ci.nodejs.org/job/node-test-pull-request/54081/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/54090/

Example

fatal: No rebase in progress?

CCTest Failure

Reason	`Value of: (!expectation.read_expected)`
Type	CC_TEST_FAILURE
Failed PR	2 (nodejs/node#49714, nodejs/node#49744)
Appeared	test-digitalocean-alpine318_container-x64-1
First CI	https://ci.nodejs.org/job/node-test-pull-request/54056/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/54113/

Example

Value of: (!expectation.read_expected)
  Actual: true
Expected: false

[  FAILED  ] InspectorSocketTest.HostIPv6NonRoutable (4999 ms)

Progress

The text was updated successfully, but these errors were encountered:

This has been flaky on many platforms for months. Mark it as flaky for now to avoid blocking the CI. PR-URL: #49854 Refs: #49853 Refs: nodejs/reliability#673 Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: LiviaMedeiros <[email protected]> Reviewed-By: Luigi Pinca <[email protected]>

This has been flaky on many platforms for months. Mark it as flaky for now to avoid blocking the CI. PR-URL: nodejs#49854 Refs: nodejs#49853 Refs: nodejs/reliability#673 Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: LiviaMedeiros <[email protected]> Reviewed-By: Luigi Pinca <[email protected]>

This was referenced Sep 25, 2023

test: mark test-runner-output as flaky nodejs/node#49854

Merged

parallel/test-runner-output is flaky nodejs/node#49853

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CI Reliability 2023-09-22 #673

CI Reliability 2023-09-22 #673

github-actions bot commented Sep 22, 2023

CI Reliability 2023-09-22 #673

CI Reliability 2023-09-22 #673

Comments

github-actions bot commented Sep 22, 2023

JSTest Failure

Jenkins Failure

Git Failure

Build Failure

CCTest Failure

Progress