saubyk/c-lightning-rest:0.9.0 and shesek/lightning-charge:0.4.23-1-standalone Restarting (1) n seconds ago #5119

clarketowson · 2023-06-25T00:01:06Z

Discussed in #5118

^{Originally posted by clarketowson June 25, 2023}
Hi Everyone,

We're having some trouble with a BTCPay Server we run for a public bar in Melbourne. Recently the server became unresponsive due to Read-errors on swap-device. We had not been actively monitoring it as much as we should have and it was unresponsive for an extended period approx 14-30 days. Upon rebooting everything came up OK except the BTCPay Server instance got stuck syncing blocks which I was able to resolve successfully by following the procedure in btcpayserver/btcpayserver-docker#519

Whilst the btcpayservice is up and running fine the lightning instance is not. Within the BTCPay UI under Lightning Balance it says "Could not fetch Lightning balance." Under Lightning Services Node Info it says BTC Lightning Node
Unavailable and has a red light. Under Wallets Lightning Public Node Info BTC Lightning Node is listed as Unavailable. Clicking on Services --> Ride The Lightning it says Getting Node Information... then I receive a big X Error 500 with Error: Getaddrinfo Enotfound Clightning_bitcoin_rest. Clicking on Public Key generates the error "Node Pubkey does not exist".

Looking under the hood in the docker instance the command docker ps everything is running OK except saubyk/c-lightning-rest:0.9.0 and shesek/lightning-charge:0.4.23-1-standalone which has a STATUS of Restarting (1) n seconds ago. n goes to around 60, it tries to restart again but can't then the process repeats.

The logfile of the saubyk/c-lightning-rest:0.9.0 and shesek/lightning-charge:0.4.23-1-standalone instances:

[root@pridebarbtcpayserver ~]# docker logs --tail 100 498839d6afaf
code: 'ECONNREFUSED',
syscall: 'connect',
address: '/root/.lightning/lightning-rpc'
}
Waiting /root/.nbxplorer/btc_fully_synched to be created...
The chain is fully synched
Reading config file
warn: --- Starting the cl-rest server ---
warn: --- cl-rest api server is ready and listening on port: 3001 ---
warn: --- cl-rest doc server is ready and listening on port: 4001 ---
error: Lightning client connection error
node:events:491
throw er; // Unhandled 'error' event
^

Error: connect ECONNREFUSED /root/.lightning/lightning-rpc
at PipeConnectWrap.afterConnect [as oncomplete] (node:net:1247:16)
Emitted 'error' event on LightningClient instance at:
at Socket. (/usr/src/app/lightning-client-js.js:82:23)
at Socket.emit (node:events:513:28)
at emitErrorNT (node:internal/streams/destroy:157:8)
at emitErrorCloseNT (node:internal/streams/destroy:122:3)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
errno: -111,
code: 'ECONNREFUSED',
syscall: 'connect',
address: '/root/.lightning/lightning-rpc'
}
Waiting /root/.nbxplorer/btc_fully_synched to be created...
The chain is fully synched
Reading config file
warn: --- Starting the cl-rest server ---
warn: --- cl-rest api server is ready and listening on port: 3001 ---
warn: --- cl-rest doc server is ready and listening on port: 4001 ---
error: Lightning client connection error
node:events:491
throw er; // Unhandled 'error' event
^

Error: connect ECONNREFUSED /root/.lightning/lightning-rpc
at PipeConnectWrap.afterConnect [as oncomplete] (node:net:1247:16)
Emitted 'error' event on LightningClient instance at:
at Socket. (/usr/src/app/lightning-client-js.js:82:23)
at Socket.emit (node:events:513:28)
at emitErrorNT (node:internal/streams/destroy:157:8)
at emitErrorCloseNT (node:internal/streams/destroy:122:3)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
errno: -111,
code: 'ECONNREFUSED',
syscall: 'connect',
address: '/root/.lightning/lightning-rpc'
}

[root@pridebarbtcpayserver ~]# docker logs --tail 100 6148448cb088
errno: 'ECONNREFUSED',
code: 'ECONNREFUSED',
syscall: 'connect',
address: '/etc/lightning/lightning-rpc'
}
events.js:287
throw er; // Unhandled 'error' event
^

Error: connect ECONNREFUSED /etc/lightning/lightning-rpc
at PipeConnectWrap.afterConnect [as oncomplete] (net.js:1141:16)
Emitted 'error' event on LightningClient instance at:
at Socket. (/opt/charged/node_modules/clightning-client/index.js:65:23)
at Socket.emit (events.js:310:20)
at emitErrorNT (internal/streams/destroy.js:92:8)
at emitErrorAndCloseNT (internal/streams/destroy.js:60:3)
at processTicksAndRejections (internal/process/task_queues.js:84:21) {
errno: 'ECONNREFUSED',
code: 'ECONNREFUSED',
syscall: 'connect',
address: '/etc/lightning/lightning-rpc'
}
Using lightningd directory mounted in /etc/lightning...
Starting Lightning Charge
Lightning client connection error Error: connect ECONNREFUSED /etc/lightning/lightning-rpc
at PipeConnectWrap.afterConnect [as oncomplete] (net.js:1141:16) {
errno: 'ECONNREFUSED',
code: 'ECONNREFUSED',
syscall: 'connect',
address: '/etc/lightning/lightning-rpc'
}
events.js:287
throw er; // Unhandled 'error' event
^

Error: connect ECONNREFUSED /etc/lightning/lightning-rpc
at PipeConnectWrap.afterConnect [as oncomplete] (net.js:1141:16)
Emitted 'error' event on LightningClient instance at:
at Socket. (/opt/charged/node_modules/clightning-client/index.js:65:23)
at Socket.emit (events.js:310:20)
at emitErrorNT (internal/streams/destroy.js:92:8)
at emitErrorAndCloseNT (internal/streams/destroy.js:60:3)
at processTicksAndRejections (internal/process/task_queues.js:84:21) {
errno: 'ECONNREFUSED',
code: 'ECONNREFUSED',
syscall: 'connect',
address: '/etc/lightning/lightning-rpc'
}

Running the command lsof -i -P -n | grep LISTEN | grep docker we get:

docker-pr 25869 root 4u IPv4 226556 0t0 TCP *:9735 (LISTEN)
docker-pr 25875 root 4u IPv6 227435 0t0 TCP *:9735 (LISTEN)
docker-pr 26456 root 4u IPv4 227965 0t0 TCP *:443 (LISTEN)
docker-pr 26467 root 4u IPv6 229469 0t0 TCP *:443 (LISTEN)
docker-pr 26510 root 4u IPv4 230678 0t0 TCP *:80 (LISTEN)
docker-pr 26518 root 4u IPv6 229487 0t0 TCP *:80 (LISTEN)

The clearnet address os the server is:
023ab7ce0bb6e690b197356b15697943d85c9f248b5a0e968003bc74e01fe058a7@202.153.222.237:9735

You can see it on lightningnetworkplus here: https://lightningnetwork.plus/nodes/023ab7ce0bb6e690b197356b15697943d85c9f248b5a0e968003bc74e01fe058a7

I did notice that in the time the server has been down another server has been listed on lightningnetworkplus with the same name:

https://lightningnetwork.plus/nodes?search=sombermontana&button=

I'm not sure what I can do to get the lightning instance started successfully.

Any tips and advice would be very much appreciated.

Kind Regards,

Clarke

Clarke Towson
CEO
INTJ Billing

m: +61 432 359 166
a: 7 Cullen Court Spotswood Victoria 3015 AUSTRALIA
w: https://intjbilling.com/
e: [email protected]

pavlenex · 2023-09-20T09:18:09Z

Tracked here Ride-The-Lightning/c-lightning-REST#199

pavlenex mentioned this issue Apr 8, 2024

lightning-rpc not start #5886

Closed

1 task

pavlenex closed this as not planned Won't fix, can't repro, duplicate, stale Jul 3, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

saubyk/c-lightning-rest:0.9.0 and shesek/lightning-charge:0.4.23-1-standalone Restarting (1) n seconds ago #5119

saubyk/c-lightning-rest:0.9.0 and shesek/lightning-charge:0.4.23-1-standalone Restarting (1) n seconds ago #5119

clarketowson commented Jun 25, 2023

pavlenex commented Sep 20, 2023

saubyk/c-lightning-rest:0.9.0 and shesek/lightning-charge:0.4.23-1-standalone Restarting (1) n seconds ago #5119

saubyk/c-lightning-rest:0.9.0 and shesek/lightning-charge:0.4.23-1-standalone Restarting (1) n seconds ago #5119

Comments

clarketowson commented Jun 25, 2023

Discussed in #5118

pavlenex commented Sep 20, 2023