Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lnd-channels-remote-backup-on-change #270

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

bereska
Copy link

@bereska bereska commented Feb 20, 2020

Currently BTCPay Server docker on every change in channels state saves static channels backup (SCB) locally on the same disk (on-site). This poses a threat to LN off-chain funds recovery in case of the server crash or fatal data loss.
Every time a change is made to channel.backup file in BTCPay Server docker it will instantly be copied to set up remotes.
This will allow you to keep the latest SCB offsite (on a remote server and/or popular cloud).
NOTE: this has been tested on Linux systems only.
TO BE CONTINUED ...)

@Bereshka
Copy link

how cool ❤️

@bereska
Copy link
Author

bereska commented Feb 24, 2020

@Kukks @NicolasDorier, the cloud backup of SCB from this PR can be integrated with #201 once the storage providers are set and are supported by rclone (currently more that 30). The rclone docker https://hub.docker.com/r/rclone/rclone would probably make more sense in this case. Let me know your thoughts


# update system
echo "Updating system"
apt update && apt upgrade -y && apt autoremove
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this, thisis not the responsibility of this script.


# generate btcpayserver ssh key
echo "Generating SSH key"
ssh-keygen -o -a 100 -t ed25519 -f /root/.ssh/id_ed25519_btcpayserver -N ''
Copy link
Member

@NicolasDorier NicolasDorier Feb 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

name the key better, like lnd_backup

Do not recreate if the key exists.


# install inotify and rsync
echo "Installing inotify and rsync"
apt install -y inotify-tools rsync
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't install if already installed.


# install rclone
echo "Installing rclone"
curl https://rclone.org/install.sh | bash
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't install if already installed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not rsync?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rsync does the remote server part, rclone does the cloud 'cause it is suited for dealing with all these clouds filesystems, permissions, etc so much better

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rclone has been well maintained, supports 30+ clouds and growing, is regular user-friendly with auto config, has built-in crypt option for advanced users

@NicolasDorier
Copy link
Member

This is a sweet idea, but I think it would be better if that was a docker image instead of separate script setup via systemctl.

The main reason is that we could then easily extend for monitoring other folder later, and it does not require modifications on the host.

@bereska
Copy link
Author

bereska commented Feb 25, 2020

yes, I agree, already started working on it)
I take it to add a new container to this stack I need to create a new rclone.yml in docker-fragments and modify docker-compose.generated.yml accordingly. Is there a documentation I can look at to do it properly? Thank you for your guidance

@NicolasDorier
Copy link
Member

@bereska try to add it to the https://github.com/btcpayserver/dockerfile-deps repo.
You will need to make a docker file running your script above.
Take example on other dockerfiles, this one should not be that complex.

@bereska bereska force-pushed the lnd-channels-remote-backup-on-change branch from 0577568 to 246ee04 Compare February 27, 2020 06:51
@bereska
Copy link
Author

bereska commented Feb 27, 2020

@NicolasDorier , just fixed the scripts, working on dockerfile at https://github.com/btcpayserver/dockerfile-deps

 Date:      Thu Feb 20 23:16:14 2020 +0300
 Committer: bereska <[email protected]>
 Changes to be committed:
	new file:   lnd-channels-backup-dependencies.sh
@bereska bereska force-pushed the lnd-channels-remote-backup-on-change branch from 246ee04 to d310861 Compare February 27, 2020 07:20
@bereska
Copy link
Author

bereska commented Feb 27, 2020

@NicolasDorier , just edited "Do not recreate if the key exists."

@bereska
Copy link
Author

bereska commented Feb 28, 2020

@NicolasDorier , hope I didn't miss the point
btcpayserver/dockerfile-deps#11

Copy link
Member

@Kukks Kukks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @bereska, sorry it has taken so long with this PR. While I don't fully understand what this does, it would be better to group these scripts under a directory like Tools/lnd/backup

@bereska, do you think this script still works as-is?
@NicolasDorier Did you have any other objections to this?
@rockstardev Maybe you know better regarding LND functionality?

@bereska
Copy link
Author

bereska commented Dec 7, 2020

@Kukks hi, my script backs up to remotes lnd channels state on change. It's been successfully running on my rpi 4 with btcpay docker since 2019. But I failed to fully integrate it into docker file correctly as directed by @NicolasDorier

@Goro2030
Copy link

Goro2030 commented Jul 6, 2023

This is a SO required feature. Why is it not being merged since December? Pleaaassseeee :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants