Timing and rpcauth

README.md

@@ -77,35 +77,38 @@ External: wpkh([b91fb6c1/84'/0'/3']xpub6D1gvTP...VeMLtH6/0/*)
 Internal: wpkh([b91fb6c1/84'/0'/3']xpub6D1gvTP...VeMLtH6/1/*)
 ```
 
-if you get an ``unsupported hash type ripemd160`` error, please see [this](https://stackoverflow.com/questions/72409563/unsupported-hash-type-ripemd160-with-hashlib-in-python)
+if you get an `unsupported hash type ripemd160` error, please see [this](https://stackoverflow.com/questions/72409563/unsupported-hash-type-ripemd160-with-hashlib-in-python)
 
 ##### Create configuration file
 
 Create a config file **`lss.json`** in your home directory.
 You can use [this](https://github.com/ledgerhq/satstack/blob/master/lss.mainnet.json) sample config file as a template.
 
-Add ```"torproxy": "socks5://127.0.0.1:9050",``` to connect to a Tor client running locally so that satstack can reach a full node behind Tor.
-Replace the ```rpcurl``` with the .onion address of your node.
+Add `"torproxy": "socks5://127.0.0.1:9050",` to connect to a Tor client running locally so that satstack can reach a full node behind Tor.
+Replace the `rpcurl` with the .onion address of your node.
 
 ###### Optional account fields
 
 - **`depth`**: override the number of addresses to derive and import in the Bitcoin wallet. Defaults to `1000`.
 - **`birthday`**: set the earliest known creation date (`YYYY/MM/DD` format), for faster account import.
-Defaults to `2013/09/10` ([BIP0039](https://github.com/bitcoin/bips/blob/master/bip-0039.mediawiki) proposal date).
-Refer to the table below for a list of safe wallet birthdays to choose from.
+  Defaults to `2013/09/10` ([BIP0039](https://github.com/bitcoin/bips/blob/master/bip-0039.mediawiki) proposal date).
+  Refer to the table below for a list of safe wallet birthdays to choose from.
 
-  | Event | Date (YYYY/MM/DD) |
-  |-------|-------------------|
-  | BIP0039 proposal created | 2013/09/10 (default) |
-  | First ever BIP39 compatible Ledger device (Nano) shipped | 2014/11/24 |
-  | First ever Ledger Nano S shipped | 2016/07/28 |
+  | Event                                                    | Date (YYYY/MM/DD)    |
+  | -------------------------------------------------------- | -------------------- |
+  | BIP0039 proposal created                                 | 2013/09/10 (default) |
+  | First ever BIP39 compatible Ledger device (Nano) shipped | 2014/11/24           |
+  | First ever Ledger Nano S shipped                         | 2016/07/28           |
 
 ##### Launch Bitcoin full node
 
 Make sure you've read the [requirements](#requirements) first, and that your node is configured properly.
 Here's the recommended configuration for `bitcoin.conf`:
 
-```config
+It is not recommended to use rpcuser/rpcpassword in the bitcoin.conf of your full node use rpcauth instead.
+If you still want to use it make sure the following settings are in your bitcoin.conf file:
+
+```
 # Enable RPC server
 server=1
 
@@ -118,6 +121,44 @@ rpcuser=<user>
 rpcpassword=<password>
 ```
 
+If you want to use the newest security standard recommended by the core-devs then read the following paragraph.
+Get the rpcauth.py script from the bitcoin repo and create a new user for satstack.
+
+```
+wget https://raw.githubusercontent.com/bitcoin/bitcoin/master/share/rpcauth/rpcauth.py
+
+python3 rpcauth.py satstack
+
+Example Output:
+rpcauth=satstack:a14191e6892facf70686a397b126423$ddd6f7480817bd6f8083a2e07e24b93c4d74e667f3a001df26c5dd0ef5eafd0d
+Your password:
+VX3z87LBVc_X7NBLABLABLABLA
+```
+
+Copy the `rpcauth=` into your bitcoin.conf
+Note down the password and use them in your lss.json. There you will use the credentials
+
+```
+In your lss.json:
+
+"rpcuser": "satstack",
+"rpcpassword": "VX3z87LBVc_X7NBLABLABLABLA"
+```
+
+```config
+# Enable RPC server
+server=1
+
+# Enable indexes
+txindex=1
+blockfilterindex=1
+
+# Set RPC credentials
+# Example Auth, replace with your own.
+# see https://bitcoin.stackexchange.com/questions/46782/rpc-cookie-authentication why we are not using normal rpcuser/password
+rpcauth=satstack:a14191e6892facf70686a397b126423$ddd6f7480817bd6f8083a2e07e24b93c4d74e667f3a001df26c5dd0ef5eafd0d
+```
+
 Then launch `bitcoind` like this:
 
 ```bash
@@ -132,6 +173,17 @@ page (Linux, Windows, MacOS). Extract the tarball, and launch it as:
 ```sh
 ./lss
 ```
+There are some cmd handles which can be useful when configuring the setup the first time. List them with
+
+`./lss -h` or `./lss --help`
+
+When setting up a new wallet, the wallet is synced form the birthday date or your custom date set in `lss.json`
+When the initial sync sucessfully completes, satstack saves a file called `lss_rescan.json` at the exact location
+where the lss.json is stored. This file includes the latest blockheight your wallet was synced to, this allows 
+satstack on every restart to not rescan the whole wallet again but only rescan the difference between the current
+blockheight. You can also change the latest blockheight manually in the file which helps you to set the rescan delta
+manually. This file is only created when an initial wallet sync was successful. Removing the file will lead satstack
+to rescan the complete wallet again when starting up.
 
 If you want to build `lss` yourself, just do the following:
 
@@ -154,18 +206,18 @@ $ EXPLORER=http://127.0.0.1:20000 <Ledger Live executable>
 
 ### Misc
 
-If you get ```error=failed to load wallet: -4: Wallet file verification failed. SQLiteDatabase: Unable to obtain an exclusive lock on the database, is it being used by another bitcoind?``` maybe this is because you have bitcoind windows opened, if this is the case, please try closing them and restart lss.
+If you get `error=failed to load wallet: -4: Wallet file verification failed. SQLiteDatabase: Unable to obtain an exclusive lock on the database, is it being used by another bitcoind?` maybe this is because you have bitcoind windows opened, if this is the case, please try closing them and restart lss.
 
 ### In the press
 
-| Title   |      Source      |
-|:----------|:-------------:|
-| 🇬🇧 [Personal sovereignty with Ledger SatStack](https://blog.ledger.com/satstack) |  [blog.ledger.com](https://blog.ledger.com) |
-| 🇫🇷 [Ledger SatStack: un pont entre Bitcoin Core et votre Ledger Wallet](https://bitcoin.fr/ledger-sat-stack-un-pont-entre-bitcoin-core-et-votre-ledger-wallet/) |    [bitcoin.fr](https://bitcoin.fr)   |
-| 🇫🇷 [Votre propre coffre-fort à bitcoins… inviolable – Ledger annonce l’arrivée des full nodes Bitcoin](https://journalducoin.com/actualites/coffre-fort-bitcoins-inviolable-ledger-annonce-noeuds-complets-bitcoin) | [Journal du Coin](https://journalducoin.com) |
-| 🇫🇷 [Il est désormais possible d’exécuter un full node Bitcoin sur Ledger Live](https://fr.beincrypto.com/technologie/5770/full-node-bitcoin-ledger-live) | [beincrypto.com](https://beincrypto.com) |
-| 🇪🇸 [Ledger Live será compatible con nodos propios de Bitcoin](https://www.criptonoticias.com/tecnologia/ledger-live-sera-compatible-nodos-propios-bitcoin) | [CriptoNoticias](https://www.criptonoticias.com) |
-| 🇬🇧 [Bitcoin Tech Talk #218: Curing Monetary Stockholm Syndrome](https://jimmysong.substack.com/p/curing-monetary-stockholm-syndrome) (mention) | [Jimmy Song](https://jimmysong.substack.com) |
+| Title                                                                                                                                                                                                               |                      Source                      |
+| :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | :----------------------------------------------: |
+| 🇬🇧 [Personal sovereignty with Ledger SatStack](https://blog.ledger.com/satstack)                                                                                                                                    |    [blog.ledger.com](https://blog.ledger.com)    |
+| 🇫🇷 [Ledger SatStack: un pont entre Bitcoin Core et votre Ledger Wallet](https://bitcoin.fr/ledger-sat-stack-un-pont-entre-bitcoin-core-et-votre-ledger-wallet/)                                                     |         [bitcoin.fr](https://bitcoin.fr)         |
+| 🇫🇷 [Votre propre coffre-fort à bitcoins… inviolable – Ledger annonce l’arrivée des full nodes Bitcoin](https://journalducoin.com/actualites/coffre-fort-bitcoins-inviolable-ledger-annonce-noeuds-complets-bitcoin) |   [Journal du Coin](https://journalducoin.com)   |
+| 🇫🇷 [Il est désormais possible d’exécuter un full node Bitcoin sur Ledger Live](https://fr.beincrypto.com/technologie/5770/full-node-bitcoin-ledger-live)                                                            |     [beincrypto.com](https://beincrypto.com)     |
+| 🇪🇸 [Ledger Live será compatible con nodos propios de Bitcoin](https://www.criptonoticias.com/tecnologia/ledger-live-sera-compatible-nodos-propios-bitcoin)                                                          | [CriptoNoticias](https://www.criptonoticias.com) |
+| 🇬🇧 [Bitcoin Tech Talk #218: Curing Monetary Stockholm Syndrome](https://jimmysong.substack.com/p/curing-monetary-stockholm-syndrome) (mention)                                                                      |   [Jimmy Song](https://jimmysong.substack.com)   |
 
 ### Community
 

bus/chain.go

@@ -12,6 +12,11 @@ func (b *Bus) GetBestBlockHash() (*chainhash.Hash, error) {
 	return b.mainClient.GetBestBlockHash()
 }
 
+func (b *Bus) GetBlockCount() (int64, error) {
+	return b.mainClient.GetBlockCount()
+
+}
+
 func (b *Bus) GetBlockHash(height int64) (*chainhash.Hash, error) {
 	return b.mainClient.GetBlockHash(height)
 }

bus/infrastructure.go

@@ -5,14 +5,18 @@ import (
 	"encoding/json"
 	"fmt"
 	"os"
+	"strconv"
 	"strings"
+	"time"
 
 	"github.com/btcsuite/btcd/chaincfg"
 
 	"github.com/btcsuite/btcd/btcjson"
 	"github.com/btcsuite/btcd/chaincfg/chainhash"
 	"github.com/btcsuite/btcd/rpcclient"
+	"github.com/ledgerhq/satstack/config"
 	"github.com/ledgerhq/satstack/utils"
+	"github.com/ledgerhq/satstack/version"
 	"github.com/patrickmn/go-cache"
 	log "github.com/sirupsen/logrus"
 )
@@ -37,8 +41,16 @@ const (
 	// bitcoind's wallet.
 	walletName = "satstack"
 
-	errDuplicateWalletLoadMsg = "Duplicate -wallet filename specified."
-	errWalletAlreadyLoadedMsg = "Wallet file verification failed. Refusing to load database. Data file"
+	errDuplicateWalletLoadMsg    = "Duplicate -wallet filename specified."
+	errWalletAlreadyLoadedMsgOld = "Wallet file verification failed. Refusing to load database. Data file"
+	// Cores Responds changes so adding the new one but keeping the old for backwards compatibility
+	errWalletAlreadyLoadedMsgNew = "Wallet file verification failed. SQLiteDatabase: Unable to obtain an exclusive lock on the database"
+)
+
+var (
+	// A new wallet needs to import the descriptors therefore
+	// we need this information when starting the import worker
+	isNewWallet bool
 )
 
 // Bus represents a transport allowing access to Bitcoin RPC methods.
@@ -73,8 +85,8 @@ type Bus struct {
 	Params *chaincfg.Params
 
 	// IsPendingScan is a boolean field to indicate if satstack is currently
-	// waiting for descriptors to be scanned. One such example is when satstack
-	// is "running the numbers".
+	// waiting for descriptors to be scanned or other initial operations like "running the numbers"
+	// before the bridge can operate correctly
 	//
 	// This value can be exported for use by other packages to avoid making
 	// explorer requests before satstack is able to serve them.
@@ -156,7 +168,7 @@ func New(host string, user string, pass string, proxy string, noTLS bool, unload
 		os.Exit(1)
 	}
 
-	isNewWallet, err := loadOrCreateWallet(mainClient)
+	isNewWallet, err = loadOrCreateWallet(mainClient)
 	if err != nil {
 		return nil, err
 	}
@@ -206,7 +218,11 @@ func (b *Bus) Close(ctx context.Context) {
 		b.mainClient.Shutdown()
 		b.secondaryClient.Shutdown()
 
-		b.UnloadWallet()
+		// Only unload wallet if we are not in a pending scan
+		// otherwise the nuclear timeout corrupts the wallet state
+		if !b.IsPendingScan {
+			b.UnloadWallet()
+		}
 		done <- true
 	}()
 
@@ -303,7 +319,12 @@ func loadOrCreateWallet(client *rpcclient.Client) (bool, error) {
 		return false, nil
 	}
 
-	if rpcErr.Code == btcjson.ErrRPCWallet && strings.Contains(rpcErr.Message, errWalletAlreadyLoadedMsg) {
+	if rpcErr.Code == btcjson.ErrRPCWallet && strings.Contains(rpcErr.Message, errWalletAlreadyLoadedMsgOld) {
+		// wallet already loaded. Ignore the error and return.
+		return false, nil
+	}
+
+	if rpcErr.Code == btcjson.ErrRPCWallet && strings.Contains(rpcErr.Message, errWalletAlreadyLoadedMsgNew) {
 		// wallet already loaded. Ignore the error and return.
 		return false, nil
 	}
@@ -449,3 +470,32 @@ func (b *Bus) UnloadWallet() {
 
 	b.janitorClient.Shutdown()
 }
+
+func (b *Bus) DumpLatestRescanTime() error {
+
+	currentHeight, err := b.GetBlockCount()
+
+	if err != nil {
+		log.WithFields(log.Fields{
+			"prefix": "worker",
+		}).Error("Error fetching blockheight: %s", err)
+		return err
+
+	}
+	data := &config.ConfigurationRescan{
+		TimeStamp:       strconv.Itoa(int(time.Now().Unix())),
+		LastSyncTime:    time.Now().Format(time.ANSIC),
+		LastBlock:       currentHeight,
+		SatstackVersion: version.Version,
+	}
+	err = config.WriteRescanConf(data)
+	if err != nil {
+		log.WithFields(log.Fields{
+			"prefix": "worker",
+		}).Errorf("Error saving last timestamp to file: %s", err)
+		return err
+	}
+
+	return nil
+
+}