Out of memory crash due to dcrwallet's encryption operations

[itswisdomagain]

Crash usually happens when a mobile app (with generally lower memory) attempts to create or open more wallets than the device memory can handle.

Crash has also been reported to happen on an app with just one wallet (900+ transactions) when unlocking the wallet for accounts discovery.

The following truncated crash log reveals the issue to be associated with attempted memory allocation in dcrwallet's secret key derivation method.

2020-01-28 07:43:51.447 com.decred.dcrandroid.testnet E/Go: runtime: out of memory: cannot allocate 268435456-byte block (1336705024 in use)
2020-01-28 07:43:51.447 com.decred.dcrandroid.testnet E/Go: fatal error: out of memory
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: runtime stack:
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: runtime.throw(0x8a275352, 0xd)
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: 	/usr/local/go/src/runtime/panic.go:774 +0x70
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: runtime.largeAlloc(0x10000000, 0x6d4f0101, 0xebaf3730)
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: 	/usr/local/go/src/runtime/malloc.go:1140 +0x11a
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: runtime.mallocgc.func1()
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: 	/usr/local/go/src/runtime/malloc.go:1033 +0x3a
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: runtime.systemstack(0x6d4fdfc8)
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: 	/usr/local/go/src/runtime/asm_386.s:399 +0x62
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: runtime.mstart()
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: 	/usr/local/go/src/runtime/proc.go:1146
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: goroutine 17 [running, locked to thread]:
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: runtime.systemstack_switch()
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: 	/usr/local/go/src/runtime/asm_386.s:360 fp=0x37d5b62c sp=0x37d5b628 pc=0x89c34f30
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: runtime.mallocgc(0x10000000, 0x8a7eecc0, 0x1, 0x89c1fcd5)
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: 	/usr/local/go/src/runtime/malloc.go:1032 +0x6ff fp=0x37d5b680 sp=0x37d5b62c pc=0x89be686f
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: runtime.makeslice(0x8a7eecc0, 0x4000000, 0x4000000, 0x999c000)
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: 	/usr/local/go/src/runtime/slice.go:49 +0x50 fp=0x37d5b694 sp=0x37d5b680 pc=0x89c1fd20
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: github.com/raedahgroup/dcrlibwallet/vendor/golang.org/x/crypto/scrypt.Key(0x2bdd65d0, 0x6, 0x6, 0x889f9f4, 0x20, 0x20, 0x40000, 0x8, 0x1, 0x20, ...)
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: 	/Users/collins/go/src/github.com/raedahgroup/dcrlibwallet/vendor/golang.org/x/crypto/scrypt/scrypt.go:205 +0x120 fp=0x37d5b6f4 sp=0x37d5b694 pc=0x89e20c10
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/internal/snacl.(*SecretKey).deriveKey(0x889f9f0, 0x8a27981d, 0x12, 0x8d1bab0, 0x20, 0x20)
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: 	/Users/collins/go/src/github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/internal/snacl/snacl.go:115 +0x6e fp=0x37d5b758 sp=0x37d5b6f4 pc=0x89e2156e
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/internal/snacl.NewSecretKey(0x8d1bab0, 0x40000, 0x8, 0x1, 0x2b653800, 0x9975e90, 0x89e3d415)
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: 	/Users/collins/go/src/github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/internal/snacl/snacl.go:261 +0x1b8 fp=0x37d5b80c sp=0x37d5b758 pc=0x89e220b8
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/udb.defaultNewSecretKey(0x8d1bab0, 0x8ad86450, 0x0, 0x0, 0x0)
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: 	/Users/collins/go/src/github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/udb/addressmanager.go:158 +0x3a fp=0x37d5b82c sp=0x37d5b80c pc=0x89e3d44a
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/udb.createAddressManager(0x8a8d1020, 0x9977d80, 0x9975590, 0x20, 0x21, 0x2bdd65d0, 0x6, 0x6, 0x2bdd65d8, 0x1, ...)
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: 	/Users/collins/go/src/github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/udb/addressmanager.go:2439 +0x850 fp=0x37d5ba38 sp=0x37d5b82c pc=0x89e4a9c0
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/udb.Initialize.func1(0x8a8cdf60, 0x9977d60, 0x8a275c88, 0xe)
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: 	/Users/collins/go/src/github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/udb/initialize.go:34 +0x330 fp=0x37d5bab4 sp=0x37d5ba38 pc=0x89e82c50
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/walletdb.Update(0x8a8cabc0, 0x99779a0, 0x8a8caca0, 0x8d1b6e0, 0x37d5bb6c, 0x0, 0x0)
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: 	/Users/collins/go/src/github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/walletdb/interface.go:249 +0x1bb fp=0x37d5bb44 sp=0x37d5bab4 pc=0x89e1c8ab
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/udb.Initialize(0x8a8cabc0, 0x99779a0, 0x8a8caca0, 0x8d1b6e0, 0x37c7c1c0, 0x9975590, 0x20, 0x21, 0x2bdd65d0, 0x6, ...)
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: 	/Users/collins/go/src/github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/udb/initialize.go:19 +0xbc fp=0x37d5bb9c sp=0x37d5bb44 pc=0x89e4e1fc
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3.Create(0x8a8cabc0, 0x99779a0, 0x8a8c8a10, 0x3bfe4520, 0x2bdd65d0, 0x6, 0x6, 0x2bdd65d8, 0x1, 0x8, ...)
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: 	/Users/collins/go/src/github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/wallet.go:4466 +0x178 fp=0x37d5bbf0 sp=0x37d5bb9c pc=0x8a011128

As explained towards the end of the method, the key derivation operation consumes a bit of memory, which is not freed immediately the method returns. For that reason, code was added at the end of that function to force garbage collection and free up memory. This forced garbage collection doesn't seem to work on mobile devices, as unlocking wallets use up high memory, which does not get freed up, causing the devices to slow and eventually the app crashes at some attempts to unlock the wallet.