sdk-wallet: some refactoring, additional explanations

sdk-wallet/README.md

@@ -1,13 +1,36 @@
-## Keystores
+## Package `controllers`
 
-Components in `sdk-wallet/keystore`, even if they do share a common purpose - storing secret keys - are not meant to be used interchangeably. Generally speaking, they do not (are not required to) share a common interface. If needed (for exotic use-cases), client code can design customized adapters over these components in order to unify their interfaces (this is not covered by specs).
+This package holds the most high-level components of **sdk-wallet**, the ones generally used to sign transactions and messages.
 
-**Implementation detail:** an instance of `EncryptedKeystore` (which is a wrapper over the well-known JSON wallet) holds decrypted data within its state.
+```mermaid
+classDiagram
+IWalletProvider <|-- LedgerWalletController : implements
+IWalletProvider <|-- SecretKeyWalletController : implements
+IWalletProvider <|-- MnemonicWalletController : implements
+<<Interface>> IWalletProvider
+IWalletProvider : get_address(address_index)
+IWalletProvider : get_current_address()
+IWalletProvider : select_current_address(address_index)
+IWalletProvider : sign_auth_token(auth_token)
+IWalletProvider : sign_transaction(serialized_transaction)
+IWalletProvider : sign_message(serialized_message)
+```
 
-**Design detail:** components in `sdk-wallet/keystore` should not depend on `Address` within their public interface (though they are allowed to depend on it within their implementation). For example, the `export` functionality of `EncryptedKeystore` requires the functionality provided by `Address` (conversion from public key bytes to bech32 representation).
+## Package `crypto`
+
+This package must be agnostic to all the other ones. Though, for languages that do no support **structural typing** (e.g. C#), the package is allowed to depend on the interfaces `ISecretKey` and `IPublicKey` (which are externally defined). In all other cases, the package must be completely self-contained and independent.
+
+## Package `keystores`
+
+TBD
+
+**Implementation detail:** an instance of `EncryptedKeystoreWithMnemonic` or `EncryptedKeystoreWithSecretKey` (wrappers over the well-known JSON wallet) hold decrypted data within their state.
+
+**Design detail:** components in `sdk-wallet/keystore` should not reference `Address` within their public interface (though they are allowed to depend on it within their implementation).
+
+### Useful links
 
-### References:
  - https://github.com/ethereumjs/keythereum
- - https://github.com/ethereumjs/ethereumjs-wallet/blob/master/docs/classes/wallet.md
- - https://github.com/multiversx/mx-sdk-js-wallet/blob/main/src/userWallet.ts
+ - https://github.com/ethereumjs/ethereumjs-wallet/blob/v1.0.2/docs/classes/wallet.md
+ - https://github.com/multiversx/mx-sdk-js-wallet/blob/v4.2.1/src/userWallet.ts
  - https://github.com/multiversx/mx-sdk-py-wallet/blob/main/multiversx_sdk_wallet/user_wallet.py

sdk-wallet/controllers/ledger_wallet_controller.md

@@ -0,0 +1,35 @@
+## LedgerWalletController
+
+Note: though the Ledger device supports `account_index`, as well, this is not used in the context of MultiversX. For all purposes, `account_index` is `0`, while `address_index` is allowed to vary.
+
+References (not implementation suggestions):
+- https://github.com/multiversx/mx-sdk-js-hw-provider/blob/v6.4.0/src/ledgerApp.ts
+- https://github.com/multiversx/mx-sdk-py-cli/blob/main/multiversx_sdk_cli/ledger/ledger_app_handler.py
+
+```
+class LedgerWalletController:
+    // The constructor is not strictly captured by the specs; it's up to the implementing library to define it.
+    // Generally speaking, the constructor would take as input: a "IKeysComputer", an "IAddressComputer".
+
+    // Returns the configuration of the MultiversX App.
+    get_configuration(): LedgerAppConfiguration;
+
+    get_address(address_index: number): Address;
+
+    get_current_address(): Address;
+
+    select_current_address(address_index: number);
+
+    // Part of the Native Authentication flow.
+    sign_auth_token(auth_token: bytes): bytes;
+
+    sign_transaction(serialized_transaction: bytes): bytes;
+    sign_message(serialized_message: bytes): bytes;
+```
+
+```
+dto LedgerAppConfiguration:
+    is_transaction_data_allowed: boolean;
+    address_index: number;
+    version: string;
+```

sdk-wallet/controllers/mnemonic_wallet_controller.md

@@ -0,0 +1,27 @@
+## MnemonicWalletController
+
+```
+class MnemonicWalletController implements IWalletController:
+    // The constructor is not strictly captured by the specs; it's up to the implementing library to define it.
+    // Generally speaking, the constructor would take as input: a "Mnemonic", an "IMnemonicComputer", a "IKeysComputer", an "IAddressComputer".
+    // Generally speaking, upon construction, the selected address would be the one at index 0.
+
+    // Generally speaking, this would use the underlying mnemonic computer to derive the secret key, 
+    // then compute the public key and, ultimately, the address. 
+    get_address(address_index: number): Address;
+
+    get_current_address(): Address;
+
+    // Generally speaking, this would use the underlying mnemonic computer to derive the secret key.
+    select_current_address(address_index: number);
+
+    // Part of the Native Authentication flow.
+    sign_auth_token(auth_token: bytes): bytes;
+
+    // Kept separate from "sign_message", for clarity, and also because some wallet controllers require separate signing flows (e.g. Ledger).
+    sign_transaction(serialized_transaction: bytes): bytes;
+    // Kept separate from "sign_transaction", for clarity, and also because some wallet controllers require separate signing flows (e.g. Ledger).
+    sign_message(serialized_message: bytes): bytes;
+```
+
+**Note:** mnemonics cannot be retrieved from the wallet controller - at least, not directly; this use case is not supported by the specs. Wallet-like applications are responsible to enable support for retrieving the mnemonic, if they wish to do so (e.g. for display).

sdk-wallet/controllers/secret_key_wallet_controller.md

@@ -0,0 +1,31 @@
+## SecretKeyWalletController
+
+```
+class SecretKeyWalletController implements IWalletController:
+    // The constructor is not strictly captured by the specs; it's up to the implementing library to define it.
+    // Generally speaking, the constructor would take as input: an "ISecretKey", a "IKeysComputer", an "IAddressComputer".
+    // If desired, the implementing library can support more than one secret key, but this is not required.
+    // Generally speaking, upon construction, the selected address would be the one at index 0.
+
+    // Generally speaking, this throws if "address_index" is not 0. Though, implementations can choose to support more than one secret key, if desired.
+    // Can throw:
+    // - ErrAddressNotAvailable
+    get_address(address_index: number): Address;
+
+    get_current_address(): Address;
+
+    // Generally speaking, this throws if "address_index" is not 0. Though, implementations can choose to support more than one secret key, if desired.
+    // Can throw:
+    // - ErrAddressNotAvailable
+    select_current_address(address_index: number);
+
+    // Part of the Native Authentication flow.
+    sign_auth_token(auth_token: bytes): bytes;
+
+    // Kept separate from "sign_message", for clarity, and also because some wallet controllers require separate signing flows (e.g. Ledger).
+    sign_transaction(serialized_transaction: bytes): bytes;
+    // Kept separate from "sign_transaction", for clarity, and also because some wallet controllers require separate signing flows (e.g. Ledger).
+    sign_message(serialized_message: bytes): bytes;
+```
+
+**Note:** secret keys cannot be retrieved from the wallet controller - at least, not directly; this use case is not supported by the specs. Wallet-like applications are responsible to enable support for retrieving the secret key, if they wish to do so (e.g. for display).

sdk-wallet/cookbook.md

@@ -0,0 +1,94 @@
+
+## Cookbook (examples of usage)
+
+Note: error handling is out of scope for this cookbook (omitted for brevity).
+
+Generate a mnemonic and derive the first secret key:
+
+```
+provider = new UserWalletProvider()
+mnemonic = provider.generate_mnemonic()
+print(mnemonic.toString())
+
+sk = provider.derive_secret_key_from_mnemonic(mnemonic, address_index=0 , passphrase="")
+pk = provider.compute_public_key_from_secret_key(sk)
+address = new Address(pk, "erd")
+print(address.bech32())
+```
+
+Generate a secret key and export it to a PEM keystore:
+
+```
+provider = new UserWalletProvider()
+sk, _ = provider.generate_keypair()
+keystore = PEMKeystore.new_from_secret_key(provider, sk)
+keystore.export_to_file("wallet.pem", "erd")
+```
+
+Generate a mnemonic and export it to a JSON keystore:
+
+```
+provider = new UserWalletProvider()
+mnemonic = provider.generate_mnemonic()
+keystore = EncryptedKeystore.new_from_mnemonic(provider, mnemonic)
+keystore.export_to_file("wallet.json", "password", "erd")
+```
+
+Load a JSON keystore which holds an encrypted mnemonic, then iterate over the first 3 accounts:
+
+```
+provider = new UserWalletProvider()
+keystore = EncryptedKeystore.import_from_file(provider, "wallet.json", "password")
+mnemonic = keystore.get_mnemonic()
+
+for i in [0, 1, 2]:
+    sk = provider.derive_secret_key_from_mnemonic(mnemonic, address_index=i, passphrase="")
+    pk = provider.compute_public_key_from_secret_key(sk)
+    address = new Address(pk, "erd")
+    print("Address", i, address.bech32())
+```
+
+Load a PEM keystore, then iterate over the first 3 accounts:
+
+```
+provider = new UserWalletProvider()
+keystore = PEMKeystore.import_from_file(provider, "wallet.pem")
+
+for i in [0, 1, 2]:
+    sk = keystore.get_secret_key(i)
+    pk = provider.compute_public_key_from_secret_key(sk)
+    address = new Address(pk, "erd")
+    print("Address", i, address.bech32())
+```
+
+Change the password of a JSON keystore:
+
+```
+provider = new UserWalletProvider()
+keystore = EncryptedKeystore.import_from_file(provider, "wallet.json", "password")
+keystore.export_to_file("wallet.json", "new_password", "erd")
+```
+
+Load a PEM keystore and sign a piece of data:
+
+```
+provider = new UserWalletProvider()
+keystore = PEMKeystore.import_from_file(provider, "wallet.pem")
+sk = keystore.get_secret_key(0)
+signature = provider.sign(data, sk)
+```
+
+Load a JSON keystore and sign data:
+
+```
+provider = new UserWalletProvider()
+keystore = EncryptedKeystore.import_from_file(provider, "wallet.json", "password")
+
+if keystore.get_kind() == "secretKey":
+    sk = keystore.get_secret_key()
+else:
+    mnemonic = keystore.get_mnemonic()
+    sk = provider.derive_secret_key_from_mnemonic(mnemonic, address_index=0, passphrase="")
+
+signature = provider.sign(data, sk)
+```

sdk-wallet/crypto/keypair_based_encryptor_decryptor.md

@@ -1,5 +1,12 @@
 ## KeyPairBasedEncryptorDecryptor
 
+Optional, can be omitted by implementing libraries.
+
+Implementation suggestions:
+ - https://github.com/multiversx/mx-sdk-js-wallet/blob/v4.2.1/src/crypto/pubkeyEncryptor.ts
+ - https://github.com/multiversx/mx-sdk-js-wallet/blob/main/src/crypto/pubkeyDecryptor.ts
+ - 
+
 ```
 class KeyPairBasedEncryptorDecryptor:
     encrypt(data: bytes, recipient_public_key: IPublicKey, auth_secret_key: ISecretKey): PublicKeyEncryptedData;

sdk-wallet/crypto/password_based_encryptor_decryptor.md

@@ -1,5 +1,11 @@
 ## PasswordBasedEncryptorDecryptor
 
+Implementation suggestions:
+ - https://github.com/multiversx/mx-sdk-py-wallet/blob/v0.8.3/multiversx_sdk_wallet/crypto/encryptor.py
+ - https://github.com/multiversx/mx-sdk-py-wallet/blob/v0.8.3/multiversx_sdk_wallet/crypto/decryptor.py
+ - https://github.com/multiversx/mx-sdk-js-wallet/blob/v4.2.1/src/crypto/encryptor.ts
+ - https://github.com/multiversx/mx-sdk-js-wallet/blob/v4.2.1/src/crypto/decryptor.ts
+
 ```
 class PasswordBasedEncryptorDecryptor:
     encrypt(data: bytes, password: string): PasswordEncryptedData;

sdk-wallet/crypto/password_encrypted_data.md

@@ -1,5 +1,14 @@
 ## PasswordEncryptedData
 
+Implementation suggestions:
+  - `EncryptedData`: https://github.com/multiversx/mx-sdk-js-wallet/blob/v4.2.1/src/crypto/encryptedData.ts
+  - `EncryptedData`: https://github.com/multiversx/mx-sdk-py-wallet/blob/v0.8.3/multiversx_sdk_wallet/crypto/encrypted_data.py
+
+**Note:** this DTO is not a representation of the JSON keystore itself. Instead, it defines a data structure that is **agnostic to the type of file it might be stored in**. This abstraction was introduced July 2021:
+ - https://github.com/multiversx/mx-sdk-js-core/pull/11
+
+In practice, such a data structure is, indeed, stored in a JSON keystore file, whose schema does resemble the DTO below - but it's not identical. Any similarities between `PasswordEncryptedData` and `EncryptedKeystoreContent` **should not be considered duplication**.
+
 ```
 dto PasswordEncryptedData:
     id: string;
@@ -13,7 +22,7 @@ dto PasswordEncryptedData:
     mac: string;
 
 // Example of class compatible with "PasswordEncryptedData.kdfparams".
-// Can be anything, and it's specific to a "PasswordEncryptedData.kdf" (e.g. "scrypt").
+// Can be anything, and it's correlated to a "PasswordEncryptedData.kdf" (e.g. "scrypt").
 // EncryptorDecryptor.encrypt() puts this into "PasswordEncryptedData.kdfparams".
 // EncryptorDecryptor.decrypt() interprets (possibly validates) it.
 dto ScryptKeyDerivationParams:

sdk-wallet/crypto/public_key_encrypted_data.md

@@ -1,5 +1,10 @@
 ## PublicKeyEncryptedData
 
+Optional, can be omitted by implementing libraries.
+
+Implementation suggestions:
+ - https://github.com/multiversx/mx-sdk-js-wallet/blob/main/src/crypto/x25519EncryptedData.ts
+
 ```
 dto PublicKeyEncryptedData:
     nonce: string;

sdk-wallet/interfaces.md

@@ -5,12 +5,30 @@ For languages that support **structural typing** (e.g. Go, Python, TypeScript),
 For languages that only support **nominal typing** (e.g. C#), these interfaces can be _exported_.
 
 ```
-interface IWalletProvider:
-    generate_keypair(): (ISecretKey, IPublicKey)
+interface IWalletController: 
+    get_address(address_index: number): Address;
+    get_current_address(): Address;
+    select_current_address(address_index: number);
+    sign_auth_token(auth_token: bytes): bytes;
+    sign_transaction(serialized_transaction: bytes): bytes;
+    sign_message(serialized_message: bytes): bytes;
+```
+
+```
+interface ISigner:
     sign(data: bytes, secret_key: ISecretKey): bytes
+```
+
+```
+interface IVerifier:
     verify(data: bytes, signature: bytes, public_key: IPublicKey): bool
-    create_secret_key_from_bytes(data: bytes): ISecretKey
-    create_public_key_from_bytes(data: bytes): IPublicKey
+```
+
+```
+interface IKeysComputer:
+    generate_keypair(): (ISecretKey, IPublicKey)
+    compute_secret_key_from_bytes(data: bytes): ISecretKey
+    compute_public_key_from_bytes(data: bytes): IPublicKey
     compute_public_key_from_secret_key(secret_key: ISecretKey): IPublicKey
 ```