Skip to content

Commit

Permalink
Rust Wallet (#39)
Browse files Browse the repository at this point in the history
* adds documentation

* adds tests

Signed-off-by: Gus <[email protected]>

* feat: mnemonic & hotkey pair

* feat: wallet & hotkey file write

* fmt

* clean up and todos

* testing sign

* docs and signing working

* password working - needs cleanup

* adds coldkey gen

* clean up

* wallet integration with python wip

* encrypt using secret box access

* Test signed commit

* wip restructure

* update structure

* update python package folder name

* working

* update structure

* fix Keypair `#pyclass` macros

* delete pyproject.toml

* update structure

* convert public_key to bytes

* convert private_key to bytes

* empty struct Keypair and getters

* impl Default for Keypair

* add pair

* add getters ss58_address and seed_hex

* all fields in Keypair work

* add __repr__ and __str__

* Minimum functionality done

* add print for test

* update README with a test for Keypair

* fix ci

* add creating from public_key

* update manual test script

* remove comments from public_key getter

* add creation with ss58_address

* add the checker `If public_key is missing (ss58_address wasn't created), return an error`

* add sign function + manual test case

* BROKEN: Add SS58 address validation utility

This commit introduces a new utility module for SS58 address validation. It includes a function `is_valid_ss58_address` and integrates it into the main library module. Additionally, a test for the validation function is provided.

* add verify function + manual test case

* fix utils::is_valid_ss58_address function

* fix utils::is_valid_ss58_address function + test

* add utils::get_ss58_format + test

* keep support crypto_type 1 only (sr25519)

* update README.md

* utils done

* update README.md

* add funcs to the pymodule in the lib.rs

* update Cargo.toml

* GREEEEN BUILD

* cargo fmt

* cargo clippy GREEN

* Add comments with keyfile.py usage across the wallet, btcli, sdk

* frm

* cargo files update

* add Keyfile.rs structure

* add Keyfile.rs structure

* add Keyfile::exists_on_device

* Add keyfile-related functions to library

Integrated multiple keyfile-related Python functions into the library, including serialization, validation, and encryption functionalities. These functions currently contain unimplemented placeholders, awaiting further development.

* add Keyfile::make_dirs

* add Keyfile::_may_overwrite

* add Keyfile::_write_keyfile_data_to_file

* add Keyfile::exists_on_device update

* add Keyfile::is_readable

* add Keyfile::_read_keyfile_data_from_file, Keyfile::is_writable

* Enable keyfile serialization and password validation

This commit implements functions to serialize/deserialize keypair data to/from keyfiles in `src/keyfile.rs`, adds password strength validation, and introduces new dependencies for JSON handling and password analysis. Also, modifies visibility of `seed_hex` in `Keypair` to `pub(crate)`.

* add Keyfile::is_encrypted

* add TODOs

* update deps in Cargo.toml

* Refactor keyfile functions to use PyBytes instead of Vec<u8>

Replaced Vec<u8> with PyBytes in keyfile serialization and deserialization functions for better compatibility with Python. Updated corresponding internal logic to handle PyBytes and ensured consistent error handling throughout. Adjusted test cases to reflect these changes.

* add pyo3 - signature: green build

* formatter

* remove unused prefixes and pub sign

* update README.md -> tests

* update README.md -> tests

* Enable encryption detection functions

Uncomment and implement the functions to detect various encryption methods in keyfile data, including NaCl, Ansible, and legacy formats. Also, update the logic for retrieving the coldkey password from environment variables.

* fix input logic and messages

* add keyfile::keyfile_data_encryption_method

* Add legacy keyfile encryption function

Integrate ansible_vault for legacy keyfile encryption. Implement the legacy_encrypt_keyfile_data function and add necessary dependencies to Cargo.toml.

* add keyfile::encrypt_keyfile_data

* update docstring

* update docstring + add pyo3 signature

* Update error handling and getter methods in keyfile.rs

Replaced `PyException` with more specific `pyo3::exceptions::PyUnicodeDecodeError` for error handling. Also, adjusted getter methods to return actual keyfile data by reading from the file instead of returning placeholder values.

* add keypair::decrypt_keyfile_data

* Refactor `set_keypair` to handle encryption and file writing

Consolidated encryption logic and streamlined file writing process in `set_keypair` function. Now, the keypair data is serialized and optionally encrypted, then written to file with overwrite handling. This improves code clarity and functionality.

* add keypair::Keypair.set_keypair, fix: keypair::encrypt_keyfile_data, add printing messages

* move nacl_decrypt and legacy_decrypt into decrypt_keyfile_data

* add keyfile::Keyfile.is_encrypted

* commented keypair getter

* add keypair::Keypair.get_keypair

* add keypair::Keypair.check_and_update_encryption

* Improve file encryption and decryption error handling

Added comprehensive checks to ensure the keyfile exists, is readable, and writable before performing encryption and decryption. Enhanced the encryption and decryption methods to handle unencrypted data properly and ensured the results are written back correctly.

* Refactor `decrypt` to handle errors with `PyResult`

Refactor the `decrypt` method to return `PyResult<()>` and improve error handling by replacing `panic!` calls with appropriate `PyOSError` returns. Also, streamline the decryption process and key pair deserialization for better readability and maintainability.

* Refactor keyfile encryption error handling

Convert panic calls to proper error handling using PyResult for better stability and clarity. Modify the encrypt function to return detailed Python exceptions when the keyfile does not exist, is not readable, or not writable. This change ensures a more robust and informative error management.

* activate GPG key

* add wallet::Wallet.create_if_non_existent

* update config

* fix keyfile.rs

* add list of functions into wallet.rs

* make keypair::Keypair.create_from_uri public

* add empty structure to wallet create, create some of the functions

* add some empty functions into wallet::Wallet

* fix cc compiler problem

* Refactor wallet initialization to use config defaults

* Add comprehensive tests for keyfile.rs

Expanded the README.md to include detailed tests for keyfile.rs. These tests cover serialization, encryption, password validation, environment password retrieval, file operations, and various Keyfile methods.

* add wallet::Wallet.regenerate_hotkey, wallet::Wallet.set_hotkey, wallet::Wallet.hotkey_file, update constants wallet path

* fix keypair::Keypair.mnemonic()

* add wallet::Wallet.regenerate_coldkeypub, coldkey_file, coldkeypub_file, set_coldkey, set_coldkeypub, get_coldkeypub, get_coldkey, get_hotkey, regenerate_coldkey

* cleanup

* fixes

* add keypair::Keypair.create_from_encrypted_json

* cls annotations fix

* deps cleanup

* import cleanup

* keypair::Keypair.create_from_encrypted_json

* temporarily remove incorrect private key usage in `create_from_encrypted_json`

* create correct python package structure

* add errors::KeyFileError, add its usage in the same way as in python wallet

* update python package structure

* README, refactoring

* README, refactoring

* README -> add test `create_from_encrypted_json`

* lib.rs -> add classes on the package top level

* Fix keypair::Keypair.create_from_encrypted_json

* update README (import logic)

* add #[pyclass(subclass)] ability for main classes, add mock subpackage

* improve keyfile::Keyfile.__str__()

* add export for wallet::display_mnemonic_msg

* fix classic import logic

* add errors::ConfigurationError

* improve keyfile::Keyfile.new logic

* add pyo3 signature to  keypair::Keypair methods (functions)

* add error into the module

* add bittensor_wallet.keypair submodule

* update ConfigurationError usage for Keypair::new

* update errors

* update types of errors in keypair

* update types of errors in keypair

* can't mock functions that are not implemented yet (rust doesn't accept this way). This limits testing from Python. temporarily commenting on mocking tests.

* add `rpassword::prompt_password` instead of input for asking password.

* Update version to 2.0.0

Upgrade the package version from 1.0.0 to 2.0.0 in Cargo.toml. This indicates significant changes, potentially including new features, improvements, or breaking changes. Ensure compatibility with the intended systems and dependencies.

* use direct import for errors types in keyfile.rs

* update .gitignore

* use direct import for error types in keypair.rs

* use direct import for error types in wallet.rs

* use direct import for error types in utils.rs

* update for wallets properties

* add *args, **kwargs

* use new instead of init

* Refactor keypair decoding and wallet initialization

Improve `keypair.rs` by adding public key conversion assertion. Refactor `wallet_mock.py` to streamline mock wallet initialization. Simplify wallet constructor signature in `wallet.rs` by removing unused arguments.

* move assert into match arm

* add back extension-module feature

* add version

* unused imports

* fix create_from_seed (add union str, bytes) processing

* add ScaleCodec to keypair::Keypair.sign and keypair::Keypair.verify

* add tests to readme

* Add getter methods for wallet name, path, and hotkey

This commit introduces getter methods to retrieve the wallet's name, path, and hotkey string. These methods enhance the usability of the wallet by providing convenient access to these attributes. Each method returns a cloned value of the respective attribute.

* use pythons stdout (#30)

* Add getters for name and path in Keyfile

Implement getter methods for retrieving the name and path from the keyfile object. This enhances the accessibility of these attributes, allowing for more efficient data manipulation and retrieval within the module.

* Updated readme docs

* add prompt and prompt password

* chore: clippy

* Adds extra line after print

* try eval bound with import first

* Add wallet::Wallet.add_args

* update config::Config -> add getters

* Refactor error handling in keyfile.rs

Replaced the usage of `PyUserWarning` with a custom `KeyFileError` for more specific error handling. This change promotes better clarity and consistency in error reporting within the `keyfile.rs` module.

* wrap with err

* remove import

* fix panic error to KeyFileError

* delete double import

* Fix typos in status messages

Corrected the phrasing of status messages in keyfile.rs from "Encryption data..." and "Decrypt data..." to "Encrypting data..." and "Decrypting data...". This enhances clarity and consistency in the user interface.

* wallet.coldkey.ss58_address doesn't ask typing pass after waller.unlock_coldkey()

* wallet.hotkey.ss58_address doesn't ask typing pass after waller.unlock_hotkey()

* Cam/no excepts (#32)

* Remove except usage

* use alias for now

* use different constructor format

* fix log

* remove extra import

* fix keypair.rs errors

* chore: ruff

* chore: clippy

* chore: fmt

* chore: ruff

* use pypi trusted release

* move `keypair::Keypair.private_key` outside the python scope

* Add re-trying to type the password after wallet creation

* update tests

* remove utils

* Feat/ci/add release (#38)

* replace with pyproject.toml

* fix release for manylinux builds

* rename package and file

* fix classifiers

* remove btwallet dir

* make non pre-release

* try ignoring tests dir

* try with maturin config

* add bip39

* test with more platforms

* remove other platforms

* comment out other platforms

* Bumps to a5

* Bumps to 2.0.0

* Updates Changelog

---------

Signed-off-by: Gus <[email protected]>
Co-authored-by: Gus <[email protected]>
Co-authored-by: Roman <[email protected]>
Co-authored-by: Cameron Fairchild <[email protected]>
Co-authored-by: Cameron Fairchild <[email protected]>
Co-authored-by: Watchmaker <[email protected]>
Co-authored-by: ibraheem-opentensor <[email protected]>
Co-authored-by: Roman <[email protected]>
  • Loading branch information
8 people authored Oct 4, 2024
1 parent 8491a77 commit 94a0f67
Show file tree
Hide file tree
Showing 34 changed files with 7,030 additions and 2,952 deletions.
197 changes: 140 additions & 57 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -1,72 +1,155 @@
name: Build and Publish Python bittensor-wallet
# This file is autogenerated by maturin v1.7.4
# To update, run
#
# maturin generate-ci github
#
name: Build wheels

on:
push:
branches:
- main
tags:
- '*'
pull_request:
workflow_dispatch:
inputs:
version:
description: 'Version to release'
required: true
type: string

permissions:
contents: read

jobs:
build:
name: Build Python distribution
runs-on: ubuntu-latest
linux:
runs-on: ${{ matrix.platform.runner }}
strategy:
matrix:
platform:
- runner: ubuntu-latest
target: x86_64
- runner: ubuntu-latest
target: x86
# - runner: ubuntu-latest
# target: aarch64
# - runner: ubuntu-latest
# target: armv7
# - runner: ubuntu-latest
# target: s390x
# - runner: ubuntu-latest
# target: ppc64le
steps:
- uses: actions/checkout@v3

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.10'
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: 3.x
- name: Build wheels
uses: PyO3/[email protected]
with:
target: ${{ matrix.platform.target }}
args: --release --out dist --find-interpreter
sccache: 'true'
manylinux: auto
before-script-linux: (apt-get update && apt-get install -y apt-utils && apt-get install -y pkg-config libssl-dev) || (yum install openssl openssl-devel -y)
- name: Upload wheels
uses: actions/upload-artifact@v4
with:
name: wheels-linux-${{ matrix.platform.target }}
path: dist

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install build wheel twine
# musllinux:
# runs-on: ${{ matrix.platform.runner }}
# strategy:
# matrix:
# platform:
# - runner: ubuntu-latest
# target: x86_64
# # - runner: ubuntu-latest
# # target: x86
# # - runner: ubuntu-latest
# # target: aarch64
# # - runner: ubuntu-latest
# # target: armv7
# steps:
# - uses: actions/checkout@v4
# - uses: actions/setup-python@v5
# with:
# python-version: 3.x
# - name: Build wheels
# uses: PyO3/[email protected]
# with:
# target: ${{ matrix.platform.target }}
# args: --release --out dist --find-interpreter
# sccache: 'true'
# manylinux: musllinux_1_2
# before-script-linux: (apt-get update && apt-get install -y apt-utils && apt-get install -y pkg-config libssl-dev) || (yum install openssl openssl-devel -y)
# - name: Upload wheels
# uses: actions/upload-artifact@v4
# with:
# name: wheels-musllinux-${{ matrix.platform.target }}
# path: dist

- name: Build package
run: python setup.py sdist bdist_wheel

- name: Check if package version already exists
run: |
PACKAGE_NAME=$(python setup.py --name)
PACKAGE_VERSION=${{ github.event.inputs.version }}
if twine check dist/*; then
if pip install $PACKAGE_NAME==$PACKAGE_VERSION; then
echo "Error: Version $PACKAGE_VERSION of $PACKAGE_NAME already exists on PyPI"
exit 1
else
echo "Version $PACKAGE_VERSION of $PACKAGE_NAME does not exist on PyPI. Proceeding with upload."
fi
else
echo "Error: Twine check failed."
exit 1
fi
macos:
runs-on: ${{ matrix.platform.runner }}
strategy:
matrix:
platform:
- runner: macos-12
target: x86_64
- runner: macos-14
target: aarch64
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: 3.x
- name: Build wheels
uses: PyO3/[email protected]
with:
target: ${{ matrix.platform.target }}
args: --release --out dist --find-interpreter
sccache: 'true'
- name: Upload wheels
uses: actions/upload-artifact@v4
with:
name: wheels-macos-${{ matrix.platform.target }}
path: dist

- name: Upload artifact
uses: actions/upload-artifact@v3
with:
name: dist
path: dist/
sdist:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Build sdist
uses: PyO3/[email protected]
with:
command: sdist
args: --out dist
- name: Upload sdist
uses: actions/upload-artifact@v4
with:
name: wheels-sdist
path: dist

approve-and-publish:
needs: build
release:
name: Release
runs-on: ubuntu-latest
environment: release
if: ${{ startsWith(github.ref, 'refs/tags/') || github.event_name == 'workflow_dispatch' }}
#needs: [linux, macos, musllinux, sdist]
needs: [linux, macos, sdist]
permissions:
contents: read
# Use to sign the release artifacts
id-token: write

# Used to upload release artifacts
contents: write
# Used to generate artifact attestation
attestations: write
steps:
- name: Download artifact
uses: actions/download-artifact@v3
with:
name: dist
path: dist/

- name: Publish package distributions to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
verbose: true
print-hash: true
- uses: actions/download-artifact@v4
- name: Generate artifact attestation
uses: actions/attest-build-provenance@v1
with:
subject-path: 'wheels-*/*'
- name: Publish to PyPI
if: "startsWith(github.ref, 'refs/tags/')"
uses: PyO3/maturin-action@v1.44.0
with:
command: upload
args: --non-interactive --skip-existing wheels-*/*
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -213,4 +213,6 @@ replicate.yaml
# Notebooks
*.ipynb

tests/zombienet/bin/**/*
tests/zombienet/bin/**/*

Cargo.lock
34 changes: 34 additions & 0 deletions CHANGELOG.MD
Original file line number Diff line number Diff line change
@@ -1,5 +1,39 @@
# Changelog

## 2.0.0 /2024-10-03

## What's Changed
* Massive wallet change after copy-paste by @roman-opentensor in https://github.com/opentensor/btwallet/pull/1
* Bump substrate-interface to 1.7.9 by @thewhaleking in https://github.com/opentensor/btwallet/pull/3
* Fix flaky tests by @roman-opentensor in https://github.com/opentensor/btwallet/pull/5
* Initial commit: circleci testing config by @ibraheem-opentensor in https://github.com/opentensor/btwallet/pull/6
* bug/roman/requirements by @roman-opentensor in https://github.com/opentensor/btwallet/pull/8
* License fix by @roman-opentensor in https://github.com/opentensor/btwallet/pull/9
* Add unlock method for wallet coldkey by @thewhaleking in https://github.com/opentensor/btwallet/pull/10
* Add `unlock_hotkey` method to the wallet. Add unit tests. by @roman-opentensor in https://github.com/opentensor/btwallet/pull/11
* Add `unlock_hotkeypub` method to the wallet. Add unit tests. by @roman-opentensor in https://github.com/opentensor/btwallet/pull/12
* Updates mnemonic command output by @ibraheem-opentensor in https://github.com/opentensor/btwallet/pull/17
* Raises a KeyFileError when one is encountered when unlocking. by @thewhaleking in https://github.com/opentensor/btwallet/pull/19
* bump up the version by @roman-opentensor in https://github.com/opentensor/btwallet/pull/20
* Password unlocking changes by @thewhaleking in https://github.com/opentensor/btwallet/pull/18
* bump up cryptography version until 43.0.1 by @roman-opentensor in https://github.com/opentensor/btwallet/pull/21
* Deployment script + updates by @ibraheem-opentensor in https://github.com/opentensor/btwallet/pull/23
* Release 1.0.0 - Changelog by @ibraheem-opentensor in https://github.com/opentensor/btwallet/pull/24
* Pins py-bip39-bindings by @ibraheem-opentensor in https://github.com/opentensor/btwallet/pull/25
* use pythons stdout by @camfairchild in https://github.com/opentensor/btwallet/pull/30
* Cam/no excepts by @camfairchild in https://github.com/opentensor/btwallet/pull/32
* Updated readme docs by @rajkaramchedu in https://github.com/opentensor/btwallet/pull/31
* Feat/ci/add release by @camfairchild in https://github.com/opentensor/btwallet/pull/38

## New Contributors
* @roman-opentensor made their first contribution in https://github.com/opentensor/btwallet/pull/1
* @thewhaleking made their first contribution in https://github.com/opentensor/btwallet/pull/3
* @ibraheem-opentensor made their first contribution in https://github.com/opentensor/btwallet/pull/6
* @camfairchild made their first contribution in https://github.com/opentensor/btwallet/pull/30
* @rajkaramchedu made their first contribution in https://github.com/opentensor/btwallet/pull/31

**Full Changelog**: https://github.com/opentensor/btwallet/commits/v2.0.0

## 1.0.0 /2024-09-25

## What's Changed
Expand Down
Loading

0 comments on commit 94a0f67

Please sign in to comment.