Skip to content

Commit

Permalink
Merge branch 'staging' into feat/roman/substrate-definition-before-in…
Browse files Browse the repository at this point in the history
…itialization

# Conflicts:
#	bittensor/core/subtensor.py
#	requirements/btcli.txt
#	requirements/prod.txt
#	setup.py
#	tests/e2e_tests/conftest.py
#	tests/e2e_tests/test_axon.py
#	tests/e2e_tests/test_commit_weights.py
#	tests/e2e_tests/test_dendrite.py
#	tests/e2e_tests/test_incentive.py
#	tests/e2e_tests/test_liquid_alpha.py
#	tests/e2e_tests/test_metagraph.py
#	tests/e2e_tests/test_subtensor_functions.py
#	tests/e2e_tests/test_transfer.py
  • Loading branch information
roman-opentensor committed Oct 1, 2024
2 parents bd8f8f5 + bff05c0 commit 51c03bd
Show file tree
Hide file tree
Showing 16 changed files with 222 additions and 31 deletions.
127 changes: 127 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,132 @@
# Changelog

## 8.0.0 /2024-09-25

## What's Changed

Removes Bittensor CLI and Wallet functionalities and changes the Bittensor SDK package to be light while maintaining backwards compatibility

* Update README.md by @rajkaramchedu in https://github.com/opentensor/bittensor/pull/2320
* remove unused code (tensor.py-> class tensor), remove old tests, add new tests by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2311
* Updating/improving/creating docstring codebase by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2310
* README updates for SDK by @rajkaramchedu in https://github.com/opentensor/bittensor/pull/2309
* Improved logic for concatenating message, prefix, and suffix in bittensor logging + test by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2306
* BTSDK: Implementation of substrait custom errors handler for bittensor by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2305
* btsdk cleanup by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2303
* Fix mypy error for btlogging by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2299
* Integrate `bt_decode` into BTSDK by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2298
* BTSDK: Corrected arguments order in logging methods + test by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2292
* BTSDK: removed exit sys call for ConnectionRefusedError in _get_substrate by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2288
* BTSDK: Move `do*` methods to related extrinsic by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2286
* add reconnection logic for correctly closed connection by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2283
* Move extrinsics, update `deprecated.py` module. by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2278
* Add substrate reconnection logic by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2269
* Prod requirements cleanup by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2266
* Decoupling chain_data.py to sub-package by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2264
* Increase Bittensor SDK test coverage by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2262
* Increase SDK test coverage (Part3) by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2257
* Increase bittensor SDK test coverage by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2256
* Increase test coverage for subtensor.py by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2252
* Adds e2e and fixes metagraph save()/load() by @ibraheem-opentensor in https://github.com/opentensor/bittensor/pull/2231
* feat/roman/reafctoring-before-test-coverage by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2230
* Enhance: Switch from format() to f-strings by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2228
* Commit-reveal re-added & e2e coverage by @ibraheem-opentensor in https://github.com/opentensor/bittensor/pull/2224
* Adds e2e setup & tests by @ibraheem-opentensor in https://github.com/opentensor/bittensor/pull/2221
* Updates after review session by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2220
* Fix the usage of env vars in default settings. by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2218
* Add dendrite reference to backwords compatibility by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2217
* Bringing `btsdk` up-to-date with `staging` branch. by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2210
* Part 3: Create new 'bittensor-sdk` package by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2206
* Part 2: Redesign, fix namespace conflicts, remove btcli by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2204
* Part1: Removing content related to the wallet. Start use the pip installable package. by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2191

**Full Changelog**: https://github.com/opentensor/bittensor/compare/v7.4.0...v8.0.0

## 7.4.0 /2024-08-29

## What's Changed
* [Fix] Allow unstake below network min by @camfairchild in https://github.com/opentensor/bittensor/pull/2016
* Tests/e2e tests staging by @open-junius in https://github.com/opentensor/bittensor/pull/1943
* Chore: Backmerge 7.2 by @gus-opentensor in https://github.com/opentensor/bittensor/pull/2020
* Fix broken tests and Enforce BTCLI usage by @opendansor in https://github.com/opentensor/bittensor/pull/2027
* Add time delay to faucet by @opendansor in https://github.com/opentensor/bittensor/pull/2030
* Skip faucet test by @opendansor in https://github.com/opentensor/bittensor/pull/2031
* Adds normalization for alpha hyperparams by @ibraheem-opentensor in https://github.com/opentensor/bittensor/pull/2035
* Revert info logging in processing response by @ibraheem-opentensor in https://github.com/opentensor/bittensor/pull/2043
* Pin numpy version to 1.26.4 in prod.txt by @rajkaramchedu in https://github.com/opentensor/bittensor/pull/2045
* Test hot key Swap by @opendansor in https://github.com/opentensor/bittensor/pull/2044
* Do not run Circle-CI on drafts by @thewhaleking in https://github.com/opentensor/bittensor/pull/1959
* Enhancement: Detailed nonce information in-case of failures by @ibraheem-opentensor in https://github.com/opentensor/bittensor/pull/2050
* fix bittensor not installing under Python 3.13 by @mjurbanski-reef in https://github.com/opentensor/bittensor/pull/2053
* Enable Faucet Test by @opendansor in https://github.com/opentensor/bittensor/pull/2056
* Add back BT_SUBTENSOR_CHAIN_ENDPOINT env variable by @bradleytf in https://github.com/opentensor/bittensor/pull/2034
* Fix: Logging configs not being set by @ibraheem-opentensor in https://github.com/opentensor/bittensor/pull/2065
* Feature/gus/liquid alpha params by @gus-opentensor in https://github.com/opentensor/bittensor/pull/2012
* Test Emissions E2E by @opendansor in https://github.com/opentensor/bittensor/pull/2036
* Prevent e2e draft by @opendansor in https://github.com/opentensor/bittensor/pull/2072
* Fix e2e to only run when PR is ready for review by @opendansor in https://github.com/opentensor/bittensor/pull/2077
* Fix Faucet and fastblocks interaction by @opendansor in https://github.com/opentensor/bittensor/pull/2083
* Float normalization for child hotkeys by @opendansor in https://github.com/opentensor/bittensor/pull/2093
* Fix e2e test hanging by @open-junius in https://github.com/opentensor/bittensor/pull/2118
* Fixes leaked semaphores by @thewhaleking in https://github.com/opentensor/bittensor/pull/2125
* Backmerge master -> staging by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2136
* fix: coldkeypub usage instead of coldkey for arbitration_stats by @Rapiiidooo in https://github.com/opentensor/bittensor/pull/2132
* Removes extra no_prompts in commands by @ibraheem-opentensor in https://github.com/opentensor/bittensor/pull/2140
* Adds timeout for e2e tests by @ibraheem-opentensor in https://github.com/opentensor/bittensor/pull/2141
* fix: updates test_axon verify body async tests by @gus-opentensor in https://github.com/opentensor/bittensor/pull/2142
* test: fix mocksubtensor query previous blocks by @timabilov in https://github.com/opentensor/bittensor/pull/2139
* Adds E2E for Metagraph command by @ibraheem-opentensor in https://github.com/opentensor/bittensor/pull/2143
* feat: Enhance dendrite error messaging by @gus-opentensor in https://github.com/opentensor/bittensor/pull/2117
* Adds E2E Tests for wallet creation commands by @ibraheem-opentensor in https://github.com/opentensor/bittensor/pull/2145
* [Ledger Integration] [Feature] bump pysub to 1.7.9+ by @camfairchild in https://github.com/opentensor/bittensor/pull/2156
* Ruff complains about an extra line by @roman-opentensor in https://github.com/opentensor/bittensor/pull/2158
* support Wallet names with hyphens when passing password through ENV vars by @mjurbanski-reef in https://github.com/opentensor/bittensor/pull/1949
* Fix naming convention of swap hotkey test by @ibraheem-opentensor in https://github.com/opentensor/bittensor/pull/2162
* Adds E2E test for wallet regenerations + fixes input bug for regen hotkey by @ibraheem-opentensor in https://github.com/opentensor/bittensor/pull/2149
* Backmerge Master -> Staging (7.4) by @ibraheem-opentensor in https://github.com/opentensor/bittensor/pull/2170
* ci: auto assigns cortex to opened PRs by @gus-opentensor in https://github.com/opentensor/bittensor/pull/2184
* CI/E2E test improvements by @mvds00 in https://github.com/opentensor/bittensor/pull/2168
* Fix multiprocessing POW errors and No Torch logging errors by @thewhaleking in https://github.com/opentensor/bittensor/pull/2186
* ci: update reviewers by @gus-opentensor in https://github.com/opentensor/bittensor/pull/2189
* Adds updated type in timeouts dendrite by @ibraheem-opentensor in https://github.com/opentensor/bittensor/pull/2196
* Bumps setuptools ~=70.0.0 by @ibraheem-opentensor in https://github.com/opentensor/bittensor/pull/2150
* Bump black from 23.7.0 to 24.3.0 in /requirements by @dependabot in https://github.com/opentensor/bittensor/pull/2197
* btlogging/loggingmachine.py: Fix bw compat API. by @mvds00 in https://github.com/opentensor/bittensor/pull/2155
* Check for participation before nomination call by @ibraheem-opentensor in https://github.com/opentensor/bittensor/pull/2193
* test: subnet list e2e by @gus-opentensor in https://github.com/opentensor/bittensor/pull/2198
* ensure msg is str in _concat_msg by @thewhaleking in https://github.com/opentensor/bittensor/pull/2200
* Fixes tests depending on explicit line numbers by @ibraheem-opentensor in https://github.com/opentensor/bittensor/pull/2211
* Merge streaming fix to staging by @ibraheem-opentensor in https://github.com/opentensor/bittensor/pull/2183
* Multiple bittensor versions e2e workflow by @ibraheem-opentensor in https://github.com/opentensor/bittensor/pull/2212
* Changes name of workflow file by @ibraheem-opentensor in https://github.com/opentensor/bittensor/pull/2213
* Enhances e2e tests to contain assertions & logging by @ibraheem-opentensor in https://github.com/opentensor/bittensor/pull/2192
* Security fix: Bumps ansible and certifi by @ibraheem-opentensor in https://github.com/opentensor/bittensor/pull/2214
* Wallet List Command e2e test by @gus-opentensor in https://github.com/opentensor/bittensor/pull/2207
* fix Synapse base performance (more than 10x speed up) by @mjurbanski-reef in https://github.com/opentensor/bittensor/pull/2161
* Child Hotkeys by @opendansor in https://github.com/opentensor/bittensor/pull/2071
* Improve child hotkeys QOL by @opendansor in https://github.com/opentensor/bittensor/pull/2225
* Child hotkeys handle excess normalization by @opendansor in https://github.com/opentensor/bittensor/pull/2229
* Fixes chain compilation timeouts by @ibraheem-opentensor in https://github.com/opentensor/bittensor/pull/2238
* Update Child Hotkey commands by @opendansor in https://github.com/opentensor/bittensor/pull/2245
* feat: return error message instead of raising exception by @gus-opentensor in https://github.com/opentensor/bittensor/pull/2244
* Backmerge master to staging (7.3.1) by @ibraheem-opentensor in https://github.com/opentensor/bittensor/pull/2254

## New Contributors
* @bradleytf made their first contribution in https://github.com/opentensor/bittensor/pull/2034
* @Rapiiidooo made their first contribution in https://github.com/opentensor/bittensor/pull/2132
* @timabilov made their first contribution in https://github.com/opentensor/bittensor/pull/2139
* @mvds00 made their first contribution in https://github.com/opentensor/bittensor/pull/2168
* @dependabot made their first contribution in https://github.com/opentensor/bittensor/pull/2197

**Full Changelog**: https://github.com/opentensor/bittensor/compare/v7.3.1...v7.4.0

## 7.3.1 / 2024-08-19

## What's Changed
* https://github.com/opentensor/bittensor/pull/2156 by @camfairchild

**Full Changelog**: https://github.com/opentensor/bittensor/compare/v7.3.0...v7.3.1

## 7.3.0 / 2024-07-12

## What's Changed
Expand Down
2 changes: 1 addition & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
7.3.0
8.0.0
2 changes: 1 addition & 1 deletion requirements/btcli.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
git+https://github.com/opentensor/btcli.git@main#egg=bittensor-cli
bittensor-cli
2 changes: 1 addition & 1 deletion requirements/prod.txt
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,4 @@ python-Levenshtein
scalecodec==1.2.11
substrate-interface~=1.7.9
uvicorn
git+https://github.com/opentensor/btwallet.git#egg=bittensor-wallet
bittensor-wallet==1.0.0
18 changes: 0 additions & 18 deletions scripts/run.sh

This file was deleted.

1 change: 0 additions & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,6 @@ def read_requirements(path):
install_requires=requirements,
extras_require={
"btcli": extra_requirements_btcli,
"cubit": extra_requirements_cubit,
"dev": extra_requirements_dev,
"torch": extra_requirements_torch,
},
Expand Down
2 changes: 1 addition & 1 deletion tests/e2e_tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
from substrateinterface import SubstrateInterface

from bittensor import logging
from tests.e2e_tests.utils.test_utils import (
from tests.e2e_tests.utils.e2e_test_utils import (
clone_or_update_templates,
install_templates,
template_path,
Expand Down
2 changes: 1 addition & 1 deletion tests/e2e_tests/test_axon.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
from bittensor import logging
from bittensor.utils import networking
from tests.e2e_tests.utils.chain_interactions import register_neuron, register_subnet
from tests.e2e_tests.utils.test_utils import (
from tests.e2e_tests.utils.e2e_test_utils import (
setup_wallet,
template_path,
templates_repo,
Expand Down
2 changes: 1 addition & 1 deletion tests/e2e_tests/test_commit_weights.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
sudo_set_hyperparameter_values,
wait_interval,
)
from tests.e2e_tests.utils.test_utils import setup_wallet
from tests.e2e_tests.utils.e2e_test_utils import setup_wallet


@pytest.mark.asyncio
Expand Down
2 changes: 1 addition & 1 deletion tests/e2e_tests/test_dendrite.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
import bittensor
from bittensor import logging, Subtensor

from tests.e2e_tests.utils.test_utils import (
from tests.e2e_tests.utils.e2e_test_utils import (
setup_wallet,
template_path,
templates_repo,
Expand Down
2 changes: 1 addition & 1 deletion tests/e2e_tests/test_incentive.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
register_subnet,
wait_epoch,
)
from tests.e2e_tests.utils.test_utils import (
from tests.e2e_tests.utils.e2e_test_utils import (
setup_wallet,
template_path,
templates_repo,
Expand Down
2 changes: 1 addition & 1 deletion tests/e2e_tests/test_liquid_alpha.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
sudo_set_hyperparameter_bool,
sudo_set_hyperparameter_values,
)
from tests.e2e_tests.utils.test_utils import setup_wallet
from tests.e2e_tests.utils.e2e_test_utils import setup_wallet


def liquid_alpha_call_params(netuid: int, alpha_values: str):
Expand Down
2 changes: 1 addition & 1 deletion tests/e2e_tests/test_metagraph.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
register_neuron,
register_subnet,
)
from tests.e2e_tests.utils.test_utils import (
from tests.e2e_tests.utils.e2e_test_utils import (
setup_wallet,
)

Expand Down
2 changes: 1 addition & 1 deletion tests/e2e_tests/test_subtensor_functions.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
register_neuron,
register_subnet,
)
from tests.e2e_tests.utils.test_utils import (
from tests.e2e_tests.utils.e2e_test_utils import (
setup_wallet,
template_path,
templates_repo,
Expand Down
2 changes: 1 addition & 1 deletion tests/e2e_tests/test_transfer.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
from bittensor import Subtensor, logging
from bittensor.core.subtensor import transfer_extrinsic
from tests.e2e_tests.utils.test_utils import setup_wallet
from tests.e2e_tests.utils.e2e_test_utils import setup_wallet


def test_transfer(local_chain):
Expand Down
83 changes: 83 additions & 0 deletions tests/e2e_tests/utils/e2e_test_utils.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
import os
import shutil
import subprocess
import sys

from substrateinterface import Keypair

import bittensor

template_path = os.getcwd() + "/neurons/"
templates_repo = "templates repository"


def setup_wallet(uri: str) -> tuple[Keypair, bittensor.Wallet]:
"""
Sets up a wallet using the provided URI.
This function creates a keypair from the given URI and initializes a wallet
at a temporary path. It sets the coldkey, coldkeypub, and hotkey for the wallet
using the generated keypair.
Side Effects:
- Creates a wallet in a temporary directory.
- Sets keys in the wallet without encryption and with overwriting enabled.
"""
keypair = Keypair.create_from_uri(uri)
wallet_path = f"/tmp/btcli-e2e-wallet-{uri.strip('/')}"
wallet = bittensor.Wallet(path=wallet_path)
wallet.set_coldkey(keypair=keypair, encrypt=False, overwrite=True)
wallet.set_coldkeypub(keypair=keypair, encrypt=False, overwrite=True)
wallet.set_hotkey(keypair=keypair, encrypt=False, overwrite=True)
return keypair, wallet


def clone_or_update_templates(specific_commit=None):
"""
Clones or updates the Bittensor subnet template repository.
This function clones the Bittensor subnet template repository if it does not
already exist in the specified installation directory. If the repository already
exists, it updates it by pulling the latest changes. Optionally, it can check out
a specific commit if the `specific_commit` variable is set.
"""
install_dir = template_path
repo_mapping = {
templates_repo: "https://github.com/opentensor/bittensor-subnet-template.git",
}

os.makedirs(install_dir, exist_ok=True)
os.chdir(install_dir)

for repo, git_link in repo_mapping.items():
if not os.path.exists(repo):
print(f"\033[94mCloning {repo}...\033[0m")
subprocess.run(["git", "clone", git_link, repo], check=True)
else:
print(f"\033[94mUpdating {repo}...\033[0m")
os.chdir(repo)
subprocess.run(["git", "pull"], check=True)
os.chdir("..")

# For pulling specific commit versions of repo
if specific_commit:
os.chdir(templates_repo)
print(
f"\033[94mChecking out commit {specific_commit} in {templates_repo}...\033[0m"
)
subprocess.run(["git", "checkout", specific_commit], check=True)
os.chdir("..")

return install_dir + templates_repo + "/"


def install_templates(install_dir):
subprocess.check_call([sys.executable, "-m", "pip", "install", install_dir])


def uninstall_templates(install_dir):
subprocess.check_call(
[sys.executable, "-m", "pip", "uninstall", "bittensor_subnet_template", "-y"]
)
# Delete everything in directory
shutil.rmtree(install_dir)

0 comments on commit 51c03bd

Please sign in to comment.