Release v4.4.0

Announcing release v4.4.0

This release includes the following changes.

Changes

• Includes the deactivation block for sunsetting 7B competition (competition ID 0). Block 3849722.
• The 0.15 reward previously attributed to 7B competition will be added to the 14B competition whose reward will total 0.57.
• Did some minor code clean-up including the removal of the epsilon experiment code.
• No action to be taken by the validators other than pip install -e . to update to the new version.

Release v4.3.0

Announcing Release v4.3.0

This release includes the following updates:

Changes

• Activation block for the 14B model sub-competition unchanged 3_750_683.
• Deactivation block for the 7B* experiment unchanged 3_750_683.
• Added linear decay of epsilon for all competition starting at the same activation block 3_750_683.
• Updated system requirements for validators to require Nvidia A100 GPUs with 80GB of VRAM.
• Taoverse dependency bumped to 1.0.5

Note to validators

• Validators will now retry models that may beat the top model based on the current epsilon. This feature was released in v4.2.0 but it will becomes useful only with the current release.

• Although only 40GB of VRAM is necessary, we have observed that A100 GPUs with 80GB are more readily available and are offered at a comparable price to the 40GB variants. The additional VRAM provided by this GPU will allows more flexibility for optimization in future releases, enabling larger validation batch sizes to enhance the stability of the validation process by reducing scoring variance.

• Please also make sure to rerun pip install to ensure updated dependencies.

python -m pip install -e .

Release v4.2.0

Announcing Release v4.2.0

This release includes the following updates:

Changes

• Includes activation block for the 14B model sub-competition 3_750_683.
• Includes deactivation block for the 7B* experiment 3_750_683.
• Validators now retry models that may beat the top model based on the current epsilon. This feature will support the future release of decaying epsilon.
• Validator schema version bumped to 3.1.0
• Taoverse dependency bumped to 1.0.4.

Notes

We recommend validators to update system storage to at least 2T to account for 14B models and system RAM to at least 80GB.

Please also make sure to rerun pip install to ensure updated dependencies

python -m pip install -e .

Release v4.1.2

Announcing Release v4.1.2

This patch release introduces the following changes:

Changes

• Bumped the Bittensor dependency to version 6.9.4.
• Bumped the Transformers library dependency to 4.44.1.
• New architectures Gemma2 and Qwen2 are now fully functional.
• StableLM and Phi3 support has been temporarily removed for further testing.
• Improved validator logging.

For validation, please make sure to rerun pip install to ensure updated dependencies

python -m pip install -e .

Release v4.1.1

This release fixes a few issues when increasing the competition count.

• Downloaded model limit is now adjusted dynamically by competition count.
• Competition specific weights rather than subnet weights are used when prioritizing models to keep. (This helps clear out UIDs who swap the competition they are competing in from their previous competition).
• Threshold for model prioritization from other top validators is lowered to 5% weight as weights are spread more thinly over multiple competitions.

Release v4.1.0

Announcing Release v4.1.0

Announcing activation blocks for two new sub-competitions.

Changes

• This version schedules the launch of the 3B model competition, and the 7B* experiment with an epsilon of 0.001. Both are scheduled to be activated on block 3_601_190.

• Sample unpacking is also scheduled to start on block 3_601_190.

• All submitted models for the 7B competition will be automatically attributed to the 7B* experiment sub-competition. The latter will show as competition 3 in the wandb logs. But please use --competition_id 0 when you upload your model. DO NOT use --competition_id 3 when you upload your model.

Note: As we continue to increase the number of competitions, we might observe some fluctuations in vTrust. We will monitor them closely.

Release v4.0.1

This release adds in the necessary subnet 9 specific CompetitionId IntEnum.

Release v4.0.0

Announcing Release v4.0.0

Subnet-9 now supports multiple concurrent competitions.

Changes

• This version implements multi-competition support and a programmed launch for the 700M model competition in parallel with the current 7B model competition. It also contains the 3B competition constraints ready for scheduling.

• The validator version shipped with this release is bumped 3.0.0. This mean that the validator state will be erased and evaluation for all models will be restarted from scratch.

• All submitted models for 7B will be automatically attributed to the 7B sub-competition with a default competition ID of 0.

• Past submissions for 772M models and less will be ignored, even after the launch of the 700M competition on Thursday.

• Miner weights and model losses will be logged to wandb along with the competition ID for each miner: https://wandb.ai/macrocosmos/pretraining-validators?nw=nwusercloudops

• Related discord announcement: https://discord.com/channels/799672011265015819/1162768567821930597/1270164103125663807

• Allowed models list has been adjusted to include new model types:

  • Gemma2ForCausalLM
  • StableLmForCausalLM
  • Phi3ForCausalLM
  • Qwen2ForCausalLM

• The flag --competition_id should now be used with the scripts/upload_model.py script to specify the competition the miner is willing to submit their model to.

• No change to the validator launch command.

Release 3.2.2

Announcing Release 3.2.2

This is a patch release with some minor changes:

Changes

• Validators Logging: The logging for validators has been redirected to the pretraining-validators project within the Macrocosmos Wandb entity.
• Repository Version Logging: The repository version is now logged in Wandb along with the validator version. The version entry in the overview tab on the validator logging page in Wandb now refers to the repository version instead of the validator version.

Release 3.2.1

Announcing Release 3.2.1

This bugfix release addresses a critical issue impacting validator performance and stability. We strongly recommend all users update to this version to ensure optimal functionality.

Bugfix

• Description of the Bug: In the previous release (v3.2.0), all models submitted before block 3256604 were set to be evaluated on RefinedWeb instead of FineWeb-Edu Score-2. This were not the desired behavior. After the specified block, we want all models to be validated on FineWeb-Edu Score-2.
• Resolution: We have fixed the issue, ensuring that all models are now correctly evaluated on the FineWeb-Edu Score-2 dataset starting from block 3307004 instead of block 3256604 (one week earlier), as was originally intended.

Validator

• Validators will now correctly evaluate all models on the FineWeb-Edu Score-2 dataset, as introduced in the previous release.
• The validator version remains decoupled from the package version, maintaining flexibility for future updates.
• We have updated package dependencies, pinning Bittensor to version v6.9.3 to enhance the validator setup experience and stability.

We urge everyone to pull this release to update their validators and benefit from the critical bug fixes included in this release.