Merge pull request #743 from opentensor/devnet-ready

Devnet deployment 20082024

.github/workflows/check-devnet.yml

@@ -2,7 +2,7 @@ name: Devnet Deploy Check
 
 on:
   pull_request:
-    branches: [devnet]
+    branches: [devnet, devnet-ready]
 
 env:
   CARGO_TERM_COLOR: always
@@ -11,6 +11,7 @@ jobs:
   check-spec-version:
     name: Check spec_version bump
     runs-on: SubtensorCI
+    if: ${{ !contains(github.event.pull_request.labels.*.name, 'no-spec-version-bump') }}
     steps:
       - name: Dependencies
         run: |
@@ -37,7 +38,7 @@ jobs:
           echo "network spec_version: $spec_version"
           if (( $(echo "$local_spec_version <= $spec_version" | bc -l) )); then echo "$local_spec_version ≯ $spec_version ❌"; exit 1; fi
           echo "$local_spec_version > $spec_version ✅"
-          
+
   check-devnet-migrations:
     name: check devnet migrations
     runs-on: ubuntu-22.04
@@ -51,4 +52,4 @@ jobs:
           runtime-package: "node-subtensor-runtime"
           node-uri: "wss://dev.chain.opentensor.ai:443"
           checks: "pre-and-post"
-          extra-args: "--disable-spec-version-check --no-weight-warnings"
+          extra-args: "--disable-spec-version-check --no-weight-warnings"

.github/workflows/check-finney.yml

@@ -11,6 +11,7 @@ jobs:
   check-spec-version:
     name: Check spec_version bump
     runs-on: SubtensorCI
+    if: ${{ !contains(github.event.pull_request.labels.*.name, 'no-spec-version-bump') }}
     steps:
       - name: Dependencies
         run: |

.github/workflows/check-rust.yml

@@ -113,6 +113,54 @@ jobs:
       - name: cargo clippy --workspace --all-targets -- -D warnings
         run: cargo clippy --workspace --all-targets -- -D warnings
 
+  cargo-check-lints:
+    name: check custom lints
+    runs-on: SubtensorCI
+    strategy:
+      matrix:
+        rust-branch:
+          - stable
+        rust-target:
+          - x86_64-unknown-linux-gnu
+        # - x86_64-apple-darwin
+        os:
+          - ubuntu-latest
+          # - macos-latest
+    env:
+      RELEASE_NAME: development
+      RUSTV: ${{ matrix.rust-branch }}
+      RUSTFLAGS: -D warnings
+      RUST_BACKTRACE: full
+      RUST_BIN_DIR: target/${{ matrix.rust-target }}
+      SKIP_WASM_BUILD: 1
+      TARGET: ${{ matrix.rust-target }}
+    steps:
+      - name: Check-out repository under $GITHUB_WORKSPACE
+        uses: actions/checkout@v4
+
+      - name: Install dependencies
+        run: |
+          sudo apt-get update &&
+          sudo apt-get install -y clang curl libssl-dev llvm libudev-dev protobuf-compiler
+
+      - name: Install Rust ${{ matrix.rust-branch }}
+        uses: actions-rs/[email protected]
+        with:
+          toolchain: ${{ matrix.rust-branch }}
+          components: rustfmt, clippy
+          profile: minimal
+
+      - name: Utilize Shared Rust Cache
+        uses: Swatinem/[email protected]
+        with:
+          key: ${{ matrix.os }}-${{ env.RUST_BIN_DIR }}
+
+      - name: check lints
+        run: |
+          set -o pipefail  # Ensure the pipeline fails if any command in the pipeline fails
+          cargo check 2>&1 | sed -r "s/\x1B\[[0-9;]*[mK]//g" | tee /dev/tty | grep -q "^warning:" && \
+          (echo "Build emitted the following warnings:" >&2 && exit 1) || echo "No warnings found."
+
   cargo-clippy-all-features:
     name: cargo clippy --all-features
     runs-on: SubtensorCI
@@ -338,7 +386,3 @@ jobs:
 
       - name: Check features
         run: zepter run check
-
-
-
-

.github/workflows/check-testnet.yml

@@ -2,7 +2,7 @@ name: Testnet Deploy Check
 
 on:
   pull_request:
-    branches: [testnet]
+    branches: [testnet, testnet-ready]
 
 env:
   CARGO_TERM_COLOR: always
@@ -11,6 +11,7 @@ jobs:
   check-spec-version:
     name: Check spec_version bump
     runs-on: SubtensorCI
+    if: ${{ !contains(github.event.pull_request.labels.*.name, 'no-spec-version-bump') }}
     steps:
       - name: Dependencies
         run: |

.rustfmt.toml

@@ -6,7 +6,7 @@
 ##
 # rustup run nightly -- rustfmt node/src/main.rs
 
-# max_width = 100
+# max_width = 180
 # hard_tabs = false
 # tab_spaces = 4
 # newline_style = "Auto"

CODEOWNERS

@@ -1 +1 @@
-* @sacha-l @lisa-parity
+* @unconst

CONTRIBUTING.md

@@ -3,7 +3,7 @@
 ## Lifecycle of a Pull Request
 
 1. Individuals wishing to contribute to subtensor should develop their change/feature/fix in a
-   [Pull Request](https://github.com/opentensor/subtensor/compare) (PR) targeting the `main`
+   [Pull Request](https://github.com/opentensor/subtensor/compare) (PR) targeting the `devnet-ready`
    branch of the subtensor GitHub repository. It is recommended to start your pull request as a
    draft initially until you are ready to have other developers actively look at it. Any
    changes to pallet/runtime code should be accompanied by integration and/or unit tests fully
@@ -13,69 +13,29 @@
    Review" and request review from "Nucleus".
 3. Core Nucleus team members will review your PR, possibly requesting changes, and will also
    add appropriate labels to your PR as shown below. Three positive reviews are required.
-4. Once the required passing reviews have been obtained, you are ready to request that your PR
-   be included in the next `devnet` deploy. To do this, you should open a companion PR merging
-   a copy of your branch into the `devnet-ready` branch. You must include a link to the parent
-   PR in the description and preface your PR title with "(Devnet Ready)" or the PR will be
-   closed/ignored. Your companion PR should have the `devnet-companion` label.
-5. A core team administrator will review your "(Devnet Ready)" PR, verifying that it logically
-   matches the changes introduced in the parent PR (there will sometimes be minor differences
-   due to merge conflicts) and will either request changes or approve the PR and merge it. Once
-   your companion PR is merged, the administrator will add the `devnet-ready` label to the
-   parent PR, indicating that the PR is on the `devnet-ready` branch and will be included in
-   the next deploy to `devnet`.
-6. At some point, a core team administrator will open a PR merging the current `devnet-ready`
+4. Once the required passing reviews have been obtained, you or an administrator may merge the
+   PR into the `devnet-ready` branch.
+5. At some point, a core team administrator will open a PR merging the current `devnet-ready`
    branch into `devnet`, and the CI will enforce some additional safety checks on this PR
    including a requirement that the new `spec_version` be greater than the current on-chain
    `spec_version`. The PR should include a bulleted list of all PRs included in the deploy so
-   they can be easily found after the fact (TODO: automate this). This PR will require two
-   reviews from the core team as a sanity check. After merging, the administrator will then
-   need to update all PRs with the `devnet-ready` label to instead have the `on-devnet` label
-   (TODO: automate this upon merge). The administrator will then deploy `devnet`.
-7. Once the `on-devnet` label appears on your PR, if you are a core team member it is your
-   responsibility to verify that the features/changes/fixes introduced by your PR are
-   functioning properly on `devnet` by interacting with the live network. If you are an
-   external contributor, a core team member will be assigned to test this for you.
-8. If your feature/change/fix is confirmed working on `devnet`, the `devnet-pass` label should
-   be added. Otherwise if there are issues, the `devnet-fail` label should be added and you
-   will need to make changes to your PR and repeat the previous steps in this process. In some
-   cases a revert PR will need to be created reverting your changes from the `pre-devnet` and
-   `devnet` branches, respectively.
-9. Once `devnet-pass` has been added to your PR, it is eligible for inclusion in the next
-   `testnet` deploy. We typically run `testnet` deploys every other wednesday.
-10. On the appropriate date, an administrator will open a PR merging the current `devnet`
+   they can be easily found after the fact.
+6. Once your feature/change/fix is on `devnet`, it is your responsibility to confirm it is
+   working properly. If it is not working and additional changes are needed, please coordinate
+   with a core team administrator and/or open up a new PR into `devnet` either reverting your
+   changes or making any required changes in order for the feature to function properly.
+7.  On the appropriate date, an administrator will open a PR merging the current `devnet`
     branch into `testnet`. This PR should include a bulleted list of all PRs included in the
-    deploy so they can be easily found after the fact (TODO: automate this). The PR should
-    exclude any PRs that currently have the `devnet-fail` label via a revert (TODO: enforce via
-    CI). This PR will require two reviews from the core team as a sanity check. After merging
-    into `testnet`, the administrator will then need to run the deploy and update all PRs
-    included in the deploy with the `on-testnet` label (TODO: automate this upon merge). Next
-    the administrator must cut a (pre-release) release in GitHub for `testnet` (TODO: github
-    action to generate the release and release notes).
-11. Once the `on-testnet` label appears on your PR, if you are a core team member it is your
-    responsibility to once again verify that the features/changes/fixes introduced by your PR
-    are functioning properly on `testnet` by interacting with the live network, if applicable.
-    If you are an external contributor, a core team member may be assigned to do this testing
-    for you but otherwise it will be your responsibility to show evidence on the PR that the
-    testing is successful. Once this has been verified, the `testnet-pass` label should be
-    added. If testing fails, the `testnet-fail` label should be added and PRs should be opened
-    reverting the change from `devnet-ready`, and then a PR should be opened merging the
-    modified `devnet` into `testnet`. These revert PRs, if they occur, _must_ be merged before
-    a new deploy can be run (TODO: enforce this via CI).
-12. After the SOP period (1 week on `testnet`) has passed and the `testnet-pass` label has been
-    added, the CI checks on your PR should now turn all green and a core team member will be
-    able to merge your PR into `main`. At this point your PR is done and is eligible to be
-    included in the next `finney` deploy (TODO: track and enforce SOP compliance on a per-PR
-    basis in CI based on the timestamps of label changes). We typically run `finney` deploys
-    every other Wednesday, so this will typically happen the Wednesday following the Wednesday
-    your PR was deployed to `testnet`. An administrator will run this deploy. The process the
-    administrator follows is to open a PR merging `main` into the `finney` branch, which will
-    always track the current state of `finney`. This PR automatically has some additional
-    checks on it such as asserting that the spec_version gets bumped properly and other sanity
-    checks designed to stop a bad deploy. Once the PR is reviewed and merged, the administrator
-    will run the actual deploy. Once that is successful, the administrator will cut a new
-    GitHub release tagged off of the latest `main` branch commit that was included in the
-    deploy, and announcements will be made regarding the release.
+    deploy so they can be easily found after the fact (TODO: automate this). This PR is merged,
+   the administrator will deploy `testnet` and cut a (pre-release) release in GitHub for
+    `testnet` (TODO: github action to generate the release and release notes).
+11. It is now your responsibility to once again check that your feature/change/fix is working
+    properly, this time on `testnet`. Once again if it is not working or additional changes are
+    needed, please coordinate with a core team administrator ASAP and/or open up a new PR into
+    `testnet` either reverting your changes or making any required changes in order for the
+    feature to function properly.
+12. At some point the administrator will merge current `testnet` into `main` and cut a new
+    deploy to mainnet/finney.
 
 ## PR Labels
 
@@ -85,40 +45,24 @@
 | `blue-team` | PR is focused on preventative/safety measures and/or dev UX improvements | none |
 | `runtime` | PR contains substantive changes to runtime / pallet code | none |
 | `breaking-change` | PR requires synchronized changes with bittensor | Triggers an automatic bot message so the relevant teams are made aware of the change well in advance |
-| `migration` | PR contains one or more migrations | none |
-| `devnet-companion` | Designates a devnet companion PR | Presence of `devnet-companion` label is checked |
-| `devnet-ready` | PR's branch has been merged into the `devnet-ready` branch and will be included in the next `devnet` deploy | none |
-| `on-devnet` | PR has been deployed to `devnet` | Removes `devnet-ready` |
-| `devnet-pass` | PR has passed manual testing on `devnet` | `devnet-pass` or `devnet-skip` required |
-| `devnet-skip` | Allows a critical hotfix PR to skip required testing on `devnet` | `devnet-pass` or `devnet-skip` required |
-| `devnet-fail` | PR has failed manual testing on `devnet` and requires modification | none |
-| `testnet-companion` | Designates a testnet companion PR | Presence of `testnet-companion` label is checked |
-| `on-testnet` | PR has been deployed to `testnet` | none |
-| `testnet-pass` | PR has passed manual testing on `testnet` | `testnet-pass` or `testnet-skip` required |
-| `testnet-skip` | Allows a critical hotfix PR to skip required manual testing and SOP on `testnet` | `testnet-pass` or `testnet-skip` required |
-| `testnet-fail` | PR has failed manual testing on `testnet` and requires modification | none |
-
 
 ## Branches
 
 
 ### `devnet-ready`
 
-Companion PRs merge into this branch, eventually accumulating into a merge of `devnet-ready`
-into `devnet`, coinciding with a deploy of `devnet`.
+All new feature/change/fix PRs should merge into this branch.
 
 #### Restrictions
 * no deleting the branch
 * no force pushes
 * no direct pushes
-* require 1 positive review from an administrator
-* new code changes invalidate existing reviews
+* require 3 positive review from an administrator
+* new code changes do _not_ invalidate existing reviews
 * only merge commit style merging allowed
 
 #### CI-Enforced Restrictions
 * `check-rust.yml` must pass
-* TODO: parent PR must be linked to in description
-* TODO: parent PR must have the required number of positive reviews
 
 
 ### `devnet`
@@ -164,33 +108,8 @@ tags for `testnet` releases.
 
 ### `main`
 
-Default branch for all new PRs. Slightly ahead of what is currently on `finney`. When a PR is all
-green and "done", meaning it has been tested on `devnet` and `testnet`, it can be merged into
-`main`. Contains tags for `finney` releases.
-
-#### Restrictions
-* no deleting the branch
-* no force pushes
-* no direct pushes
-* require 3 positive reviews from core team members
-* new code changes invalidate existing reviews
-* all conversations must be resolved
-* only merge commit style merging allowed
-
-#### CI-Enforced Restrictions
-* `check-rust.yml` must pass
-* `check-labels.yml` must pass
-* must have `devnet-skip` or `devnet-pass` label
-* must have `testnet-skip` or `testnet-pass` label
-* if `breaking-change` label is present, bot will message the appropriate teams
-* TODO: when we get auditing, presence of `needs-audit` label = require a review from auditor
-* TODO: track SOP on PR based on label age
-
-
-### `finney`
-
 Tracks the current state of what is deployed to `finney` (mainnet). Updated via an
-administrator-submitted PR merging `main` into `finney` in concert with a `finney` deploy.
+administrator-submitted PR merging `testnet` into `main` in concert with a `finney` deploy.
 
 #### Restrictions
 * no deleting the branch
@@ -203,5 +122,5 @@ administrator-submitted PR merging `main` into `finney` in concert with a `finne
 #### CI-Enforced Restrictions
 * `check-rust.yml` must pass
 * `check-finney.yml` must pass
-* spec_version must be greater than what is currently on live `finney`
+* `spec_version` must be greater than what is currently on live `finney`
 * TODO: other pre-deploy sanity checks here