|
| 1 | +<div align="left"> |
| 2 | + |
| 3 | +# **Cortex.t Subnet** <!-- omit in toc --> |
| 4 | +[](https://opensource.org/licenses/MIT) |
| 5 | +--- |
| 6 | + |
| 7 | +--- |
| 8 | +- [Introduction](#introduction) |
| 9 | +- [Setup](#setup) |
| 10 | +- [Mining](#mining) |
| 11 | +- [Validating](#validating) |
| 12 | +- [License](#license) |
| 13 | + |
| 14 | + |
| 15 | +## Introduction |
| 16 | + |
| 17 | +**IMPORTANT**: If you are new to Bittensor, please checkout the [Bittensor Website](https://bittensor.com/) before proceeding to the [Setup](#setup) section. |
| 18 | + |
| 19 | +## Setup |
| 20 | + |
| 21 | +### Before you proceed |
| 22 | +Before you proceed with the installation of the subnet, note the following: |
| 23 | + |
| 24 | +We **strongly recommend** before proceeding that you test both subtensor and all API keys. Ensure you are running Subtensor locally to minimize chances of outages and improve the latency/connection. |
| 25 | + |
| 26 | +### API Key Requirements |
| 27 | + |
| 28 | +API requirements for this subnet are constantly evolving as we seek to meet the demands of the users and stay up to date with the latest developements. The current key requirements are as follows: |
| 29 | + |
| 30 | +- OpenAI key (GPT) |
| 31 | +- Google API key (Gemini) |
| 32 | +- Anthropic API key (Claude3) |
| 33 | +- Groq API key (Llama, Gemini, Mistral) |
| 34 | +- AWS Acces key and Secret key (Bedrock models) |
| 35 | +- Pixabay API key |
| 36 | + |
| 37 | +Please test the API keys from ```.env.example``` with ```test_scripts/t2t/``` before starting the miner/validator. Detailed instructions on how to aquire the API keys below. |
| 38 | + |
| 39 | +### Requesting Access for AWS Bedrock Models |
| 40 | + |
| 41 | +#### 1. AWS Account |
| 42 | +Ensure you have an active AWS account. If you don't have one, you can create it at [AWS Account Creation](https://aws.amazon.com/). |
| 43 | + |
| 44 | +#### 2. Sign In to AWS Management Console |
| 45 | +Go to the [AWS Management Console](https://aws.amazon.com/console/) and sign in with your AWS credentials. |
| 46 | + |
| 47 | +#### 3. Navigate to AWS Bedrock |
| 48 | +- In the AWS Management Console, use the search bar at the top to search for "Bedrock". |
| 49 | +- Select AWS Bedrock from the search results. |
| 50 | + |
| 51 | +#### 4. Request Access |
| 52 | +- If AWS Bedrock is not directly available, you might see a page to request access. |
| 53 | +- Follow the prompts to fill out any required information. This might include your use case for the models, your AWS account ID, and other relevant details. |
| 54 | + |
| 55 | +#### 5. Submit a Request |
| 56 | +- Complete any forms or questionnaires provided to describe your intended use of AWS Bedrock models. |
| 57 | +- Submit the request for review. |
| 58 | + |
| 59 | +#### 6. Wait for Approval |
| 60 | +- AWS will review your request. This can take some time depending on the specifics of your request and the current availability of AWS Bedrock. |
| 61 | +- You will receive an email notification once your request is approved or if further information is needed. |
| 62 | + |
| 63 | +### Obtaining AWS Access Key and Secret Key |
| 64 | + |
| 65 | +#### 1. Sign In to AWS Management Console |
| 66 | +Go to the [AWS Management Console](https://aws.amazon.com/console/) and sign in with your AWS credentials. |
| 67 | + |
| 68 | +#### 2. Navigate to My Security Credentials |
| 69 | +- Click on your account name at the top right corner of the AWS Management Console. |
| 70 | +- Select "Security Credentials" from the dropdown menu. |
| 71 | + |
| 72 | +#### 3. Create New Access Key |
| 73 | +- In the "My Security Credentials" page, go to the "Access keys" section. |
| 74 | +- Click on "Create Access Key". |
| 75 | +- A pop-up will appear showing your new Access Key ID and Secret Access Key. |
| 76 | + |
| 77 | +#### 4. Download Credentials |
| 78 | +- Download the `.csv` file containing these credentials or copy them to a secure location. |
| 79 | + - **Important**: This is the only time you will be able to view the secret access key. If you lose it, you will need to create new credentials. |
| 80 | + |
| 81 | +### 5. Alternative - create dedicated user (more secure) |
| 82 | +- Navigate to IAM |
| 83 | +- Create New User - name the user `sn-18` or similar |
| 84 | +- Attach `AmazonBedrockFullAccess` policy to user or apply the following permissions |
| 85 | +```json |
| 86 | +{ |
| 87 | + "Version": "2012-10-17", |
| 88 | + "Statement": [ |
| 89 | + { |
| 90 | + "Sid": "BedrockAll", |
| 91 | + "Effect": "Allow", |
| 92 | + "Action": [ |
| 93 | + "bedrock:*" |
| 94 | + ], |
| 95 | + "Resource": "*" |
| 96 | + }, |
| 97 | + { |
| 98 | + "Sid": "DescribeKey", |
| 99 | + "Effect": "Allow", |
| 100 | + "Action": [ |
| 101 | + "kms:DescribeKey" |
| 102 | + ], |
| 103 | + "Resource": "arn:*:kms:*:::*" |
| 104 | + }, |
| 105 | + { |
| 106 | + "Sid": "APIsWithAllResourceAccess", |
| 107 | + "Effect": "Allow", |
| 108 | + "Action": [ |
| 109 | + "iam:ListRoles", |
| 110 | + "ec2:DescribeVpcs", |
| 111 | + "ec2:DescribeSubnets", |
| 112 | + "ec2:DescribeSecurityGroups" |
| 113 | + ], |
| 114 | + "Resource": "*" |
| 115 | + }, |
| 116 | + { |
| 117 | + "Sid": "PassRoleToBedrock", |
| 118 | + "Effect": "Allow", |
| 119 | + "Action": [ |
| 120 | + "iam:PassRole" |
| 121 | + ], |
| 122 | + "Resource": "arn:aws:iam::*:role/*AmazonBedrock*", |
| 123 | + "Condition": { |
| 124 | + "StringEquals": { |
| 125 | + "iam:PassedToService": [ |
| 126 | + "bedrock.amazonaws.com" |
| 127 | + ] |
| 128 | + } |
| 129 | + } |
| 130 | + } |
| 131 | + ] |
| 132 | +} |
| 133 | +``` |
| 134 | + |
| 135 | +### Obtaining API Key from OpenAI |
| 136 | + |
| 137 | +#### 1. OpenAI Account |
| 138 | +Ensure you have an active OpenAI account. If you don't have one, you can create it at [OpenAI Account Creation](https://platform.openai.com/signup). |
| 139 | + |
| 140 | +#### 2. Sign In to OpenAI |
| 141 | +Go to the [OpenAI Platform](https://platform.openai.com/api-keys) and sign in with your OpenAI credentials. |
| 142 | + |
| 143 | +#### 3. Create New API Key |
| 144 | +- Click on the "Create new secret key" button. |
| 145 | +- Follow the instructions provided to create your API key. |
| 146 | + |
| 147 | + |
| 148 | +### Obtaining API Key from Google AI Platform |
| 149 | + |
| 150 | +#### 1. Sign In to Google AI Platform |
| 151 | +Go to the [Google AI Platform](https://aistudio.google.com/) and sign in with your Google credentials. |
| 152 | + |
| 153 | +#### 2. Get API Key |
| 154 | +- In the Google AI Platform, click on the "Get API key" button at the top left corner. |
| 155 | +- Follow the instructions provided to create and retrieve your API key. |
| 156 | + |
| 157 | + |
| 158 | +### Obtaining API Key from Anthropic |
| 159 | + |
| 160 | +#### 1. Anthropic Account |
| 161 | +Ensure you have an active Anthropic account. If you don't have one, you can create it at [Anthropic Account Creation](https://www.anthropic.com/signup). |
| 162 | + |
| 163 | +#### 2. Sign In to Anthropic |
| 164 | +Go to the [Anthropic Platform](https://console.anthropic.com/settings/keys) and sign in with your Anthropic credentials. |
| 165 | + |
| 166 | +#### 3. Get API Key |
| 167 | +- In the Settings, go to the "API keys" tab and click on the "Create key" button at the top right corner. |
| 168 | +- Follow the instructions provided to create and retrieve your API key. |
| 169 | + |
| 170 | + |
| 171 | +### Obtaining API Key from Groq |
| 172 | + |
| 173 | +#### 1. Groq Account |
| 174 | +Ensure you have an active Groq account. If you don't have one, you can create it at [Groq Account Creation](https://groq.com/signup). |
| 175 | + |
| 176 | +#### 2. Sign In to Groq |
| 177 | +Go to the [Groq Platform](https://console.groq.com/) and sign in with your Groq credentials. |
| 178 | + |
| 179 | +#### 3. Get API Key |
| 180 | +- In the Groq Platform, click on the "API keys" button at the left side. |
| 181 | +- Click "Create API key" |
| 182 | +- Follow the instructions provided to create and retrieve your API key. |
| 183 | + |
| 184 | + |
| 185 | +### Obtaining API Key from Pixabay |
| 186 | + |
| 187 | +#### 1. Pixabay Account |
| 188 | +Ensure you have an active Pixabay account. If you don't have one, you can create it at [Pixabay Account Creation](https://pixabay.com/ru/accounts/register/). |
| 189 | + |
| 190 | +#### 2. Sign In to Pixabay |
| 191 | +Go to the [Pixabay API docs](https://pixabay.com/api/docs/) and sign in with your Pixabay credentials. |
| 192 | + |
| 193 | +#### 3. Get API Key |
| 194 | +- Scroll down this page a little. Your key will be highlighted in green in the parameters for one of the requests. |
| 195 | + |
| 196 | + |
| 197 | +### Installation |
| 198 | + |
| 199 | +Before starting make sure update your system and have pm2 installed to run the scripts in the background. |
| 200 | + |
| 201 | +```bash |
| 202 | +apt update -y && apt-get install git -y && apt install python3-pip -y |
| 203 | + |
| 204 | +``` |
| 205 | + |
| 206 | +Download the repository, navigate to the folder and then create virtual env and install the necessary requirements with the following chained command. |
| 207 | + |
| 208 | +```bash |
| 209 | +git clone https://github.com/corcel-api/cortex.t.git && cd cortex.t && pip3 install -e . |
| 210 | +``` |
| 211 | + |
| 212 | +Prior to proceeding, ensure you have a registered hotkey on subnet 18 mainnet. If not, run the command |
| 213 | +```bash |
| 214 | +btcli s register --netuid 18 --wallet.name [wallet_name] --wallet.hotkey [wallet.hotkey] |
| 215 | +``` |
| 216 | + |
| 217 | +After installing it, copy `env.example` to `.env` and substitute |
| 218 | +all env vars with values appropriate for your accounts. |
| 219 | + |
| 220 | +## Mining |
| 221 | +# step1. |
| 222 | +go to cortext/constants.py and change bandwidth_to_model value as per limit of api. |
| 223 | +currently we support only 3 models: "gpt-4o", "claude-3-5-sonnet-20240620", "llama-3.1-70b-versatile". |
| 224 | +so don't add more than that. |
| 225 | +You can launch your miners via python3 using the following command. |
| 226 | +```bash |
| 227 | +bash start_miner.sh |
| 228 | +``` |
| 229 | + |
| 230 | + |
| 231 | +## Validating |
| 232 | + |
| 233 | +Login to wandb using |
| 234 | + |
| 235 | +```bash |
| 236 | +wand login |
| 237 | +``` |
| 238 | + |
| 239 | +You can launch your validator using following command |
| 240 | + |
| 241 | +```python |
| 242 | +pm2 start start_validator.py --interpreter python3 -- --wallet_name "default" --wallet_hotkey "default" --subtensor.chain_endpoint <URL here> --autoupdate --wandb_on |
| 243 | +``` |
| 244 | +--- |
| 245 | + |
| 246 | +# Cursor App Setup |
| 247 | +add env varialbe CURSOR_API_KEY to .env file |
| 248 | +go to inside of cursor directory and run this command. |
| 249 | +```bash |
| 250 | +pm2 start start_cursor.sh --name cursor_app |
| 251 | +``` |
| 252 | +for settings in cursor. Pls check this [Cursor README](./cursor/readme.md) |
| 253 | + |
| 254 | +## License |
| 255 | +This repository is licensed under the MIT License. |
| 256 | +```text |
| 257 | +# The MIT License (MIT) |
| 258 | +# Copyright © 2023 Yuma Rao |
| 259 | +
|
| 260 | +# Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated |
| 261 | +# documentation files (the “Software”), to deal in the Software without restriction, including without limitation |
| 262 | +# the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, |
| 263 | +# and to permit persons to whom the Software is furnished to do so, subject to the following conditions: |
| 264 | +
|
| 265 | +# The above copyright notice and this permission notice shall be included in all copies or substantial portions of |
| 266 | +# the Software. |
| 267 | +
|
| 268 | +# THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO |
| 269 | +# THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL |
| 270 | +# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION |
| 271 | +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER |
| 272 | +# DEALINGS IN THE SOFTWARE. |
| 273 | +``` |
0 commit comments