- 
                Notifications
    You must be signed in to change notification settings 
- Fork 0
Add TPS monitoring tool #2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
          
     Open
      
      
            Vsevolod-Rusinskiy
  wants to merge
  43
  commits into
  master
  
    
      
        
          
  
    
      Choose a base branch
      
     
    
      
        
      
      
        
          
          
        
        
          
            
              
              
              
  
           
        
        
          
            
              
              
           
        
       
     
  
        
          
            
          
            
          
        
       
    
      
from
add-tps-monitoring-tool
  
      
      
   
  
    
  
  
  
 
  
      
    base: master
Could not load branches
            
              
  
    Branch not found: {{ refName }}
  
            
                
      Loading
              
            Could not load tags
            
            
              Nothing to show
            
              
  
            
                
      Loading
              
            Are you sure you want to change the base?
            Some commits from the old base branch may be removed from the timeline,
            and old review comments may become outdated.
          
          
  
     Open
                    Changes from all commits
      Commits
    
    
            Show all changes
          
          
            43 commits
          
        
        Select commit
          Hold shift + click to select a range
      
      ba0e89c
              
                Add TPS monitoring tool
              
              
                Vsevolod-Rusinskiy 088f93a
              
                Update .gitignore to include docs/ directory and optimize showStats m…
              
              
                Vsevolod-Rusinskiy 9b2793d
              
                Refactor: modular TPS monitor with Utils class
              
              
                Vsevolod-Rusinskiy 883b6bd
              
                Add CSV export to dedicated folder
              
              
                Vsevolod-Rusinskiy 21c391e
              
                Move common components to shared folder
              
              
                Vsevolod-Rusinskiy 7175944
              
                Decompose transaction sender into modular architecture
              
              
                Vsevolod-Rusinskiy 045a33f
              
                Simplify CLI parameters with defaults
              
              
                Vsevolod-Rusinskiy 138e027
              
                fix: add missing getApi method to ApiConnector
              
              
                Vsevolod-Rusinskiy 3abc506
              
                Remove csv-report files from tracking and update .gitignore
              
              
                Vsevolod-Rusinskiy 2670ddb
              
                feat: rename orchestrator to dashboard and update Phase 1 progress
              
              
                Vsevolod-Rusinskiy c7e9119
              
                feat: implement structured logging system - Phase 3a complete
              
              
                Vsevolod-Rusinskiy 56eda61
              
                feat: migrate monitor core modules to structured logging
              
              
                Vsevolod-Rusinskiy 3238559
              
                feat: migrate statistics-reporter to structured logging
              
              
                Vsevolod-Rusinskiy 43cd860
              
                feat: complete monitor modules logging migration
              
              
                Vsevolod-Rusinskiy b45cea7
              
                feat: complete sender modules logging migration
              
              
                Vsevolod-Rusinskiy 0a5ce5f
              
                feat(dashboard): complete TUI component system
              
              
                Vsevolod-Rusinskiy 263e8a1
              
                Fix TUI dashboard startup and event loop
              
              
                Vsevolod-Rusinskiy 5598e1d
              
                Fix dashboard layout and API logs display
              
              
                Vsevolod-Rusinskiy 758beb5
              
                feat: increase log file size limits from 5MB to 20MB
              
              
                Vsevolod-Rusinskiy 06ba6ee
              
                refactor: remove unused blessed-contrib dependency and simplify TPS g…
              
              
                Vsevolod-Rusinskiy afe4a34
              
                feat(dashboard): enhance process management and control panel functio…
              
              
                Vsevolod-Rusinskiy d12b29c
              
                refactor(dashboard): improve keyboard handling and logging
              
              
                Vsevolod-Rusinskiy c49075b
              
                feat(dashboard): implement file-based logging for console outputs
              
              
                Vsevolod-Rusinskiy 78f07fe
              
                feat(dashboard): extend file-based logging to include console.warn
              
              
                Vsevolod-Rusinskiy 4f96afd
              
                refactor(dashboard): update comments for clarity and consistency
              
              
                Vsevolod-Rusinskiy 6580385
              
                refactor(dashboard): remove general update loop and replace TPS graph…
              
              
                Vsevolod-Rusinskiy c76c1e7
              
                feat(dashboard): add memory usage logging to EventLogComponent
              
              
                Vsevolod-Rusinskiy cac9c25
              
                refactor(dashboard): update comments for clarity in logging functiona…
              
              
                Vsevolod-Rusinskiy 263b4c4
              
                refactor(dashboard): comment out TPS metrics updates due to conflict
              
              
                Vsevolod-Rusinskiy 1a2fa00
              
                feat(dashboard): implement dynamic log file initialization in LogTPSR…
              
              
                Vsevolod-Rusinskiy ca178f3
              
                feat(dashboard): enhance logging functionality with custom debug logger
              
              
                Vsevolod-Rusinskiy 9f6a3e7
              
                refactor(dashboard): simplify structure
              
              
                Vsevolod-Rusinskiy c9e9ff4
              
                refactor(dashboard): remove unused scripts and files, add documentation
              
              
                Vsevolod-Rusinskiy e1336eb
              
                chore: update .gitignore to include git directory
              
              
                Vsevolod-Rusinskiy 93f19b9
              
                refactor(tps-monitoring): update README and package.json
              
              
                Vsevolod-Rusinskiy 773440d
              
                docs: translate README to English and clean up project structure
              
              
                Vsevolod-Rusinskiy 416caa2
              
                refactor(tps-monitoring): update author and license in package.json, …
              
              
                Vsevolod-Rusinskiy 078789e
              
                fix(transaction-sender): add proper nonce management
              
              
                Vsevolod-Rusinskiy e6008d4
              
                feat(load-testing): add sub-flood load testing tool structure
              
              
                Vsevolod-Rusinskiy ac44ebe
              
                refactor(tps-monitoring): update package.json and package-lock.json f…
              
              
                Vsevolod-Rusinskiy b598a2e
              
                refactor(tps-monitoring): update package dependencies and clean up pa…
              
              
                Vsevolod-Rusinskiy c11de81
              
                chore(tps-monitoring): update .gitignore and tsconfig.json, enhance p…
              
              
                Vsevolod-Rusinskiy 93ee192
              
                refactor(tps-monitoring): add detailed comments
              
              
                Vsevolod-Rusinskiy File filter
Filter by extension
Conversations
          Failed to load comments.   
        
        
          
      Loading
        
  Jump to
        
          Jump to file
        
      
      
          Failed to load files.   
        
        
          
      Loading
        
  Diff view
Diff view
There are no files selected for viewing
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
|  | @@ -6,3 +6,4 @@ | |
| /.idea | ||
| .env | ||
| .DS_Store | ||
| git/ | ||
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| node_modules/ | ||
|  | ||
|  | ||
|  | ||
| *.log | ||
| .DS_Store | ||
|  | ||
| # Drafts folder for temporary files and notes | ||
| drafts/ | 
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,379 @@ | ||
| # 🚀 Simple TPS Monitor: Substrate TPS Measurement Tool | ||
|  | ||
| A simple tool for measuring real TPS (transactions per second) in Polkadot/Substrate blockchains with modular architecture. | ||
|  | ||
| ## 🎯 Features | ||
|  | ||
| - ✅ **Accurate TPS measurement** - analyzes blocks, not sending speed | ||
| - ✅ **Real balance transactions** - uses transferKeepAlive | ||
| - ✅ **Automatic nonce management** - proper handling of multiple transactions | ||
| - ✅ **Continuous monitoring** - tracks blocks in real-time | ||
| - ✅ **Configurable load** - --tps parameter for controlling sending frequency | ||
| - ✅ **Monitor-only mode** - --tps 0 for analysis without load generation | ||
| - ✅ **Simple CLI interface** - one command to run | ||
| - ✅ **Graceful shutdown** - proper termination with Ctrl+C | ||
| - ✅ **Modular architecture** - clean separation of concerns | ||
|  | ||
| ## 📦 Installation | ||
|  | ||
| ```bash | ||
| # Clone the repository | ||
| git clone https://github.com/your-username/tps-monitoring.git | ||
| cd tps-monitoring | ||
|  | ||
| # Install dependencies | ||
| npm install | ||
| ``` | ||
|  | ||
| ## 🔧 Requirements | ||
|  | ||
| - Node.js >= 16.0.0 | ||
| - Access to Polkadot/Substrate node via WebSocket | ||
| - Alice account with balance for testing | ||
|  | ||
| ## 🏗️ How It Works - Modular Architecture | ||
|  | ||
| ### Project Structure | ||
| ``` | ||
| tps-monitoring/ | ||
| ├── src/ | ||
| │ ├── index.js # CLI entry point | ||
| │ ├── modules/ | ||
| │ │ ├── TPSMonitor.js # Main coordinator | ||
| │ │ ├── TransactionSender.js # Transaction generation | ||
| │ │ └── BlockMonitor.js # Block monitoring & TPS calculation | ||
| │ └── simple-monitor.js # Legacy monolithic version | ||
| ``` | ||
|  | ||
| ### System Flow | ||
|  | ||
| #### 1. **Entry Point (index.js)** | ||
| ```javascript | ||
| // Parse CLI arguments and create TPSMonitor instance | ||
| const monitor = new TPSMonitor() | ||
| await monitor.start(options) | ||
| ``` | ||
|  | ||
| **What happens:** | ||
| - Parses command line arguments (`--node`, `--tps`) | ||
| - Creates TPSMonitor instance | ||
| - Starts the monitoring system | ||
|  | ||
| #### 2. **Main Coordinator (TPSMonitor.js)** | ||
| ```javascript | ||
| // Initialize blockchain connection and components | ||
| await this.initialize(options.node) | ||
| this.blockMonitor.startMonitoring(this.api) | ||
| if (options.tps > 0) { | ||
| await this.transactionSender.startSending(options.tps) | ||
| } | ||
| ``` | ||
|  | ||
| **What happens:** | ||
| - Connects to blockchain via WebSocket | ||
| - Initializes cryptography and keyring | ||
| - Creates TransactionSender and BlockMonitor instances | ||
| - Starts both monitoring and transaction sending (if enabled) | ||
| - Handles graceful shutdown on Ctrl+C | ||
|  | ||
| #### 3. **Transaction Sender (TransactionSender.js)** | ||
| ```javascript | ||
| // Send transactions at specified TPS rate | ||
| const intervalMs = 1000 / tpsTarget | ||
| const sendTx = async () => { | ||
| const transfer = this.api.tx.balances.transferKeepAlive(this.alice.address, 1) | ||
| await transfer.signAndSend(this.alice) | ||
| setTimeout(sendTx, intervalMs) | ||
| } | ||
| ``` | ||
|  | ||
| **What happens:** | ||
| - Creates Alice test account | ||
| - Sends `transferKeepAlive` transactions to self | ||
| - Maintains specified TPS rate using `setTimeout` | ||
| - Logs progress every 10 transactions | ||
| - Can be stopped gracefully | ||
|  | ||
| #### 4. **Block Monitor (BlockMonitor.js)** | ||
| ```javascript | ||
| // Subscribe to new blocks and calculate TPS | ||
| api.derive.chain.subscribeNewHeads(async (header) => { | ||
| const block = await api.rpc.chain.getBlock(header.hash) | ||
| const txCount = block.block.extrinsics.length | ||
|  | ||
| // Store data and calculate TPS | ||
| this.blockTimes.push(now) | ||
| this.txCounts.push(txCount) | ||
| const avgTPS = this.calculateTPS() | ||
| }) | ||
| ``` | ||
|  | ||
| **What happens:** | ||
| - Subscribes to new blocks via WebSocket | ||
| - Extracts transaction count from each block | ||
| - Maintains sliding window of last 10 blocks | ||
| - Calculates real-time TPS: `total_transactions / time_span` | ||
| - Displays statistics for each block | ||
|  | ||
| ### Component Interaction | ||
|  | ||
| ``` | ||
| ┌─────────────┐ ┌─────────────────┐ ┌─────────────────┐ | ||
| │ index.js │───▶│ TPSMonitor.js │───▶│TransactionSender│ | ||
| │ (CLI) │ │ (Coordinator) │ │ .js │ | ||
| └─────────────┘ └─────────────────┘ └─────────────────┘ | ||
| │ │ | ||
| ▼ │ | ||
| ┌─────────────────┐ │ | ||
| │ BlockMonitor.js │◀────────────┘ | ||
| │ (TPS Calculator)│ (API shared) | ||
| └─────────────────┘ | ||
| ``` | ||
|  | ||
| ### Key Benefits of Modular Design | ||
|  | ||
| #### **Separation of Concerns** | ||
| - **TransactionSender**: Only handles transaction generation | ||
| - **BlockMonitor**: Only handles block monitoring and TPS calculation | ||
| - **TPSMonitor**: Only handles coordination and lifecycle | ||
| - **index.js**: Only handles CLI interface | ||
|  | ||
| #### **Testability** | ||
| ```javascript | ||
| // Test components independently | ||
| const sender = new TransactionSender(api, keyring) | ||
| await sender.startSending(5) | ||
|  | ||
| const monitor = new BlockMonitor() | ||
| monitor.startMonitoring(api) | ||
| ``` | ||
|  | ||
| #### **Reusability** | ||
| - TransactionSender can be used in other load testing tools | ||
| - BlockMonitor can be extended for other blockchain metrics | ||
| - TPSMonitor can be adapted for different blockchain networks | ||
|  | ||
| #### **Maintainability** | ||
| - Easy to find and fix bugs in specific components | ||
| - Simple to add new features (e.g., different transaction types) | ||
| - Clear interfaces between components | ||
|  | ||
| ## 🚀 Usage | ||
|  | ||
| ### Basic Command | ||
|  | ||
| ```bash | ||
| # Run with load generation (10 TPS by default) | ||
| node src/index.js | ||
|  | ||
| # Connect to custom node | ||
| node src/index.js --node ws://your-node:9944 | ||
|  | ||
| # Set target TPS | ||
| node src/index.js --tps 50 | ||
|  | ||
| # Monitor only without load generation | ||
| node src/index.js --tps 0 | ||
|  | ||
| # Full example | ||
| node src/index.js \ | ||
| --node ws://localhost:9944 \ | ||
| --tps 25 | ||
| ``` | ||
|  | ||
| ### CLI Parameters | ||
|  | ||
| - `-n, --node <url>` - Node WebSocket URL (default: ws://localhost:9944) | ||
| - `-t, --tps <number>` - Target TPS to generate (0 = monitor only, default: 10) | ||
|  | ||
| ### NPM Scripts | ||
|  | ||
| ```bash | ||
| # Run with default parameters | ||
| npm start | ||
| ``` | ||
|  | ||
| ## 📊 How it works | ||
|  | ||
| ### 1. Blockchain Connection | ||
| - Initializes cryptography | ||
| - Connects to node via WebSocket | ||
| - Creates Alice account for testing | ||
|  | ||
| ### 2. Load Generation (if --tps > 0) | ||
| - Sends Alice → Alice transactions at specified frequency | ||
| - Uses `transferKeepAlive` for safe transfers | ||
| - Automatically manages nonce for each transaction | ||
| - Shows progress every 10 transactions | ||
|  | ||
| ### 3. TPS Monitoring | ||
| - Subscribes to new blocks | ||
| - Counts transactions in each block | ||
| - Calculates average TPS over last 10 blocks | ||
| - Shows statistics in real-time | ||
|  | ||
| ### 4. Statistics Output | ||
| ``` | ||
| Block #1234: 8 txs, 13.3 TPS (45s runtime) | ||
| ``` | ||
| Where: | ||
| - `#1234` - block number | ||
| - `8 txs` - number of transactions in block | ||
| - `13.3 TPS` - average TPS over last 10 blocks | ||
| - `45s runtime` - program runtime | ||
|  | ||
| ## 🏃♂️ Usage Examples | ||
|  | ||
| ### Node Performance Testing | ||
|  | ||
| ```bash | ||
| # Start with low load | ||
| node src/index.js --tps 5 | ||
|  | ||
| # Gradually increase load | ||
| node src/index.js --tps 10 | ||
| node src/index.js --tps 20 | ||
| node src/index.js --tps 50 | ||
| ``` | ||
|  | ||
| ### Monitoring Existing Network | ||
|  | ||
| ```bash | ||
| # Monitor TPS without generating load | ||
| node src/index.js --tps 0 --node ws://mainnet-node:9944 | ||
| ``` | ||
|  | ||
| ### Local Testing | ||
|  | ||
| ```bash | ||
| # Test local node | ||
| node src/index.js --node ws://localhost:9944 --tps 15 | ||
| ``` | ||
|  | ||
| ## 🔍 Differences from Legacy Script | ||
|  | ||
| ### ❌ Problems in original script: | ||
| 1. **Wrong transaction type**: `system.remark` instead of `balances.transfer` | ||
| 2. **Automatic nonce**: led to transaction rejections | ||
| 3. **Sending speed measurement**: instead of real TPS | ||
| 4. **One-time execution**: send batch and exit | ||
|  | ||
| ### ✅ Fixes in Simple TPS Monitor: | ||
| 1. **Balance transfers** - real money transfers | ||
| 2. **Automatic nonce** - proper handling of multiple transactions | ||
| 3. **Block reading** - real TPS measurement | ||
| 4. **Continuous operation** - configurable sending frequency | ||
| 5. **Block monitoring** - real-time analysis | ||
| 6. **Modular architecture** - clean separation of concerns | ||
|  | ||
| ## 🛠️ Troubleshooting | ||
|  | ||
| ### Transactions not going through: | ||
| - Check Alice account balance | ||
| - Ensure node is accessible and running | ||
| - Verify node URL is correct | ||
|  | ||
| ### Low TPS: | ||
| - Try reducing sending frequency (--tps) | ||
| - Check node load | ||
| - Ensure node doesn't limit TPS | ||
|  | ||
| ### Connection errors: | ||
| - Check node accessibility | ||
| - Ensure correct WebSocket URL is used | ||
| - Check network settings | ||
|  | ||
| ### Common Error Messages: | ||
| - `Priority is too low`: Transaction pool is full, reduce TPS | ||
| - `Transaction is outdated`: Nonce issue, restart the tool | ||
| - `Connection failed`: Check node URL and network | ||
|  | ||
| ## 📈 Load Testing Recommendations | ||
|  | ||
| ```bash | ||
| # 1. Start with monitoring without load | ||
| node src/index.js --tps 0 | ||
|  | ||
| # 2. Add minimal load | ||
| node src/index.js --tps 1 | ||
|  | ||
| # 3. Gradually increase load | ||
| node src/index.js --tps 5 | ||
| node src/index.js --tps 10 | ||
| node src/index.js --tps 20 | ||
|  | ||
| # 4. Find maximum load | ||
| node src/index.js --tps 50 | ||
| node src/index.js --tps 100 | ||
| ``` | ||
|  | ||
| ## 🎯 Result | ||
|  | ||
| Simple TPS Monitor provides: | ||
| - ✅ **Accurate TPS measurement** - based on blockchain data | ||
| - ✅ **Proper transaction sending** - with nonce increment | ||
| - ✅ **Real load** - balance transfers, not just messages | ||
| - ✅ **Configuration flexibility** - --tps parameter for load control | ||
| - ✅ **Ease of use** - single file, one command launch | ||
| - ✅ **Modular design** - clean architecture, easy to extend | ||
|  | ||
| **Result:** Reliable and accurate tool for measuring Polkadot/Substrate network performance. | ||
|  | ||
| ## 📋 Requirements for Testing | ||
|  | ||
| ### Node Setup | ||
| - Running Substrate/Polkadot node | ||
| - WebSocket endpoint available (default: ws://localhost:9944) | ||
| - Node configured for development/testing | ||
|  | ||
| ### Account Setup | ||
| - Alice account with sufficient balance | ||
| - Account should be able to send transactions | ||
| - Recommended: Use development node with pre-funded accounts | ||
|  | ||
| ## 🔧 Development | ||
|  | ||
| ### Dependencies | ||
| - `@polkadot/api` - Polkadot/Substrate API | ||
| - `@polkadot/util-crypto` - Cryptographic utilities | ||
| - `commander` - CLI argument parsing | ||
|  | ||
| ### Architecture Benefits | ||
|  | ||
| #### **Single Responsibility Principle (SRP)** | ||
| - Each class has one responsibility | ||
| - Easier to understand and maintain | ||
|  | ||
| #### **Testability** | ||
| ```javascript | ||
| // Can test components separately | ||
| const sender = new TransactionSender(api, keyring) | ||
| await sender.startSending(5) | ||
|  | ||
| const monitor = new BlockMonitor() | ||
| monitor.startMonitoring(api) | ||
| ``` | ||
|  | ||
| #### **Reusability** | ||
| - TransactionSender can be used in other projects | ||
| - BlockMonitor can be extended for other metrics | ||
|  | ||
| #### **Extensibility** | ||
| - Easy to add new transaction types | ||
| - Can add new metrics to BlockMonitor | ||
|  | ||
| ## 📄 License | ||
|  | ||
| This project is unlicensed. All rights reserved. | ||
|  | ||
| ## 🤝 Contributing | ||
|  | ||
| 1. Fork the repository | ||
| 2. Create your feature branch | ||
| 3. Commit your changes | ||
| 4. Push to the branch | ||
| 5. Create a Pull Request | ||
|  | ||
| --- | ||
|  | ||
| **Note:** This tool is designed for testing and development purposes. Use responsibly and ensure you have proper permissions for the target network. | ||
      
      Oops, something went wrong.
        
    
  
  Add this suggestion to a batch that can be applied as a single commit.
  This suggestion is invalid because no changes were made to the code.
  Suggestions cannot be applied while the pull request is closed.
  Suggestions cannot be applied while viewing a subset of changes.
  Only one suggestion per line can be applied in a batch.
  Add this suggestion to a batch that can be applied as a single commit.
  Applying suggestions on deleted lines is not supported.
  You must change the existing code in this line in order to create a valid suggestion.
  Outdated suggestions cannot be applied.
  This suggestion has been applied or marked resolved.
  Suggestions cannot be applied from pending reviews.
  Suggestions cannot be applied on multi-line comments.
  Suggestions cannot be applied while the pull request is queued to merge.
  Suggestion cannot be applied right now. Please check back later.
  
    
  
    
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add
.nvmrc.