v0.5 and quickfixes

README.md

@@ -1,42 +1,67 @@
+<div align="center">
+  <img src="assets/logo.png" alt="Memoraith Logo" width="600"/>
+
 # Memoraith
 
-Memoraith is a cutting-edge, lightweight model profiler for deep learning frameworks, developed by Mehdi El Jouhfi. It's designed to revolutionize the optimization of neural network models by providing unparalleled insights into their performance characteristics.
+[![PyPI version](https://badge.fury.io/py/memoraith.svg)](https://badge.fury.io/py/memoraith)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Python 3.7+](https://img.shields.io/badge/python-3.7+-blue.svg)](https://www.python.org/downloads/)
+[![Documentation Status](https://readthedocs.org/projects/memoraith/badge/?version=latest)](https://memoraith.readthedocs.io/en/latest/?badge=latest)
+
+**Advanced Lightweight Model Profiler for Deep Learning**
+</div>
+
+## Overview
+
+Memoraith is a cutting-edge, lightweight model profiler for deep learning frameworks, providing unparalleled insights into neural network performance. Developed with precision and efficiency in mind, it helps developers and researchers optimize their models through detailed performance analysis.
+
+## ✨ Key Features
 
-## Features
+- 🔍 **Advanced Profiling**
+    - High-precision memory tracking (CPU & GPU)
+    - Microsecond-accurate computation timing
+    - Layer-by-layer performance analysis
 
-- Advanced support for PyTorch and TensorFlow models
-- High-precision profiling of memory usage (CPU and GPU)
-- Microsecond-accurate computation time measurement for each layer
-- Sophisticated bottleneck and anomaly detection algorithms
-- Generation of comprehensive, interactive reports with advanced visualizations
-- Real-time visualization capabilities with minimal overhead
-- Flexible programmatic and command-line interfaces
+- 🎯 **Intelligent Analysis**
+    - Sophisticated bottleneck detection
+    - Anomaly identification
+    - Optimization recommendations
 
-## Installation
+- 📊 **Rich Visualization**
+    - Interactive dashboards
+    - Real-time monitoring
+    - Comprehensive reports
 
-Install Memoraith using pip:
+- 🛠 **Framework Support**
+    - PyTorch integration
+    - TensorFlow support
+    - Extensible architecture
 
+## 🚀 Installation
+
+Basic installation:
 ```bash
 pip install memoraith
 ```
 
-For GPU support and additional features:
-
+Full installation with GPU support and extra features:
 ```bash
 pip install memoraith[full]
 ```
 
-## Quick Start
+## 🎮 Quick Start
 
-Here's an example of Memoraith in action with a PyTorch model:
+Here's a simple example using PyTorch:
 
 ```python
 from memoraith import profile_model, set_output_path
 import torch
 import torch.nn as nn
 
+# Set output directory for profiling results
 set_output_path('profiling_results/')
 
+# Define your model
 class AdvancedNet(nn.Module):
     def __init__(self):
         super(AdvancedNet, self).__init__()
@@ -50,7 +75,8 @@ class AdvancedNet(nn.Module):
         x = x.view(x.size(0), -1)
         return self.fc(x)
 
-@profile_model(memory=True, computation=True, gpu=True, network=True)
+# Add profiling decorator
+@profile_model(memory=True, computation=True, gpu=True)
 def train_model(model):
     optimizer = torch.optim.Adam(model.parameters())
     for _ in range(100):
@@ -65,37 +91,52 @@ if __name__ == "__main__":
     train_model(model)
 ```
 
-This will generate a comprehensive profiling report in the 'profiling_results/' directory.
+## 📚 Documentation
 
-## Documentation
+Visit our [comprehensive documentation](https://memoraith.readthedocs.io) for:
+- Detailed API reference
+- Advanced usage examples
+- Best practices
+- Troubleshooting guides
 
-For detailed information on Memoraith's advanced features, please refer to our [comprehensive documentation](https://memoraith.readthedocs.io).
+## 🤝 Contributing
 
-## Contributing
+We welcome contributions! See our [Contributing Guide](CONTRIBUTING.md) for:
+- Code of conduct
+- Development setup
+- Submission guidelines
+- Testing procedures
 
-Contributions are welcome! Please read our [Contributing Guide](CONTRIBUTING.md) for details on our code of conduct and the process for submitting pull requests.
+## 📝 License
 
-## License
+Memoraith is released under the MIT License. See [LICENSE](LICENSE) file for details.
 
-Memoraith is released under the MIT License. See the [LICENSE](LICENSE) file for more details.
+## 🆘 Support
 
-## Support
+Need help?
+- 📋 [GitHub Issues](https://github.com/mehdi342/Memoraith/issues)
+- 📚 [Documentation](https://memoraith.readthedocs.io)
+- 📧 [Email Support](mailto:[email protected])
 
-If you encounter any issues or have questions, please file an issue on the [GitHub issue tracker](https://github.com/mehdi342/Memoraith/issues).
+## 📖 Citation
 
-## Citing Memoraith
-
-If you use Memoraith in your research, please cite it as follows:
+If you use Memoraith in your research, please cite:
 
 ```bibtex
 @software{memoraith,
-  author = {El Jouhfi, Mehdi},
-  title = {Memoraith: Advanced Lightweight Model Profiler for Deep Learning},
-  year = {2024},
-  url = {https://github.com/mehdi342/Memoraith}
+    author = {El Jouhfi, Mehdi},
+    title = {Memoraith: Advanced Lightweight Model Profiler for Deep Learning},
+    year = {2024},
+    url = {https://github.com/mehdi342/Memoraith},
+    version = {0.5.0}
 }
 ```
 
-## Contact
+## 📬 Contact
+
+For inquiries, reach out to [Mehdi El Jouhfi](mailto:[email protected])
 
-For inquiries, please contact Mehdi El Jouhfi at [email protected].
+---
+<div align="center">
+  Made with ❤️ and sweat by Mehdi El Jouhfi
+</div>

memoraith/analysis/analyzer.py

@@ -1,71 +1,101 @@
-import logging
+# memoraith/analysis/analyzer.py
 from typing import Dict, Any, List
+import logging
+import numpy as np
 from .metrics import MetricsCalculator
 from .bottleneck import BottleneckDetector
 from .recommendations import RecommendationEngine
 from .anomaly_detection import AnomalyDetector
 
-class AnalyzerManager:
+class Analyzer:
+    """Main analysis coordinator for profiling data."""
+
     def __init__(self, profiling_data: Dict[str, Any]):
         self.data = profiling_data
         self.logger = logging.getLogger(__name__)
-        self.metrics = MetricsCalculator(profiling_data)
+        self.metrics_calculator = MetricsCalculator(profiling_data)
         self.bottleneck_detector = BottleneckDetector()
         self.anomaly_detector = AnomalyDetector()
         self.recommendation_engine = RecommendationEngine()
 
-    async def analyze(self) -> Dict[str, Any]:
-        """Run complete analysis on profiling data"""
+    async def run_analysis(self) -> Dict[str, Any]:
+        """Run complete analysis pipeline."""
         try:
-            # Calculate basic metrics
-            metrics = await self.metrics.calculate()
+            metrics = await self.metrics_calculator.calculate()
+            if not metrics:
+                raise ValueError("No metrics calculated")
 
-            # Detect issues and generate recommendations
             bottlenecks = await self.bottleneck_detector.detect(metrics)
             anomalies = await self.anomaly_detector.detect(metrics)
             recommendations = await self.recommendation_engine.generate(metrics)
-
-            # Calculate performance score
-            score = self._calculate_performance_score(metrics)
+            performance_score = self._calculate_performance_score(metrics)
 
             return {
                 'metrics': metrics,
                 'bottlenecks': bottlenecks,
                 'anomalies': anomalies,
                 'recommendations': recommendations,
-                'performance_score': score
+                'performance_score': performance_score,
+                'summary': await self._generate_summary(metrics)
             }
         except Exception as e:
             self.logger.error(f"Analysis failed: {str(e)}")
             raise
 
     def _calculate_performance_score(self, metrics: Dict[str, Any]) -> float:
-        """Calculate overall performance score"""
+        """Calculate overall performance score."""
         try:
-            # Simple scoring based on key metrics
-            memory_score = 100 - min(metrics.get('peak_memory_percent', 0), 100)
-            cpu_score = 100 - min(metrics.get('peak_cpu_percent', 0), 100)
-            network_score = 100 - min(metrics.get('network_utilization_percent', 0), 100)
+            # Weight factors for different metrics
+            weights = {
+                'memory': 0.4,
+                'compute': 0.4,
+                'efficiency': 0.2
+            }
+
+            # Memory score (lower is better)
+            memory_usage = metrics.get('peak_memory_percent', 0)
+            memory_score = max(0, 100 - memory_usage)
 
-            # Average the scores
-            return (memory_score + cpu_score + network_score) / 3
+            # Compute score (based on time efficiency)
+            compute_time = metrics.get('total_time', 0)
+            compute_score = 100 * np.exp(-compute_time / 10)  # Exponential decay
+
+            # Efficiency score (based on resource utilization)
+            efficiency_score = metrics.get('resource_efficiency', 80)
+
+            # Calculate weighted score
+            final_score = (
+                    weights['memory'] * memory_score +
+                    weights['compute'] * compute_score +
+                    weights['efficiency'] * efficiency_score
+            )
+
+            return min(100, max(0, final_score))
         except Exception as e:
-            self.logger.error(f"Score calculation failed: {str(e)}")
+            self.logger.error(f"Error calculating performance score: {str(e)}")
             return 0.0
 
-    async def get_summary(self) -> Dict[str, Any]:
-        """Get a concise summary of the analysis"""
+    async def _generate_summary(self, metrics: Dict[str, Any]) -> Dict[str, Any]:
+        """Generate a concise summary of analysis results."""
+        return {
+            'total_time': metrics.get('total_time', 0),
+            'peak_memory': metrics.get('peak_memory', 0),
+            'gpu_utilization': metrics.get('gpu_utilization', 0),
+            'bottleneck_count': len(metrics.get('bottlenecks', [])),
+            'anomaly_count': len(metrics.get('anomalies', [])),
+            'recommendation_count': len(metrics.get('recommendations', []))
+        }
+
+    async def export_results(self, format: str = 'json') -> Dict[str, Any]:
+        """Export analysis results in specified format."""
         try:
-            analysis = await self.analyze()
-            return {
-                'performance_score': analysis['performance_score'],
-                'bottleneck_count': len(analysis['bottlenecks']),
-                'anomaly_count': len(analysis['anomalies']),
-                'recommendation_count': len(analysis['recommendations']),
-                'peak_memory': analysis['metrics'].get('peak_memory', 0),
-                'peak_cpu': analysis['metrics'].get('peak_cpu', 0),
-                'total_time': analysis['metrics'].get('total_time', 0)
-            }
+            results = await self.run_analysis()
+            if format == 'json':
+                return results
+            elif format == 'summary':
+                return await self._generate_summary(results['metrics'])
+            else:
+                raise ValueError(f"Unsupported export format: {format}")
         except Exception as e:
-            self.logger.error(f"Summary generation failed: {str(e)}")
-            return {}
+            self.logger.error(f"Error exporting results: {str(e)}")
+            raise

memoraith/data_collection/base_collector.py

@@ -0,0 +1,88 @@
+# memoraith/data_collection/base_collector.py
+from abc import ABC, abstractmethod
+from typing import Dict, Any, Optional
+import asyncio
+import logging
+from ..exceptions import DataCollectionError
+
+class BaseDataCollector(ABC):
+    """Base class for all data collectors."""
+
+    def __init__(self, interval: float = 0.1):
+        self.interval = interval
+        self.logger = logging.getLogger(__name__)
+        self.data: Dict[str, Any] = {}
+        self._is_collecting = False
+        self._task: Optional[asyncio.Task] = None
+        self._lock = asyncio.Lock()
+
+    async def start(self) -> None:
+        """Start data collection."""
+        try:
+            async with self._lock:
+                if self._is_collecting:
+                    raise DataCollectionError("Collection already running")
+                self._is_collecting = True
+                self._task = asyncio.create_task(self._collection_loop())
+                self.logger.info(f"{self.__class__.__name__} started collecting data")
+        except Exception as e:
+            self.logger.error(f"Failed to start collection: {str(e)}")
+            raise DataCollectionError(f"Start failed: {str(e)}")
+
+    async def stop(self) -> None:
+        """Stop data collection."""
+        try:
+            async with self._lock:
+                if not self._is_collecting:
+                    return
+                self._is_collecting = False
+                if self._task:
+                    self._task.cancel()
+                    try:
+                        await self._task
+                    except asyncio.CancelledError:
+                        pass
+                    self._task = None
+                self.logger.info(f"{self.__class__.__name__} stopped collecting data")
+        except Exception as e:
+            self.logger.error(f"Failed to stop collection: {str(e)}")
+            raise DataCollectionError(f"Stop failed: {str(e)}")
+
+    @abstractmethod
+    async def _collect_data(self) -> Dict[str, Any]:
+        """Collect a single data point."""
+        raise NotImplementedError
+
+    async def _collection_loop(self) -> None:
+        """Main collection loop."""
+        while self._is_collecting:
+            try:
+                data_point = await self._collect_data()
+                async with self._lock:
+                    self._process_data_point(data_point)
+                await asyncio.sleep(self.interval)
+            except asyncio.CancelledError:
+                break
+            except Exception as e:
+                self.logger.error(f"Error in collection loop: {str(e)}")
+
+    def _process_data_point(self, data_point: Dict[str, Any]) -> None:
+        """Process and store a collected data point."""
+        timestamp = data_point.get('timestamp', 0)
+        if timestamp not in self.data:
+            self.data[timestamp] = data_point
+
+    async def get_data(self) -> Dict[str, Any]:
+        """Get collected data."""
+        async with self._lock:
+            return self.data.copy()
+
+    async def clear_data(self) -> None:
+        """Clear collected data."""
+        async with self._lock:
+            self.data.clear()
+
+    @abstractmethod
+    async def validate_data(self) -> bool:
+        """Validate collected data."""
+        raise NotImplementedError