CartPole is a classic reinforcement learning environment from OpenAI Gym where the goal is to balance a pole on a moving cart. This repository includes:
- CartPole.py: A script for rendering and evaluating a trained DQN agent.
- CartPoleAlgorithm.py: Deep Q-Learning implementation to train the agent on the CartPole-v1 environment.
- Visualize_q_table.py: Tools for analyzing and visualizing training metrics like rewards, losses, and epsilon decay.
- ModelData/: A folder containing:
- Pre-trained DQN model (
cartpole_dqn_optimized.pth). - Training logs (
training_logs.json) for insights.
- Pre-trained DQN model (
"Master the art of balancing with Deep Q-Learning!"
- ✨ Features
- 🛠️ Tech Stack
- 📸 Screenshots
- ⚙️ Setup Instructions
- 🚨 Resource Warning
- 🎯 Target Audience
- 🤝 Contributing
- 🌟 Awesome Contributors
- 📜 License
- 🤖 Deep Q-Learning: Train a neural network to approximate the Q-value function for optimal policy learning.
- 📊 Training Logs: Analyze rewards, epsilon decay, and losses to understand the learning process.
- 📈 Visualization Tools: View training progress with clear and detailed graphs.
- 🖥️ Pre-Trained Model: Quickly test and render the pre-trained model for instant results.
- 💻 Modular Codebase: Separate scripts for training, evaluation, and visualization.
- Reinforcement Learning: OpenAI Gym
- Deep Learning: PyTorch
- Visualization: Matplotlib, NumPy
- Code Management: JSON for logs, Torch for saving models
- CartPole/: Folder containing:
CartPole.py: Script for rendering the trained agent and observing its performance.CartPoleAlgorithm.py: Core DQN training implementation.Visualize_q_table.py: Tools to visualize training metrics and analyze learning progress.
- ModelData/: Folder containing:
cartpole_dqn_optimized.pth: Pre-trained DQN model.training_logs.json: Saved training metrics for detailed analysis.
Here are visualizations showcasing the training process and results:
-
Total Rewards Per Episode
Visualizes the total rewards collected by the agent over episodes, showing trends and improvement over time.
-
Training Loss Over Time
Displays the loss curve over the course of training, providing insights into model convergence.
-
Epsilon Decay During Training
Highlights how the epsilon value decreases during training, balancing exploration and exploitation.
-
CartPole Agent in Action
Watch the trained agent perform in the CartPole-v1 environment as it successfully balances the pole.
- Clone the Repository
git clone https://github.com/alienx5499/CartPole.git
- Navigate to the Project Directory
cd CartPole - Install Dependencies
pip install -r requirements.txt
- Run Training Script
python CartPoleAlgorithm.py
- Visualize Training Metrics
python Visualize_q_table.py
- Render the Trained Agent
python CartPole.py
This project includes a pre-trained model trained for 10,000 episodes, which is resource-intensive and can be CPU/GPU consuming.
If you have limited hardware resources, we recommend:
- Training with fewer episodes (e.g., 2000–5000 episodes) to achieve reasonable performance.
- Ensure your system can handle training workloads to prevent hardware strain.
Alternative: If you prefer not to train the model, you can use the pre-trained model and logs provided in the ModelData/ folder, which contains data for 10,000 episodes.
- Reinforcement Learning Enthusiasts: Dive deep into Deep Q-Learning and OpenAI Gym.
- AI Researchers: Analyze and experiment with the classic CartPole environment.
- Students and Educators: Use as a learning tool for understanding reinforcement learning.
- Developers: Expand the repository with new features or algorithms.
We ❤️ open source! Contributions are welcome to make this project even better.
- Fork the repository.
- Create your feature branch.
git checkout -b feature/new-feature
- Commit your changes.
git commit -m "Add a new feature" - Push to the branch and open a pull request.
Refer to our CONTRIBUTING.md for detailed contribution guidelines.
This project is licensed under the MIT License. See the LICENSE file for details.
We value your input! Share your thoughts through GitHub Issues.
🔗 Visit the Live Demo | 📑 Explore Documentation
💡 Let's master the CartPole challenge together!
