This repository contains code for training and evaluating a Lunar Lander agent using the Proximal Policy Optimization (PPO) algorithm.
- Python 3.x
- TensorFlow
- Stable Baselines3
- Gymnasium
-
Clone this repository:
git clone https://github.com/yourusername/LunarLander-PPO.git cd Lunar-Lander-Using-PPO
-
Install the required libraries using the environment.yml file using conda :
conda env create -f environment.yml
The main script sets up the environment, trains the model using PPO, evaluates the model, and saves the trained model.
import os
os.environ['KMP_DUPLICATE_LIB_OK'] = 'TRUE'
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.optimizers import Adam
from stable_baselines3 import PPO
from stable_baselines3.common.vec_env import DummyVecEnv
from stable_baselines3.common.evaluation import evaluate_policy
import gymnasium as gym
LOAD = 0
# Setting up our environment
env = gym.make("LunarLander-v2", render_mode="human")
env = DummyVecEnv([lambda: env])
# Setting up the model
model = PPO("MlpPolicy", env, verbose = 1)
# Uncomment the following lines to load a pre-trained model
# if LOAD == 1:
# model = PPO.load("ppo_model.zip")
# Training the model
model.learn(total_timesteps=100000)
# Evaluating our model
evaluate_policy(model, env, n_eval_episodes=10, render=True)
# Saving the model
model.save("ppo_model.zip")
# Closing the environment
env.close()
For comparison, a random action implementation is provided to understand the difference between a trained agent and a random policy.
import os
os.environ['KMP_DUPLICATE_LIB_OK'] = 'TRUE'
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.optimizers import Adam
from stable_baselines3 import PPO
from stable_baselines3.common.vec_env import DummyVecEnv
from stable_baselines3.common.evaluation import evaluate_policy
import gymnasium as gym
import numpy as np
from collections import deque
import random
episodes = 10
for episode in range(0, episodes):
state = env.reset()
done = False
score = 0
while not done:
env.render()
action = env.action_space.sample()
obs, reward, done, truncated, info = env.step(action)
score += reward
print(f"Episode: {episode} Score: {score}")
env.close()
-
To train and evaluate the PPO model, simply run the main script:
python lunar_lander_deepQ.py
-
If you want to see the performance of a random action policy:
python random_aciton_lunar_lander.py
- The script trains a PPO model for the Lunar Lander environment and evaluates its performance.
- The model is saved as
ppo_model.zip
after training. - The performance is evaluated over 10 episodes, and the average score is printed.
- Tune hyperparameters for better performance.
- Experiment with different algorithms available in Stable Baselines3.
- Implement additional evaluation metrics and visualizations.
- Sadegh Khedery
This project is based on the tutorial by Nicholas Renotte on training a Lunar Lander agent. You can find the tutorial here https://www.youtube.com/watch?v=nRHjymV2PX8&t=551s .
This project is licensed under the Apache-2.0 License - see the LICENSE.md file for details.