Atari Breakout PPO Agent

This repository implements a Proximal Policy Optimization (PPO) agent to play Atari Breakout using PyTorch and Gymnasium. It includes training, evaluation, and video rendering utilities.

Features

• PPO Algorithm: Stable policy gradient method (ppo.py).
• Actor-Critic Model: CNN-based policy/value network (actor_critic.py).
• Frame Preprocessing: Cropping, resizing, grayscale conversion.
• Training Script: Train agent and save checkpoints (main.py, main.ipynb).
• Evaluation Script: Run trained agent interactively (test.py).
• Video Rendering: Generate crisp, squashed, slow-motion videos (make_best_video.py).
• Best Model: Pretrained weights at models/best_model.pth.
• Best Run Video: Example gameplay at videos/best_video.mp4.

Getting Started

Requirements

• Python 3.10+
• PyTorch
• Gymnasium
• ALE-py
• OpenCV
• imageio

Install dependencies:

pip install torch gymnasium[atari] ale-py opencv-python imageio

Training

To train the agent:

python main.py

or use main.ipynb for interactive training.

Checkpoints are saved in models/.

Evaluation

To run the trained agent:

python test.py

This loads models/best_model.pth and plays Breakout.

Video Generation

To render a video of the best run:

python make_best_video.py

Output is saved to videos/ (see videos/best_video.mp4).

File Overview

• ppo.py: PPO algorithm implementation (PPO).
• actor_critic.py: CNN actor-critic model (ActorCritic).
• main.py: Training loop.
• main.ipynb: Notebook for training/experimentation.
• test.py: Evaluation script.
• make_best_video.py: Video rendering utility.
• models/: Saved model checkpoints.
• videos/: Rendered gameplay videos.

Example Results

• Best Model: models/best_model.pth
• Best Video: videos/best_video.mp4

License

MIT License

---

References:

• OpenAI PPO Paper
• Gymnasium
• ALE-py