Skip to content

Commit c710e7a

Browse files
committed
Update README, add instructions
1 parent baede37 commit c710e7a

File tree

1 file changed

+157
-1
lines changed

1 file changed

+157
-1
lines changed

README.md

Lines changed: 157 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,159 @@
11
# minitorch
22

3-
This is a mini, torch-like deep learning library. The goal is that you can use syntax similar to PyTorch to build deep learning model using this library. This project starting points is the [minitorch exercises](https://github.com/minitorch/minitorch). However, after completing the exercises, I want to turn it into a functional deep learning library that can utilize GPU for model training, and I also want to refactor the code to make it clearer. This project is my attempt to do so.
3+
This is a mini, torch-like deep learning library. The goal is that you can use syntax similar to PyTorch to build deep learning model using this library. This project starting points is the [minitorch exercises](https://github.com/minitorch/minitorch). However, after completing the exercises, I want to turn it into a functional deep learning library that can utilize GPU for model training, and I also want to refactor the code to make it clearer. This project is my attempt to do so.
4+
5+
## Installation
6+
7+
```bash
8+
pip install -r requirements.txt
9+
```
10+
11+
## Usage
12+
13+
### Basic Tensor Operations
14+
15+
```python
16+
import minitorch
17+
18+
# Create tensors with different backends
19+
FastTensorBackend = minitorch.TensorBackend(minitorch.FastOps)
20+
GPUBackend = minitorch.TensorBackend(minitorch.CudaOps) # If CUDA is available
21+
22+
# Create a tensor
23+
x = minitorch.tensor([1, 2, 3, 4], backend=FastTensorBackend)
24+
y = minitorch.tensor([5, 6, 7, 8], backend=FastTensorBackend)
25+
26+
# Perform operations
27+
z = x + y
28+
result = z.sum()
29+
```
30+
31+
### Building Neural Networks
32+
33+
Create custom models by subclassing `minitorch.Module`:
34+
35+
```python
36+
class Network(minitorch.Module):
37+
def __init__(self, backend):
38+
super().__init__()
39+
self.fc1 = minitorch.Linear(784, 128, backend=backend)
40+
self.fc2 = minitorch.Linear(128, 10, backend=backend)
41+
42+
def forward(self, x):
43+
x = self.fc1(x).relu()
44+
x = minitorch.dropout(x, 0.2, not self.training)
45+
x = self.fc2(x)
46+
return minitorch.logsoftmax(x, dim=1)
47+
```
48+
49+
### Convolutional Neural Networks
50+
51+
```python
52+
class CNN(minitorch.Module):
53+
def __init__(self, backend):
54+
super().__init__()
55+
self.conv1 = minitorch.Conv2d(in_channels=1, out_channels=6, kernel=(5, 5), stride=1, backend=backend)
56+
self.conv2 = minitorch.Conv2d(in_channels=6, out_channels=16, kernel=(5, 5), stride=1, backend=backend)
57+
self.fc1 = minitorch.Linear(16 * 4 * 4, 120, backend=backend)
58+
self.fc2 = minitorch.Linear(120, 10, backend=backend)
59+
60+
def forward(self, x):
61+
batch_size = x.shape[0]
62+
x = self.conv1(x).relu()
63+
x = minitorch.avgpool2d(x, kernel=(2, 2), stride=(2, 2))
64+
x = self.conv2(x).relu()
65+
x = minitorch.avgpool2d(x, kernel=(2, 2), stride=(2, 2))
66+
x = x.view(batch_size, -1)
67+
x = self.fc1(x).relu()
68+
x = self.fc2(x)
69+
return x
70+
```
71+
72+
### Training Loop
73+
74+
```python
75+
# Initialize model and optimizer
76+
model = Network(backend=FastTensorBackend)
77+
optimizer = minitorch.RMSProp(model.parameters(), lr=0.01)
78+
79+
# Training
80+
model.train()
81+
for epoch in range(num_epochs):
82+
for X_batch, y_batch in train_loader:
83+
optimizer.zero_grad()
84+
output = model(X_batch)
85+
loss = minitorch.nll_loss(output, y_batch)
86+
loss.backward()
87+
optimizer.step()
88+
89+
# Evaluation
90+
model.eval()
91+
for X_batch, y_batch in val_loader:
92+
output = model(X_batch)
93+
predictions = minitorch.argmax(output, dim=1)
94+
```
95+
96+
### Data Loading
97+
98+
```python
99+
from minitorch.datasets import mnist
100+
from minitorch.dataloader import DataLoader
101+
102+
# Load dataset
103+
mnist_train = mnist.MNISTDataset("/path/to/data", train=True)
104+
105+
# Create dataloader
106+
train_loader = DataLoader(
107+
mnist_train,
108+
batch_size=32,
109+
shuffle=True,
110+
backend=FastTensorBackend,
111+
transform=lambda x: x.astype(np.float64) / 255.0
112+
)
113+
```
114+
115+
### GPU Acceleration
116+
117+
```python
118+
import numba
119+
120+
# Check CUDA availability and use GPU backend
121+
if numba.cuda.is_available():
122+
backend = minitorch.TensorBackend(minitorch.CudaOps)
123+
print("Using GPU backend")
124+
else:
125+
backend = minitorch.TensorBackend(minitorch.FastOps)
126+
print("Using CPU backend")
127+
128+
model = Network(backend=backend)
129+
```
130+
131+
### Saving and Loading Models
132+
133+
```python
134+
# Save model weights
135+
model.save_weights("model.npz")
136+
137+
# Load model weights
138+
model.load_weights("model.npz")
139+
```
140+
141+
### Available Optimizers
142+
143+
- `SGD(parameters, lr, momentum)` - Stochastic Gradient Descent with optional momentum
144+
- `RMSProp(parameters, lr, decay_rate, eps)` - RMSProp optimizer
145+
146+
### Available Loss Functions
147+
148+
- `nll_loss(output, target)` - Negative Log Likelihood Loss
149+
- `bce_loss(output, target)` - Binary Cross Entropy Loss
150+
- `cross_entropy_loss(output, target)` - Cross Entropy Loss
151+
- `mse_loss(output, target)` - Mean Squared Error Loss
152+
153+
### Example: Training MNIST
154+
155+
See `examples/run_mnist_multiclass.py` for a complete example of training a LeNet-5 CNN on MNIST:
156+
157+
```bash
158+
python examples/run_mnist_multiclass.py --backend gpu --batch_size 32 --epochs 10 --lr 0.01
159+
```

0 commit comments

Comments
 (0)