Skip to content

Commit d4a2a14

Browse files
authored
Merge pull request #111 from ricj/master
figure
2 parents bc83278 + 069aac1 commit d4a2a14

File tree

1 file changed

+3
-2
lines changed

1 file changed

+3
-2
lines changed

_pages/dat450/assignment2.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -31,9 +31,10 @@ Copy the skeleton from SOMEWHERE.
3131

3232
## Step 1: Setting up a Transformer neural network
3333

34-
To be fully compatible with the Olmo 2 implementation, note that `nn.Linear` inside of all layers are bias-free (`bias=False`).
3534

36-
<img src="https://raw.githubusercontent.com/ricj/dsai-nlp.github.io/refs/heads/master/_pages/dat450/olmo2_overview.svg" alt="Olmo2 overview" style="width:30%; height:auto;">
35+
<img src="https://raw.githubusercontent.com/ricj/dsai-nlp.github.io/refs/heads/master/_pages/dat450/olmo2_overview.svg" alt="Olmo2 overview" style="width:10%; height:auto;">
36+
37+
To be fully compatible with the Olmo 2 implementation, note that all the `nn.Linear` inside of all layers are bias-free (`bias=False`). This includes Q, K, V, and O projections inside attention layers, all parts of the MLP layers, and the unembedding layer.
3738

3839
### Configuration
3940

0 commit comments

Comments
 (0)