Skip to content

Commit 2ce9b77

Browse files
nihuiCopilot
andauthored
Update src/layer/sdpa.cpp
Co-authored-by: Copilot <[email protected]>
1 parent 69b041a commit 2ce9b77

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/layer/sdpa.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -287,7 +287,7 @@ int SDPA::forward_int8(const std::vector<Mat>& bottom_blobs, std::vector<Mat>& t
287287
const int cur_seqlen = cur_key.h;
288288
const int num_group = cur_key.c;
289289
const int out_embed_dim = cur_value.w;
290-
const int past_seqlen = past_key.h;
290+
const int past_seqlen = kv_cache ? past_key.h : 0;
291291
const int dst_seqlen = past_seqlen + cur_seqlen;
292292

293293
// assert cur_key.w == embed_dim

0 commit comments

Comments
 (0)