We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent 01a634d commit 262cce7Copy full SHA for 262cce7
examples/backends/vllm/launch/agg_kvbm_router.sh
@@ -32,7 +32,7 @@ CUDA_VISIBLE_DEVICES=0 DYN_KVBM_CPU_CACHE_GB=2 \
32
DYN_KVBM_LEADER_ZMQ_PUB_PORT=56003 \
33
DYN_KVBM_LEADER_ZMQ_ACK_PORT=56004 \
34
VLLM_NIXL_SIDE_CHANNEL_PORT=20097 \
35
-CUDA_VISIBLE_DEVICES=0 DYN_KVBM_CPU_CACHE_GB=2 \
+CUDA_VISIBLE_DEVICES=1 DYN_KVBM_CPU_CACHE_GB=2 \
36
python3 -m dynamo.vllm \
37
--model $MODEL \
38
--enforce-eager \
0 commit comments