Skip to content

Commit 03bdced

Browse files
authored
refactor: remove sglang config and util files from components (#3467)
Signed-off-by: Anant Sharma <[email protected]>
1 parent 51f6575 commit 03bdced

File tree

7 files changed

+6
-332
lines changed

7 files changed

+6
-332
lines changed

components/backends/sglang/benchmarks/bench.sh

Lines changed: 0 additions & 175 deletions
This file was deleted.

components/backends/sglang/benchmarks/generate_bench_data.py

Lines changed: 0 additions & 88 deletions
This file was deleted.

components/backends/sglang/docs/dsr1-wideep-gb200.md

Lines changed: 2 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -56,13 +56,7 @@ docker run \
5656
dynamo-wideep-gb200:latest
5757
```
5858

59-
3. On the head prefill node, run the helper script provided to generate commands to start the `nats-server`, `etcd`. This script will also tell you which environment variables to export on each node to make deployment easier.
60-
61-
```bash
62-
./utils/gen_env_vars.sh
63-
```
64-
65-
4. Run the ingress and prefill worker
59+
3. Run the ingress and prefill worker
6660

6761
```bash
6862
# run ingress
@@ -115,7 +109,7 @@ python3 -m dynamo.sglang \
115109
--log-level debug
116110
```
117111

118-
5. Run the decode worker on the head decode node
112+
4. Run the decode worker on the head decode node
119113

120114
```bash
121115
SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK=768 \

components/backends/sglang/docs/dsr1-wideep-h100.md

Lines changed: 2 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -41,13 +41,7 @@ docker run \
4141

4242
In each container, you should be in the `/sgl-workspace/dynamo/components/backends/sglang` directory.
4343

44-
3. On the head prefill node, run the helper script provided to generate commands to start the `nats-server`, `etcd`. This script will also tell you which environment variables to export on each node to make deployment easier.
45-
46-
```bash
47-
./utils/gen_env_vars.sh
48-
```
49-
50-
4. Run the ingress and prefill worker
44+
3. Run the ingress and prefill worker
5145

5246
```bash
5347
# run ingress
@@ -87,7 +81,7 @@ python3 -m dynamo.sglang \
8781

8882
On the other prefill node (since this example has 4 total prefill nodes), run the same command but change `--node-rank` to 1,2, and 3
8983

90-
5. Run the decode worker on the head decode node
84+
4. Run the decode worker on the head decode node
9185

9286
```bash
9387
python3 -m dynamo.sglang \

components/backends/sglang/docs/multinode-examples.md

Lines changed: 2 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -18,12 +18,7 @@ docker build -f container/Dockerfile.sglang-wideep . -t dynamo-wideep --no-cache
1818

1919
You can use a specific tag from the [lmsys dockerhub](https://hub.docker.com/r/lmsysorg/sglang/tags) by adding `--build-arg SGLANG_IMAGE_TAG=<tag>` to the build command.
2020

21-
**Step 1**: Use the provided helper script to generate commands to start NATS/ETCD on your head prefill node. This script will also give you environment variables to export on each other node. You will need the IP addresses of your head prefill and head decode node to run this script.
22-
```bash
23-
./utils/gen_env_vars.sh
24-
```
25-
26-
**Step 2**: Ensure that your configuration file has the required arguments. Here's an example configuration that runs prefill and the model in TP16:
21+
**Step 1**: Ensure that your configuration file has the required arguments. Here's an example configuration that runs prefill and the model in TP16:
2722

2823
Node 1: Run HTTP ingress, processor, and 8 shards of the prefill worker
2924
```bash
@@ -104,7 +99,7 @@ python3 -m dynamo.sglang \
10499
--mem-fraction-static 0.82
105100
```
106101

107-
**Step 3**: Run inference
102+
**Step 2**: Run inference
108103
SGLang typically requires a warmup period to ensure the DeepGEMM kernels are loaded. We recommend running a few warmup requests and ensuring that the DeepGEMM kernels load in.
109104

110105
```bash

components/backends/sglang/utils/gen_env_vars.sh

Lines changed: 0 additions & 46 deletions
This file was deleted.

components/backends/sglang/configs/deepseek_r1/wideep/deepep.json renamed to recipes/deepseek-r1/sglang-wideep/deepep.json

File renamed without changes.

0 commit comments

Comments
 (0)