[feat] make warmup timeout configurable

[billishyahao]

Motivation

Remove the hardcode timeout during warming up by introducing

parser.add_argument(
        "--warmup-timeout",
        type=float,
        default=ServerArgs.warmup_timeout,
        help="Set warmup timeout in seconds. If a warmup forward batch takes longer than this, the server will crash to prevent hanging. Recommend to increase warmup timeout to 1800 to accommodate some kernel JIT precache e.g. deep gemm",
    )

Modifications

Accuracy Tests

Benchmarking and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[ShangmingCai]

Changes are clean. But I think maybe we can bypass timeout caused by deepgemm compile through --skip-server-warmup already?

[billishyahao]

Yes, @ShangmingCai Without this change, we can also workaround the issue through two ways:

• specify --skip-server-warmup to bypass the whole server warmup process
• AOT/prebuild the computation kernel e.g. python3 -m sglang.compile_deep_gemm --model deepseek-ai/DeepSeek-V3 --tp 8 --trust-remote-code for deep gemm

This patch aims to eliminate the hardcode in default code path to allow user to preserve the warmup through extending the timeout limitation

[hnyls2002]

This configuration is rarely used. Do not add it to ServerArgs as too many args for now. Put this into sglang.srt.environ and use an environment variable to configure the server warmup timeout.

[billishyahao]

This configuration is rarely used. Do not add it to ServerArgs as too many args for now. Put this into sglang.srt.environ and use an environment variable to configure the server warmup timeout.

Thanks for the advice. Let me propose a new PR to introduce new env variable in sglang.srt.environ rather than ServerArgs . Also we have observed random warmup timeout at the server side first time due to gemm library AOT.