feat: install Run:ai model streamer for vllm #4848

gtbai · 2025-12-10T09:12:33Z

Overview:

Install RunAI dependency for Dynamo vLLM so that we can use Run:ai model streamer to load models from local paths

Test

Build Dynamo image:

./container/build.sh --framework vllm

Set nats and ectd env vars:

export NATS_SERVER="nats://localhost:4222"
export ETCD_ENDPOINTS="http://localhost:2379"
export DYN_VLLM_KV_EVENT_PORT=20081

Start nats and ectd servers:

docker compose -f deploy/docker-compose.yml up -d

Run frontend and worker

./container/run.sh --framework vllm -it --mount-workspace
python -m dynamo.frontend

Run vLLM worker

./container/run.sh --framework vllm -it --mount-workspace
DYN_SYSTEM_PORT=9090 python3 -m dynamo.vllm \
  --model Qwen/Qwen3-VL-4B-Instruct-FP8 \
  --trust-remote-code \
  --enable-prefix-caching \
  --max-num-batched-tokens 512 \
  --download-dir /root/.cache/huggingface \
  --dyn-tool-call-parser hermes \
  --connector lmcache \
  --max-model-len 32768 \
  --load-format runai_streamer \
  --model-loader-extra-config '{"distributed":true, "concurrency":16}'

Without the change saw:

2025-12-05T08:01:42.893308Z ERROR core.run_engine_core: EngineCore failed to start.
Traceback (most recent call last):
  File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 833, in run_engine_core
    engine_core = EngineCoreProc(*args, **kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 606, in __init__
    super().__init__(
  File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 102, in __init__
    self.model_executor = executor_class(vllm_config)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/v1/executor/abstract.py", line 101, in __init__
    self._init_executor()
  File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/v1/executor/uniproc_executor.py", line 48, in _init_executor
    self.driver_worker.load_model()
  File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/v1/worker/gpu_worker.py", line 273, in load_model
    self.model_runner.load_model(eep_scale_up=eep_scale_up)
  File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/v1/worker/gpu_model_runner.py", line 3276, in load_model
    self.model = model_loader.load_model(
                 ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/model_executor/model_loader/base_loader.py", line 55, in load_model
    self.load_weights(model, model_config)
  File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/model_executor/model_loader/runai_streamer_loader.py", line 115, in load_weights
    self._get_weights_iterator(model_weights, model_config.revision)
  File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/model_executor/model_loader/runai_streamer_loader.py", line 100, in _get_weights_iterator
    hf_weights_files = self._prepare_weights(model_or_path, revision)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/model_executor/model_loader/runai_streamer_loader.py", line 82, in _prepare_weights
    hf_weights_files = list_safetensors(path=hf_folder)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/transformers_utils/runai_utils.py", line 39, in list_safetensors
    return runai_list_safetensors(path)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/utils/import_utils.py", line 157, in __call__
    return self.__getattr__("__call__")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/utils/import_utils.py", line 313, in __getattr__
    getattr(self.__module, f"{self.__attr_path}.{key}")
  File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/utils/import_utils.py", line 293, in __getattr__
    raise exc
  File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/utils/import_utils.py", line 286, in __getattr__
    importlib.import_module(name)
  File "/usr/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1324, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'runai_model_streamer'
(EngineCore_DP0 pid=170) Process EngineCore_DP0:
(EngineCore_DP0 pid=170) Traceback (most recent call last):
(EngineCore_DP0 pid=170)   File "/usr/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
(EngineCore_DP0 pid=170)     self.run()
(EngineCore_DP0 pid=170)   File "/usr/lib/python3.12/multiprocessing/process.py", line 108, in run
(EngineCore_DP0 pid=170)     self._target(*self._args, **self._kwargs)
(EngineCore_DP0 pid=170)   File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 846, in run_engine_core
(EngineCore_DP0 pid=170)     raise e
(EngineCore_DP0 pid=170)   File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 833, in run_engine_core
(EngineCore_DP0 pid=170)     engine_core = EngineCoreProc(*args, **kwargs)
(EngineCore_DP0 pid=170)                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=170)   File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 606, in __init__
(EngineCore_DP0 pid=170)     super().__init__(
(EngineCore_DP0 pid=170)   File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 102, in __init__
(EngineCore_DP0 pid=170)     self.model_executor = executor_class(vllm_config)
(EngineCore_DP0 pid=170)                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=170)   File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/v1/executor/abstract.py", line 101, in __init__
(EngineCore_DP0 pid=170)     self._init_executor()
(EngineCore_DP0 pid=170)   File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/v1/executor/uniproc_executor.py", line 48, in _init_executor
(EngineCore_DP0 pid=170)     self.driver_worker.load_model()
(EngineCore_DP0 pid=170)   File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/v1/worker/gpu_worker.py", line 273, in load_model
(EngineCore_DP0 pid=170)     self.model_runner.load_model(eep_scale_up=eep_scale_up)
(EngineCore_DP0 pid=170)   File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/v1/worker/gpu_model_runner.py", line 3276, in load_model
(EngineCore_DP0 pid=170)     self.model = model_loader.load_model(
(EngineCore_DP0 pid=170)                  ^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=170)   File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/model_executor/model_loader/base_loader.py", line 55, in load_model
(EngineCore_DP0 pid=170)     self.load_weights(model, model_config)
(EngineCore_DP0 pid=170)   File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/model_executor/model_loader/runai_streamer_loader.py", line 115, in load_weights
(EngineCore_DP0 pid=170)     self._get_weights_iterator(model_weights, model_config.revision)
(EngineCore_DP0 pid=170)   File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/model_executor/model_loader/runai_streamer_loader.py", line 100, in _get_weights_iterator
(EngineCore_DP0 pid=170)     hf_weights_files = self._prepare_weights(model_or_path, revision)
(EngineCore_DP0 pid=170)                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=170)   File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/model_executor/model_loader/runai_streamer_loader.py", line 82, in _prepare_weights
(EngineCore_DP0 pid=170)     hf_weights_files = list_safetensors(path=hf_folder)
(EngineCore_DP0 pid=170)                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=170)   File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/transformers_utils/runai_utils.py", line 39, in list_safetensors
(EngineCore_DP0 pid=170)     return runai_list_safetensors(path)
(EngineCore_DP0 pid=170)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=170)   File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/utils/import_utils.py", line 157, in __call__
(EngineCore_DP0 pid=170)     return self.__getattr__("__call__")
(EngineCore_DP0 pid=170)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=170)   File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/utils/import_utils.py", line 313, in __getattr__
(EngineCore_DP0 pid=170)     getattr(self.__module, f"{self.__attr_path}.{key}")
(EngineCore_DP0 pid=170)   File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/utils/import_utils.py", line 293, in __getattr__
(EngineCore_DP0 pid=170)     raise exc
(EngineCore_DP0 pid=170)   File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/utils/import_utils.py", line 286, in __getattr__
(EngineCore_DP0 pid=170)     importlib.import_module(name)
(EngineCore_DP0 pid=170)   File "/usr/lib/python3.12/importlib/__init__.py", line 90, in import_module
(EngineCore_DP0 pid=170)     return _bootstrap._gcd_import(name[level:], package, level)
(EngineCore_DP0 pid=170)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=170)   File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
(EngineCore_DP0 pid=170)   File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
(EngineCore_DP0 pid=170)   File "<frozen importlib._bootstrap>", line 1324, in _find_and_load_unlocked
(EngineCore_DP0 pid=170) ModuleNotFoundError: No module named 'runai_model_streamer'
[rank0]:[W1205 08:01:43.144571432 ProcessGroupNCCL.cpp:1524] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/workspace/components/src/dynamo/vllm/__main__.py", line 7, in <module>
    main()
  File "/workspace/components/src/dynamo/vllm/main.py", line 786, in main
    uvloop.run(worker())
  File "/opt/dynamo/venv/lib/python3.12/site-packages/uvloop/__init__.py", line 109, in run
    return __asyncio.run(
           ^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
  File "/opt/dynamo/venv/lib/python3.12/site-packages/uvloop/__init__.py", line 61, in wrapper
    return await main
           ^^^^^^^^^^
  File "/workspace/components/src/dynamo/vllm/main.py", line 105, in worker
    await init(runtime, config)
  File "/workspace/components/src/dynamo/vllm/main.py", line 484, in init
    ) = setup_vllm_engine(config, factory)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/components/src/dynamo/vllm/main.py", line 295, in setup_vllm_engine
    engine_client = AsyncLLM.from_vllm_config(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/utils/func_utils.py", line 116, in inner
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/v1/engine/async_llm.py", line 203, in from_vllm_config
    return cls(
           ^^^^
  File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/v1/engine/async_llm.py", line 133, in __init__
    self.engine_core = EngineCoreClient.make_async_mp_client(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/v1/engine/core_client.py", line 121, in make_async_mp_client
    return AsyncMPClient(*client_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/v1/engine/core_client.py", line 808, in __init__
    super().__init__(
  File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/v1/engine/core_client.py", line 469, in __init__
    with launch_core_engines(vllm_config, executor_class, log_stats) as (
  File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
    next(self.gen)
  File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/v1/engine/utils.py", line 907, in launch_core_engines
    wait_for_engine_startup(
  File "/opt/dynamo/venv/lib/python3.12/site-packages/vllm/v1/engine/utils.py", line 964, in wait_for_engine_startup
    raise RuntimeError(
RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}

With the change saw:

Loading safetensors using Runai Model Streamer:   0% Completed | 0/966 [00:00<?, ?it/s]
Loading safetensors using Runai Model Streamer:  30% Completed | 285/966 [00:02<00:05, 125.32it/s]
Loading safetensors using Runai Model Streamer:  55% Completed | 536/966 [00:12<00:11, 38.62it/s]
Loading safetensors using Runai Model Streamer:  61% Completed | 586/966 [00:16<00:12, 29.36it/s]
Loading safetensors using Runai Model Streamer:  73% Completed | 703/966 [00:25<00:12, 21.07it/s]
Loading safetensors using Runai Model Streamer:  81% Completed | 779/966 [00:30<00:09, 19.49it/s]
Loading safetensors using Runai Model Streamer:  87% Completed | 837/966 [00:35<00:07, 17.28it/s]
Loading safetensors using Runai Model Streamer:  91% Completed | 883/966 [00:39<00:05, 15.91it/s]
Loading safetensors using Runai Model Streamer:  95% Completed | 922/966 [00:42<00:02, 14.84it/s]
Loading safetensors using Runai Model Streamer:  99% Completed | 957/966 [00:44<00:00, 15.25it/s]
Loading safetensors using Runai Model Streamer: 100% Completed | 966/966 [00:44<00:00, 21.52it/s]
(EngineCore_DP0 pid=18845)
(EngineCore_DP0 pid=18845) [2025-12-06 01:09:24] INFO file_streamer.py:66: [RunAI Streamer] Overall time to stream 5.6 GiB of all files to cuda:0: 44.91s, 127.9 MiB/s

Chat completions API sanity check also ok:

❯ curl localhost:8000/v1/models | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   124  100   124    0     0  82173      0 --:--:-- --:--:-- --:--:--  121k
{
  "object": "list",
  "data": [
    {
      "id": "Qwen/Qwen3-VL-4B-Instruct-FP8",
      "object": "object",
      "created": 1764983425,
      "owned_by": "nvidia"
    }
  ]
}
❯ curl localhost:8000/v1/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-VL-4B-Instruct-FP8",
    "prompt": "Please write a detailed explanation about cloud computing, including its benefits, types of services (IaaS, PaaS, SaaS), major cloud providers, and security considerations. Make this explanation comprehensive and suitable for someone learning about cloud technology for the first time.",
    "max_tokens": 300
  }' | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2329  100  1953  100   376    538    103  0:00:03  0:00:03 --:--:--   642
{
  "id": "cmpl-f68ce7bf-eb59-4092-80e8-1fb61d3463ca",
  "choices": [
    {
      "text": " \n\n---\n\n**Title: A Comprehensive Guide to Cloud Computing for Beginners**\n\n---\n\n**Introduction**\n\nCloud computing has revolutionized how businesses and individuals access and manage computing resources. Instead of relying on local hardware, users can now leverage powerful, scalable, and on-demand computing resources over the internet. This guide will walk you through the basics of cloud computing, its benefits, the different types of services, major providers, and essential security considerations.\n\n---\n\n**What is Cloud Computing?**\n\nCloud computing refers to the delivery of computing services — including servers, storage, databases, networking, software, and analytics — over the internet (“the cloud”) rather than on local hardware. These services are typically managed by third-party providers who operate data centers around the world.\n\nThink of it as renting computing power and storage from a remote provider, rather than buying and maintaining your own servers and infrastructure.\n\n---\n\n**Key Benefits of Cloud Computing**\n\nCloud computing offers numerous advantages that make it an attractive solution for organizations of all sizes:\n\n1. **Scalability**  \n   You can easily scale up or down based on demand. For example, during peak traffic, you can instantly add more computing power, and during off-peak hours, you can reduce it.\n\n2. **Cost Efficiency**  \n   You pay only for what you use, eliminating the need for expensive upfront hardware investments. This is often called “pay-as-you-go” or “usage-based pricing.”\n\n3. **Accessibility and Flexibility**  \n   Cloud resources are accessible",
      "index": 0,
      "finish_reason": "length"
    }
  ],
  "created": 1764983461,
  "model": "Qwen/Qwen3-VL-4B-Instruct-FP8",
  "system_fingerprint": null,
  "object": "text_completion",
  "usage": {
    "prompt_tokens": 51,
    "completion_tokens": 300,
    "total_tokens": 351
  }

Summary by CodeRabbit

Chores
- Updated vLLM to include the runai extra, enabling runai integration support across installation scripts and project configuration.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

copy-pr-bot · 2025-12-10T09:12:38Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

github-actions · 2025-12-10T09:12:44Z

👋 Hi gtbai! Thank you for contributing to ai-dynamo/dynamo.

Just a reminder: The NVIDIA Test Github Validation CI runs an essential subset of the testing framework to quickly catch errors.Your PR reviewers may elect to test the changes comprehensively before approving your changes.

🚀

coderabbitai · 2025-12-10T09:15:28Z

Walkthrough

Added the runai extra specifier to vLLM package installations across two configuration files. Updated the vllm dependency from vllm[flashinfer]==$VERSION to vllm[flashinfer,runai]==$VERSION in both the shell installation script and pyproject.toml.

Changes

Cohort / File(s)	Change Summary
vLLM extras configuration `container/deps/vllm/install_vllm.sh`, `pyproject.toml`	Added `runai` extra specifier to vLLM dependency declarations, extending the extras from `[flashinfer]` to `[flashinfer,runai]` in both files

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

These are consistent, homogeneous dependency configuration updates with no logic changes or structural complexity.

Poem

🐰 Two config files, a change so light,
Adding runai makes the bundle right!
From [flashinfer] to [flashinfer,runai] we go,
The hoppy updater's done with a bow! 🎀

Pre-merge checks

✅ Passed checks (3 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Title check	✅ Passed	The PR title 'feat: install Run:ai model streamer for vllm' accurately describes the main change: adding the Run:ai model streamer (runai extra) to the vLLM installation across both shell script and pyproject.toml.
Description check	✅ Passed	The PR description comprehensively covers all template sections with clear context, detailed testing steps, error logs, and validation results.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Signed-off-by: Guangtong Bai <[email protected]>

rmccorm4 · 2025-12-11T01:29:17Z

/ok to test af221f0

rmccorm4 · 2025-12-11T01:29:41Z

Thanks for the contribution @gtbai !

athreesh · 2025-12-11T01:42:50Z

@ganeshku1 @nicolasnoble @itay for viz

gtbai · 2025-12-11T07:53:27Z

@rmccorm4 thanks for the review! I saw check Docker Build and Test / vllm (amd64) (push) failed on my previous commit, but seems due to an irrelevant error.

Thus I updated the branch hoping to retry the failed check but it did not get triggered.

Could you help rerun the checks? Thanks!

rmccorm4 · 2025-12-11T15:18:56Z

/ok to test b1a0b67

gtbai · 2025-12-11T18:29:43Z

@rmccorm4 some checks failed again, this time seems due to another transient failure : )
https://github.com/ai-dynamo/dynamo/actions/runs/20138041240/job/57796957668?pr=4848#step:6:746

gtbai requested review from a team as code owners December 10, 2025 09:12

pull-request-size bot added the size/XS label Dec 10, 2025

github-actions bot added the external-contribution Pull request is from an external contributor label Dec 10, 2025

gtbai changed the title ~~Install runai dep for vllm~~ feat: install runai dep for vllm Dec 10, 2025

github-actions bot added the feat label Dec 10, 2025

Install runai dep for vllm

af221f0

Signed-off-by: Guangtong Bai <[email protected]>

gtbai force-pushed the gbai/install-runai-dep-for-vllm branch from a6504db to af221f0 Compare December 10, 2025 09:20

gtbai changed the title ~~feat: install runai dep for vllm~~ feat: install Run:ai model streamer dep for vllm Dec 10, 2025

rmccorm4 assigned ganeshku1 Dec 10, 2025

gtbai changed the title ~~feat: install Run:ai model streamer dep for vllm~~ feat: install Run:ai model streamer for vllm Dec 11, 2025

rmccorm4 approved these changes Dec 11, 2025

View reviewed changes

copy-pr-bot bot temporarily deployed to GITLAB December 11, 2025 01:29 Inactive

copy-pr-bot bot temporarily deployed to GITLAB December 11, 2025 01:30 Inactive

rmccorm4 enabled auto-merge (squash) December 11, 2025 01:31

Merge branch 'main' into gbai/install-runai-dep-for-vllm

b1a0b67

copy-pr-bot bot had a problem deploying to GITLAB December 11, 2025 15:19 Failure

rmccorm4 merged commit 53cec4a into ai-dynamo:main Dec 11, 2025
41 of 46 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: install Run:ai model streamer for vllm #4848

feat: install Run:ai model streamer for vllm #4848

gtbai commented Dec 10, 2025 •

edited

Loading

Uh oh!

copy-pr-bot bot commented Dec 10, 2025

Uh oh!

github-actions bot commented Dec 10, 2025

Uh oh!

coderabbitai bot commented Dec 10, 2025 •

edited

Loading

Uh oh!

rmccorm4 commented Dec 11, 2025

Uh oh!

rmccorm4 commented Dec 11, 2025

Uh oh!

athreesh commented Dec 11, 2025

Uh oh!

gtbai commented Dec 11, 2025

Uh oh!

rmccorm4 commented Dec 11, 2025

Uh oh!

gtbai commented Dec 11, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

feat: install Run:ai model streamer for vllm #4848

feat: install Run:ai model streamer for vllm #4848

Conversation

gtbai commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview:

Test

Summary by CodeRabbit

Uh oh!

copy-pr-bot bot commented Dec 10, 2025

Uh oh!

github-actions bot commented Dec 10, 2025

Uh oh!

coderabbitai bot commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Poem

Pre-merge checks

Uh oh!

rmccorm4 commented Dec 11, 2025

Uh oh!

rmccorm4 commented Dec 11, 2025

Uh oh!

athreesh commented Dec 11, 2025

Uh oh!

gtbai commented Dec 11, 2025

Uh oh!

rmccorm4 commented Dec 11, 2025

Uh oh!

gtbai commented Dec 11, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

gtbai commented Dec 10, 2025 •

edited

Loading

coderabbitai bot commented Dec 10, 2025 •

edited

Loading