fix: Cherry-pick multimodal audio support and vLLM v0.12.0 fixes to release/0.7.1 #4871
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Cherry-pick of PR #2760 to release/0.7.1. Adds audio multimodal support for vLLM including: - AudioEncodeWorker for processing audio inputs - Audio aggregated and disaggregated serving examples - Audio loader utilities - Documentation for audio multimodal serving Original PR: #2760 Original author: Yuekai Zhang <[email protected]> Co-authored-by: Kris Hung <[email protected]> Signed-off-by: Dan Gil <[email protected]>
…4849) Cherry-pick of PR #4849 to release/0.7.1 for vLLM v0.12.0 compatibility. Updates import paths and fixes for vLLM v0.12.0: - Updated AnyTokenizer import from vllm.tokenizers - Updated FlexibleArgumentParser import from vllm.utils.argparse_utils - Updated RequestMetrics to RequestStateStats - Fixed GPU device assignments in launch scripts - Added StubEngineClient for preprocessing-only operations Closes DIS-1160 Original PR: #4849 Original author: Kris Hung <[email protected]> Signed-off-by: Dan Gil <[email protected]>
Contributor
Author
|
/ok to test 84c405c |
Contributor
Author
|
Closing PR - will revisit the cherry-pick approach |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
fix: Cherry-pick multimodal audio support and vLLM v0.12.0 fixes to release/0.7.1
Overview
This PR cherry-picks two related changes to
release/0.7.1:feat: Add vLLM multimodal audio supportfix: Fix multimodal EPD examples for vllm version bumpWhy Both PRs?
PR #4849 fixes vLLM v0.12.0 compatibility issues in multimodal examples, including audio multimodal files. However, the audio feature (PR #2760) was never cherry-picked to the 0.7.x release branch. To properly apply the fixes from #4849, the audio feature must be present first.
Changes Included
From PR #2760 (Audio Multimodal Support)
AudioEncodeWorkerfor processing audio inputsaudio_agg.sh)audio_disagg.sh)audio_loader.py)From PR #4849 (vLLM v0.12.0 Fixes)
AnyTokenizerimport fromvllm.tokenizersFlexibleArgumentParserimport fromvllm.utils.argparse_utilsRequestMetricstoRequestStateStatsStubEngineClientfor preprocessing-only operationsConflict Resolutions
docs/backends/vllm/multimodal.md- Fixed relative path references (keptREADME.mdinstead of../../docs/backends/vllm/README.md)examples/multimodal/launch/audio_agg.sh- Applied GPU device assignment fix from fix: Fix multimodal EPD examples for vllm version bump #4849Testing
Related Issues
Original PRs
Note on GPG Signing
These commits are not GPG-signed. To trigger automated backend tests, please comment: