Skip to content

Conversation

@hadiFute
Copy link

Description

  • Don't register set device function when we use existing stream
  • Fix bug nv_execution_provider.cc : set device only if user did not provide existing stream

Motivation and Context

In some use cases, we push a user generated CUDA context, create streams using this context, and then provide these streams to TRT-RTX.
However, we noticed that after calling Run(), the custom context is replaced by another CUDA context created by ORT. This means that TRT-RTX is no longer using the original CUDA context.
After investigating further, we found that the new context is being created in onnxruntime/core/framework/stream_execution_context.cc.
The solution we propose is to not register set device function if we provide the stream.

Also there is a bug in onnxruntime\core\providers\nv_tensorrt_rtx\nv_execution_provider.cc.
We should set the device only if the user has not provided any stream. (coherent with the original comment)

@hadiFute hadiFute changed the title Don't register set device function when we use existing stream [TRT RTX EP] Don't register set device function when we use existing stream Nov 10, 2025
@hadiFute
Copy link
Author

@microsoft-github-policy-service agree company="NVIDIA"

@hadiFute
Copy link
Author

For review @chilo-ms @edgchen1 @ishwar-raut1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant