Skip to content

[QNN EP] Fuse Gelu pattern into a QNN Gelu Node #26417

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Nov 24, 2025
Merged

[QNN EP] Fuse Gelu pattern into a QNN Gelu Node #26417

merged 4 commits into from
Nov 24, 2025
+1,053 −5

Conversation

@quic-tirupath
Copy link
Contributor

@quic-tirupath quic-tirupath commented Oct 27, 2025

Description

  • ONNX models exported with older Opset version contains Gelu operator decomposed into multiple operators (Div, Erf, Add, Mul).
  • QNN doesn't support Erf operator but supports Gelu operator
  • Since QNN doesn't support Erf operator, the graphs contain Gelu pattern partition between QNN and CPU EPs and degrading the inference time.

Motivation and Context

  • Identify and fuse the Gelu pattern into a QNN Gelu node improves the inference time.

@quic-tirupath quic-tirupath mentioned this pull request Oct 27, 2025
Open
@quic-tirupath
Copy link
Contributor Author

quic-tirupath commented Oct 27, 2025

@chilo-ms
As i mentioned in #26332, i would like to use this PR for merging this fusion.
Could you please help to trigger CI job ?

@quic-tirupath
Copy link
Contributor Author

quic-tirupath commented Oct 30, 2025

@chilo-ms, @devang-ml
could you please refer to my comment in #26332 (comment).

Could you please help to trigger CI job on this PR.

Thanks,

edgchen1
edgchen1 reviewed Oct 31, 2025
View reviewed changes
@adrianlizarraga
Copy link
Contributor

adrianlizarraga commented Nov 21, 2025

/azp run Linux QNN CI Pipeline, Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline,Windows x64 QNN CI Pipeline

@azure-pipelines
Copy link

azure-pipelines bot commented Nov 21, 2025

Azure Pipelines successfully started running 4 pipeline(s).

edgchen1
edgchen1 reviewed Nov 22, 2025
View reviewed changes
@quic-tirupath
Copy link
Contributor Author

quic-tirupath commented Nov 22, 2025 •
edited
Loading

Hi @adrianlizarraga ,
I rebased the PR with removing dependency on ORT Core's QDQ selector.
Could you please trigger CI and help with merge after successful CI run.

@adrianlizarraga
Copy link
Contributor

adrianlizarraga commented Nov 22, 2025

/azp run Linux QNN CI Pipeline, Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline,Windows x64 QNN CI Pipeline

@azure-pipelines
Copy link

azure-pipelines bot commented Nov 22, 2025

Azure Pipelines successfully started running 4 pipeline(s).

@adrianlizarraga
Copy link
Contributor

adrianlizarraga commented Nov 22, 2025

Hi @quic-tirupath, you may have to sync with the latest main branch to get CIs to pass.

quic-tirupath and others added 4 commits November 24, 2025 08:25
@quic-tirupath
[QNN EP] Fuse Gelu pattern into a QNN Gelu Node
e291f34 
 - ONNX models exported with older Opset version contains Gelu operator
   decomposed into multiple operators (Div, Erf, Add, Mul).
 - QNN doesn't support Erf operator but supports Gelu operator
 - Since QNN doesn't support Erf operator, the graphs contain Gelu pattern
   partition between QNN and CPU EPs and degrading the inference time.
 - Identify and fuse the Gelu pattern into a QNN Gelu node improves
   the inference time.
@quic-tirupath
Fix Minor issues in GeluFusion
9cef8db
@quic-tirupath
Handle DQ->Erf->Q pattern
1ad5d05
Correct the comments as per latest implementation
5812d30
@quic-tirupath quic-tirupath force-pushed the dev/tirupath/erf_gelu_qnn_fusion branch from e881532 to 5812d30 Compare November 24, 2025 16:25
@quic-tirupath
Copy link
Contributor Author

quic-tirupath commented Nov 24, 2025

Hi @quic-tirupath, you may have to sync with the latest main branch to get CIs to pass.

@adrianlizarraga
I rebased the PR on tip. Could you please re-trigger CI and help to merge ?
Thanks,

@quic-tirupath
Copy link
Contributor Author

quic-tirupath commented Nov 24, 2025

Hi @quic-tirupath, you may have to sync with the latest main branch to get CIs to pass.

@adrianlizarraga
I rebased the PR on tip. Could you please re-trigger CI and help to merge.

@adrianlizarraga
Copy link
Contributor

adrianlizarraga commented Nov 24, 2025

/azp run Linux QNN CI Pipeline, Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline,Windows x64 QNN CI Pipeline

@azure-pipelines
Copy link

azure-pipelines bot commented Nov 24, 2025

Azure Pipelines successfully started running 4 pipeline(s).

@adrianlizarraga adrianlizarraga merged commit e8bcd0d into microsoft:main Nov 24, 2025
90 of 97 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@edgchen1 edgchen1 edgchen1 left review comments

@adrianlizarraga adrianlizarraga adrianlizarraga approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@quic-tirupath @adrianlizarraga @edgchen1