0.16.3: Android Qualcomm NPU + GPU sampler fix#297
Merged
Conversation
litert_dispatch.cc uses opendir() to scan dispatch_lib_dir for libLiteRtDispatch_Qualcomm.so. With AGP 8+ default extractNativeLibs=false, .so files stay inside the APK and are never extracted to nativeLibraryDir, so the scan finds nothing and NPU init fails with 'No dispatch library found'. Extract the QNN dispatch stack (~7 MB) to codeCacheDir/npu_libs/ on first NPU engine_create. Subsequent calls skip already-extracted files.
) - Add libQnnHtpV73/V75/V79Skel.so to androidExtraLibs (extracted from Google AI Edge Gallery 1.0.15 APKs for SM8550/SM8650/SM8750) - Update native-v0.12.0-a android_arm64 tarball sha256 - Extract Skel libs alongside Stub libs to codeCacheDir/npu_libs/ so FastRPC can find them via ADSP_LIBRARY_PATH
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PreferredBackend.npu): extracts QNN dispatch stack from APK tocodeCacheDir/npu_libs/at runtime solitert_dispatch.ccopendir()scan finds the libs (AGP 8+extractNativeLibs=falsekeeps.socompressed in APK, never extracted tonativeLibraryDir). Bundles Qualcomm QNN libs for SM8550/SM8650/SM8750/SM8850 extracted from Google AI Edge Gallery APKs.patchelf --add-needed libLiteRtLm.sotolibLiteRtTopKOpenClSampler.soandlibLiteRtTopKWebGpuSampler.so— restores ~3× decode speedup on Android GPU.Test plan
gemma-4-E2B-it_qualcomm_sm8750.litertlm+PreferredBackend.npuon SM8750 device (hacker1024 confirming)gemma-4-E2B-it.litertlm+PreferredBackend.gpu— verify no GPU sampler CPU fallback in logcat