feat(0.16.3): Android Qualcomm NPU + qdrant-edge 0.7.1#296
Merged
Conversation
Builds the Qualcomm QNN dispatch bridge from LiteRT source (commit d865fd82, matching LiteRT-LM ffed38a / native-v0.12.0). Required for PreferredBackend.npu on Android Snapdragon devices (#293).
- Bump qdrant-edge dependency: 0.6.1 → 0.7.1 - Replace EdgeShard::load_with_wal_options (patched upstream) with EdgeShard::load + wal_options field in EdgeConfig (now native in 0.7) - Remove [patch.crates-io] block and ../vendored/qdrant-edge/ dependency - Add rust-toolchain.toml pinning stable (0.7+ needs rustc 1.95+) - build_local.sh: prefer rustup rustc over Homebrew's if available - CI workflow: update default version, clean up release body text
….npu - AndroidManifest: declare libcdsprpc.so as optional native library so Bionic can resolve the CDSP RPC symbols needed by HTP dispatch - FlutterGemmaPlugin: add getNativeLibraryDir to bundledChannel so Dart can resolve the path where APK-bundled .so files land at runtime - litert_lm_client.dart: call set_litert_dispatch_lib_dir with nativeLibraryDir when backend=='npu' on Android (mirrors Windows NPU) - build_android.sh: copy Qualcomm dispatch+QNN libs from SCRIPT_DIR prebuilt dir into the output tarball (separate from LiteRT-LM prebuilts) Qualcomm libs in prebuilt/android_arm64/ cover sm8550(V73), sm8650(V75), sm8750(V79), sm8850(V81) — extracted from Google AI Edge Gallery APKs, ABI-verified against litert_dispatch.h at LiteRT commit d865fd82.
- _NativeBundle: add androidExtraLibs field (parallel to windowsExtraLibs) for Android-only companion libraries - _processBundle: register androidExtraLibs as CodeAssets on Android - litertlm bundle: declare Qualcomm NPU libs as androidExtraLibs so Native Assets bundles them into the APK (LiteRtDispatch_Qualcomm + QnnHtp + QnnSystem + V73/V75/V79/V81 Stubs) - qdrant-edge bundle: bump to 0.7.1 (vendored fork dropped), clear placeholder checksums pending tarball rebuild - Mark Android litertlm sha256 as TODO pending QNN-inclusive tarball rebuild
litertlm android: new tarball includes Qualcomm QNN dispatch libs (NDK r29, LiteRT-LM ffed38a + QnnHtp/System/V73/V75/V79/V81 Stubs) qdrant-edge 0.7.1: all 7 platforms — macOS/iOS/iOS-sim/Android (local), Linux x86_64/arm64 + Windows x86_64 (CI workflow run 26667793192) build_android.sh: skip cp when source==dest for Qualcomm libs
…ix build_qualcomm_dispatch.sh, drop vendored qdrant-edge fork - litert_lm_client.dart: throw StateError instead of silent skip when getNativeLibraryDir returns null (C-1 from PR review) - FlutterGemmaPlugin.kt: wrap getNativeLibraryDir in try/catch for consistent error reporting (I-1) - build_qualcomm_dispatch.sh: NDK_API_LEVEL 28→30 (matches minSdk for .litertlm); update stale LiteRT-LM ref comment 032334d→ffed38ad (I-4, I-5) - Remove native/qdrant_edge/vendored/ (0.6.1 fork, dead since 0.7.1) - Remove rust-toolchain.toml (dead weight, rustup detection in build_local.sh is sufficient; qdrant-edge 0.7.1 on crates.io requires stable ≥1.95.0)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PreferredBackend.npu): bundleslibLiteRtDispatch_Qualcomm.so+ QNN runtime stack (libQnnHtp.so,libQnnSystem.so,libQnnHtpV73/V75/V79/V81Stub.so) for sm8550/sm8650/sm8750/sm8850 chipsets. Wired vialitert_lm_engine_settings_set_litert_dispatch_lib_dirusingnativeLibraryDirfrom a platform channel call beforeengine_create.wal_optionsis now native in upstream. Requires Rust 1.95+ (stable);rust-toolchain.toml+RUSTCdetection inbuild_local.shensure correct toolchain on macOS with Homebrew.androidExtraLibsfield parallel towindowsExtraLibs; all 7 platform sha256s updated; version bumped tonative-v0.12.0-a(new release tag that includes QNN libs without clobberingnative-v0.12.0).libcdsprpc.soas optional native library (required for Qualcomm HTP dispatch).Test plan
litertlm_ffi_test.dart) — all 23 tests passed on Pixel 8 (exit 0).litertlmmodel