feat(ai): Add App Functions for system AI integration#5585
Open
jamesarich wants to merge 16 commits into
Open
Conversation
Contributor
✅ Docs staleness check passedThis PR includes updates to |
jamesarich
added a commit
that referenced
this pull request
May 30, 2026
Squash merge of PR #5585 into release/2.8.0. Enable system AI assistants (Gemini, etc.) to interact with Meshtastic mesh networks through the Android AppFunctions framework. Adds read access to mesh status, nodes, channels, messages, and metrics plus the ability to send messages without opening the app UI. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Collaborator
Author
🔗 Release 2.8.0 Integration ReportBranch: Integration Changes RequiredWhen merging alongside the other 2.8.0 PRs, the following adaptations were needed:
Demo Results (Emulator + Live Node)All 9 App Functions registered, indexed, and executed successfully:
Merge Guidance
|
This was referenced May 31, 2026
Collaborator
Author
🎉 2.8.0 Integration Demo — All Features VerifiedDemo branch: Features Verified
App Functions Live Test Results// getDeviceStatus
{"model":"HELTEC_V3","firmwareVersion":"2.7.20.6658ec2","batteryLevel":100,"isActive":true}
// getMeshStatus
{"connectionState":"CONNECTED","totalNodeCount":200,"onlineNodeCount":21,"localNodeName":"Meshtastic 18c4"}
// sendMessage (DM to specific node)
{"messageId":1769805483,"channel":"DM to olm3c @ Meshtastic"}Recommended Merge Order
Blockers for Production Merge
See individual PR comments for per-PR integration details. |
Expose Meshtastic mesh networking capabilities (sendMessage, getMeshStatus) to Android system AI agents via the App Functions API. Architecture: - AiFunctionProvider interface in core/data commonMain (platform-agnostic) - FuzzyNameResolver for node/channel name matching (LCS algorithm) - RateLimiter with 5-call/60s sliding window to protect mesh radio - AiFunctionProviderImpl wiring repositories and use cases - @appfunction declarations in androidApp Google flavor only - GoogleMeshUtilApplication with AppFunctionConfiguration.Provider - DI via AppFunctionsModule included in FlavorModule Key design decisions: - No confirmation dialog (AI invocation = user intent) - Fuzzy name matching with 50% LCS threshold, error on ambiguity - Admin channels excluded from resolution - 5-second operation timeout - 237-byte message length limit (Meshtastic standard) Includes unit tests for RateLimiter and FuzzyNameResolver (LCS algorithm). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Previously, AiFunctionProviderImpl returned a hardcoded messageId of 0 for all successful sends, preventing unique message identification. The underlying SendMessageUseCase generates a packetId but had no return value to expose it. Changes: - Modified SendMessageUseCase interface to return Int (the packetId) - Updated SendMessageUseCaseImpl to return the generated packetId - Updated AiFunctionProviderImpl to capture and use the returned messageId This enables the AI system to track individual messages and correlate responses to specific send requests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…fo, getDeviceStatus) Implement three Phase 2a read-only query functions for AI agent integration: Core Data (commonMain): - Extended AiFunctionProvider interface with 3 new methods - Implemented getNodeList() - queries all mesh nodes with battery and online status - Implemented getChannelInfo() - lists mesh channels with settings - Implemented getDeviceStatus() - returns local device info and status - Added result types: GetNodeListResult, GetChannelInfoResult, GetDeviceStatusResult - Added data models: NodeSummary, ChannelSummary, DeviceStatus - All functions protected by timeout (5s), rate limiter (5 calls/60s), connection check Android AppFunctions: - Added 3 @appfunction methods with KSP annotations - Response models marked @AppFunctionSerializable for AI runtime - Proper exception handling and timeout protection Constants: - HEX_RADIX (16): For node ID formatting - MS_PER_SEC (1000): Time unit conversions - ONLINE_THRESHOLD_MS (30000): Node online detection threshold Fixes: - Use node.user.long_name (not longName) - Use node.deviceMetrics.battery_level?.coerceIn() for nullable battery - Use nodeRepository.nodeDBbyNum (not nodes) - Convert node.lastHeard (seconds) to milliseconds - Suppress MagicNumber and ReturnCount lints appropriately All builds and tests pass: ✓ :core:data:compileKotlinJvm ✓ :androidApp:compileGoogleDebugKotlin ✓ :androidApp:compileFdroidDebugKotlin ✓ detekt clean ✓ spotlessApply clean ✓ :core:data:allTests pass Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Extend AiFunctionProvider with two new suspend methods for advanced queries - getNodeDetails: Retrieve per-node telemetry (16 fields) by hex or user ID - getMeshMetrics: Aggregate mesh statistics and compute health score - Add result types (GetNodeDetailsResult, GetMeshMetricsResult) and data models - Add response models (@AppFunctionSerializable) for KSP serialization - Both methods support timeout protection and rate limiting - Health score calculation: 50 base + 50 online ratio, clamped 0-100 - All JVM, Android, detekt, spotless checks passing Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Fix location filtering: Only treat (0,0) as invalid if position.time is 0 - Previously filtered all (0,0) coords as null, losing valid equatorial data - Now checks position.time to distinguish 'no fix' from real coordinates - Fix mostRecentPacketTime: Use max lastHeard from all nodes, not current time - Previously returned current time, making mesh appear always active - Now computes from actual node activity data - Fix meshUptimeSeconds: Use local device's actual uptime, not epoch time - Previously returned epoch seconds (~1.7B), not elapsed time - Now uses device's DeviceMetrics.uptime_seconds All checks passing: Android (Google/fdroid), detekt, spotless, unit tests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…Heard=0
- Add takeIf check to distinguish lastHeard=0 (never heard) from no nodes
- Previously: maxOfOrNull returns 0, Elvis operator doesn't trigger (0 is not null)
- Now: takeIf { it > 0 } filters out zero, falling back to current time
- Ensures API returns meaningful timestamp instead of epoch 1970
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add TimeoutCancellationException handling to getNodeDetails and getMeshMetrics AppFunctions (consistent with Phase 1 functions) - Rethrow CancellationException in all provider catch blocks to preserve structured concurrency semantics - Fix voltage documentation: millivolts → volts (matches actual Float field) - Fix stale test comment referencing non-existent test class Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- SendMessageUseCase now rethrows exceptions after logging (Finding #1) - AiFunctionProviderImpl catches send failures and returns InvalidArgument - Added AiFunctionProviderImplTest with 10 unit tests covering: - Disconnection checks for all three function groups - Node lookup (found, not found, null position, invalid hex) - Metrics aggregation (active nodes, empty, zero lastHeard, degraded health) - Rate limiting behavior - Expanded FuzzyNameResolverTest with 8 behavioral tests (Finding #5): - resolveNodeName: exact, fuzzy, ambiguous, not found - resolveChannelName: exact, admin exclusion, empty channels Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The model's isOnline property uses the project-standard 2-hour window (onlineTimeThreshold), consistent with getMeshStatus().onlineNodeCount. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The interface now returns Int (packetId), so the test mock must return an Int instead of Unit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Phase 3 of App Functions integration: read-only message history functions that enable 'catch me up' voice queries via system AI. New functions: - getRecentMessages: Retrieve recent messages with optional contact filter and configurable limit (1-50, default 20) - getUnreadSummary: Per-contact unread breakdown excluding muted contacts, sorted by most recent Implementation details: - KMP interface + sealed result types in core:data - Android @appfunction declarations with @AppFunctionSerializable models - Fuzzy name resolution for contact filtering - Channel name resolution for broadcast contacts - Tests for contact-not-found and empty unread scenarios Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Audit findings addressed: - Add res/xml/app_metadata.xml with LLM-facing operational patterns, workflow dependencies, and constraints for the AppFunctions suite - Register app_metadata in Google flavor AndroidManifest.xml - Convert all @AppFunctionSerializable class-level @Property tags to inline KDoc per property (required by KSP for doc extraction) - Add app_description string resource for displayDescription Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add @AppFunctionStringValueConstraint to connectionState and chargingStatus properties for agent-visible enum values - Add @AppFunctionIntValueConstraint to ChannelInfo.index (0-7) and getRecentMessages limit parameter (1,5,10,20,50) - Replace AppFunctionInvalidArgumentException with semantic types: - NotConnected states -> AppFunctionNotSupportedException (code 3501) - NotFound states -> AppFunctionElementNotFoundException (code 2001) - Keeps AppFunctionInvalidArgumentException for actual bad input (ambiguous names, invalid args, rate limits, generic errors) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add a settings sub-screen that allows users to control which App Functions are exposed to system AI (Gemini). Includes a master toggle and per-function granularity for all 9 functions. - AppFunctionsPrefs interface + DataStore implementation in core/prefs - AppFunctionsSettingsScreen with master + per-function toggles - AppFunctionStateSync in Google flavor syncs prefs to AppFunctionManager - Settings visibility is flavor-gated via Koin-provided boolean (googleServicesAvailable) -- hidden on fdroid, shown on Google - Analytics toggle visibility also now properly flavor-gated Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add MeshtasticAppFunctionsTest with coverage for all 9 AI functions (getDeviceStatus, getNodeList, sendMessage, etc.) - Fix GoogleAiModule to inject NodeRepository into GeminiNanoDocAssistant (constructor changed on main) - Add App Check TODO note for future cloud fallback consideration Found during release/2.8.0 integration testing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
jamesarich
force-pushed
the
jamesarich/crispy-barnacle
branch
from
439e821 to
b46d1a2
Compare
|
The App Functions branch accidentally carried a proto pointer to a non-public commit. Reset to main's v2.7.24-2 tag. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Enable system AI assistants (Gemini, etc.) to interact with Meshtastic mesh networks through the Android AppFunctions framework. This gives AI agents read access to mesh status, nodes, channels, messages, and metrics -- plus the ability to send messages -- without opening the app UI.
AppFunctions act as on-device MCP (Model Context Protocol) tools, making the app's capabilities discoverable and invocable by authorized system agents on Android 16+.
Approach
Implements 9 App Functions in a layered KMP architecture:
core/data/ai/): Platform-agnosticAiFunctionProviderinterface and implementation with rate limiting, fuzzy name resolution, and structured error handlingandroidApp/src/google/):@AppFunctiondeclarations with@AppFunctionSerializableresponse models, wired via Koin DIFunctions implemented
sendMessagegetMeshStatusgetNodeListgetChannelInfogetDeviceStatusgetNodeDetailsgetMeshMetricsgetRecentMessagesgetUnreadSummaryNotable design decisions
@AppFunctionStringValueConstrainton enum-like string fields (connectionState, chargingStatus) and@AppFunctionIntValueConstrainton bounded integers (channel index, message limit) so AI agents know valid values upfrontAppFunctionNotSupportedExceptionwhen disconnected (code 3501),AppFunctionElementNotFoundExceptionwhen nodes/channels not found (code 2001),AppFunctionInvalidArgumentExceptionfor bad input (code 1001)NotSupportedException@AppFunctionSerializableproperties use inline KDoc (not class-level@propertytags) per KSP extraction requirementsTesting
adb shell cmd app_function execute-app-function-- all 9 functions respond correctly with proper constraint metadata and error codesandroidx.appfunctions:appfunctions:1.0.0-alpha09(latest as of 2026-05-06)Notes for reviewers
googlebuild flavor only; fdroid builds are unaffectedDataPacket.ID_BROADCASTtoNodeAddress.ID_BROADCAST(7 import references inAiFunctionProviderImpl.kt)specs/directory contains the feature spec, plan, and task breakdown used during development