[Synced Transcripts] Add fingerprint timing manager and supporting business logic#5323
Open
sztomek wants to merge 1 commit into
Open
Conversation
Collaborator
Generated by 🚫 Danger |
Contributor
There was a problem hiding this comment.
Pull request overview
Adds the core “synced transcripts” audio fingerprint timing pipeline in the repositories layer. The new FingerprintTimingManager orchestrates reference fingerprint retrieval + decoding/matching + drift filtering + time-mapping interpolation, with optional on-disk caching, gated behind Feature.SYNCED_TRANSCRIPTS.
Changes:
- Introduces
FingerprintTimingManager(state machine + streaming fingerprint generation + match filtering + time mapping queries). - Adds reference retrieval/decoding and a mapping cache (
FingerprintReferenceRetriever,ReferenceFingerprint,FingerprintMappingCache,FingerprintConstants). - Adds initial unit tests for reference decoding, interpolation/window alignment, and drift-filter behavior.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| modules/services/repositories/src/main/java/au/com/shiftyjelly/pocketcasts/repositories/fingerprint/FingerprintTimingManager.kt | New orchestrator for fingerprint-based time mapping (decode/match/filter/interpolate/cache integration). |
| modules/services/repositories/src/main/java/au/com/shiftyjelly/pocketcasts/repositories/fingerprint/FingerprintReferenceRetriever.kt | Fetches + (optionally) caches gzip-compressed reference fingerprints with retry/backoff and request de-duping. |
| modules/services/repositories/src/main/java/au/com/shiftyjelly/pocketcasts/repositories/fingerprint/ReferenceFingerprint.kt | Moshi model + decoding and conversion of compact reference checkpoints into library checkpoints. |
| modules/services/repositories/src/main/java/au/com/shiftyjelly/pocketcasts/repositories/fingerprint/FingerprintMappingCache.kt | Persists/loads validated time-mapping snapshots keyed by audio + reference fingerprints. |
| modules/services/repositories/src/main/java/au/com/shiftyjelly/pocketcasts/repositories/fingerprint/FingerprintConstants.kt | Central constants for matching thresholds, drift filtering, cache schema, and codec timeouts. |
| modules/services/repositories/src/test/java/au/com/shiftyjelly/pocketcasts/repositories/fingerprint/ReferenceFingerprintTest.kt | Unit tests for reference decoding and checkpoint conversion behavior. |
| modules/services/repositories/src/test/java/au/com/shiftyjelly/pocketcasts/repositories/fingerprint/FingerprintTimingManagerTest.kt | Unit tests for interpolation and window-grid alignment. |
| modules/services/repositories/src/test/java/au/com/shiftyjelly/pocketcasts/repositories/fingerprint/DriftFilterTest.kt | Unit tests for drift filter insertion/ordering and acceptance/rejection behavior. |
| modules/services/repositories/build.gradle.kts | Adds fingerprint service module dependency to repositories. |
Comment on lines
+16
to
+34
| import kotlin.math.abs | ||
| import kotlin.math.floor | ||
| import kotlin.math.max | ||
| import kotlin.math.min | ||
| import kotlinx.coroutines.CancellationException | ||
| import kotlinx.coroutines.CoroutineScope | ||
| import kotlinx.coroutines.Dispatchers | ||
| import kotlinx.coroutines.Job | ||
| import kotlinx.coroutines.SupervisorJob | ||
| import kotlinx.coroutines.cancel | ||
| import kotlinx.coroutines.currentCoroutineContext | ||
| import kotlinx.coroutines.delay | ||
| import kotlinx.coroutines.ensureActive | ||
| import kotlinx.coroutines.flow.MutableStateFlow | ||
| import kotlinx.coroutines.flow.StateFlow | ||
| import kotlinx.coroutines.flow.asStateFlow | ||
| import kotlinx.coroutines.flow.first | ||
| import kotlinx.coroutines.isActive | ||
| import kotlinx.coroutines.launch |
Comment on lines
+540
to
+545
| outputIndex >= 0 -> { | ||
| if (bufferInfo.flags and MediaCodec.BUFFER_FLAG_END_OF_STREAM != 0) { | ||
| codec.releaseOutputBuffer(outputIndex, false) | ||
| break | ||
| } | ||
|
|
| gen = generation | ||
| } | ||
| // Abort if a newer stop()/prepare() has started since we acquired the lock. | ||
| if (gen != generation) return@launch |
|
|
||
| private var lastProgressPositionMs = -1 | ||
| private var generationJob: Job? = null | ||
| private var preparationJob: Job? = null |
Comment on lines
+147
to
+151
| try { | ||
| tmpFile.writeText(adapter.toJson(cached)) | ||
| if (!tmpFile.renameTo(File(path))) { | ||
| Timber.w("FingerprintMappingCache: atomic rename failed for $path, falling back to direct write") | ||
| File(path).writeText(adapter.toJson(cached)) |
|
|
||
| fun decode(data: ByteArray): ReferenceFingerprint? { | ||
| return try { | ||
| val fingerprint = adapter.fromJson(String(data)) ?: return null |
Comment on lines
+41
to
+55
| fun load( | ||
| audioFilePath: String, | ||
| referenceFilePath: String, | ||
| referenceData: ByteArray, | ||
| ): LoadResult? { | ||
| val path = mappingPath(audioFilePath) | ||
| val file = File(path) | ||
| if (!file.exists()) return null | ||
|
|
||
| val cached = try { | ||
| adapter.fromJson(file.readText()) | ||
| } catch (e: Exception) { | ||
| Timber.w("FingerprintMappingCache: failed to decode cache at $path — discarding") | ||
| return null | ||
| } ?: return null |
Comment on lines
+6
to
+9
| import java.nio.ByteBuffer | ||
| import java.nio.ByteOrder | ||
| import java.util.Base64 | ||
| import timber.log.Timber |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR is part of the Radical Speed Month initiative.
Implements the audio fingerprinting pipeline that powers synced transcripts. The core component is
FingerprintTimingManager, a@Singletonorchestrator that: - Fetches gzip-compressed reference fingerprints from S3 (viaFingerprintReferenceRetriever)MediaExtractor/MediaCodecand generates streaming fingerprints via the UniFFI native libraryCheckpointMatcherFingerprintMappingCachewith SHA-256 validation) to avoid recomputationThe manager exposes a
StateFlow(Idle/Preparing/Active/Failed/Unavailable) and two public query methods:referenceTime(forPlaybackTimeMs)andplaybackTimeMs(forReferenceTime).Gated behind the
SYNCED_TRANSCRIPTSfeature flag.Testing Instructions
Review the code please, the next PR will connect the dots, making the entire flow testable
Checklist
./gradlew spotlessApplyto automatically apply formatting/linting)modules/services/localization/src/main/res/values/strings.xml