-
Notifications
You must be signed in to change notification settings - Fork 76
tests(store-sync): fix flaky reconciliation test on macOS by anchoring timestamps to a fixed baseTime and aligning sync window via offset #3619
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
…g timestamps to a fixed baseTime and aligning sync window via offset Root cause: - Separate calls to getNowInNanosecondTime() for test message timestamps and sync window boundaries led to mismatched ranges on slower macOS runs. - This caused in-window messages to be miscategorized as out-window, producing assertion failures (e.g., remoteNeeds.len==0 while diffInWin==18). Fix: - Introduce a single fixed baseTime at test start and construct all message timestamps relative to it. - Define the intended sync window as [baseTime - 1s, baseTime] and compute the reconciliation offset so effective now equals baseTime (offset = actualNow - baseTime). - Place in-window messages safely inside the window with 50ms margins, and out-window messages well before the window to avoid overlap. - Use consistent int64 timestamp arithmetic. Result: - The test is deterministic and passes reliably on macOS; eliminates the timing race. Recommend running CI matrix (macOS + Ubuntu) to confirm cross-platform stability.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Fix a flaky reconciliation test on macOS by implementing deterministic timestamp handling to eliminate timing race conditions between message creation and sync window computation.
- Anchors all time calculations to a single fixed
baseTimeto prevent timestamp drift - Aligns the reconciliation sync window with message timestamps using a calculated offset
- Replaces relative timestamp calculations with absolute positioning based on the fixed base time
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| let inWinStart = Timestamp(syncWindowStart.int64 + 50_000_000'i64) # 50ms after window starts | ||
| let inWinEnd = Timestamp(syncWindowEnd.int64 - 50_000_000'i64) # 50ms before window ends |
Copilot
AI
Oct 8, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The magic number 50_000_000'i64 (50ms) is repeated. Consider extracting it as a named constant like BUFFER_MARGIN_NS to improve maintainability and make the buffer size configurable.
|
You can find the image built from this PR at Built from 4f435a4 |
|
Ah clever! It use the offset as a way to fix the time window. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! Thanks for it! 💯
|
Seems still not ok. Now ci/test-ubuntu-22.04 failed with this exact test case (While macos ci succeeded): |
Summary
Fix a flaky reconciliation test in
tests/waku_store_sync/test_protocol.nimthat intermittently failed on macOS due to timestamp drift between message creation and the reconciliation sync window.Root cause
getNowInNanosecondTime()while the reconciliation time window was computed separately viacalculateTimeRange().remoteNeeds.len == 0whilediffInWin == 18Solution
baseTimecaptured once at the start of the test.[baseTime - 1s, baseTime].baseTimeby passing an explicit offset:offset = actualNow - baseTimecalculateTimeRange(now, jitter)usesnow = getNow() - jitter, settingjitter = actualNow - baseTimeyieldsnow_effective = baseTime.int64arithmetic for timestamp math.Implementation details
tests/waku_store_sync/test_protocol.nimbaseTimeonce and computesyncWindowStart/Endfrom it.baseTimewith clear buffers.timeOffset = actualNow - baseTimeand pass it tostoreSynchronization(offset = timeOffset)so the window is[baseTime - 1s, baseTime].int64to avoid signed/unsigned pitfalls.Verification
make test tests/waku_store_sync/test_protocol.nimpasses locally.Scope / Risk
Notes