-
Notifications
You must be signed in to change notification settings - Fork 1.4k
chore: fix flaky test #6765
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
chore: fix flaky test #6765
Conversation
WalkthroughThese changes refine E2E test flows by implementing platform-specific iOS key input handling, improving element visibility wait conditions, switching tap selectors from text/index-based to id-based targeting, and adjusting in-app notification duration to extend timeouts during E2E test execution. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes
Possibly related PRs
Suggested reviewers
Poem
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Jira integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (4)
.maestro/tests/assorted/join-from-directory.yaml(1 hunks).maestro/tests/assorted/setting.yaml(1 hunks).maestro/tests/room/jump-to-message.yaml(1 hunks)app/containers/InAppNotification/index.tsx(1 hunks)
🔇 Additional comments (3)
.maestro/tests/room/jump-to-message.yaml (1)
111-111: Good improvement to selector strategy.Switching from text/index-based to id-based selector improves test reliability by eliminating dependencies on text content or element position.
.maestro/tests/assorted/setting.yaml (1)
107-112: Good addition of explicit synchronization.Adding
waitForAnimationToEndandextendedWaitUntilbefore tapping ensures the element is fully ready for interaction, reducing flakiness from timing issues in CI environments..maestro/tests/assorted/join-from-directory.yaml (1)
105-109: Verify iOS search behavior and test consistency across all directory search types.The SearchBox component correctly configures
returnKeyType='search'andonSubmitEditing={this.search}, which should trigger search submission on both iOS and Android. However, the test file shows inconsistent behavior: explicitpressKey: enterappears only for the iOS user search (line 109), but not for channel or team searches in the same file.Additionally, no platform-specific iOS mitigations for the "extra request" issue mentioned in the PR objectives are present in the codebase. To confirm this change resolves the stated concerns:
- Clarify why only the user search requires explicit Enter on iOS, while channel and team searches do not
- Verify that
returnKeyType='search'alone is sufficient to trigger submission on iOS, or if the explicitpressKey: enteris necessary as a workaround- Confirm that the extra request and lag issues do not occur on iOS during search
| notification | ||
| }, | ||
| duration: notification.customTime || 3000, // default 3s, | ||
| duration: notification.customTime || process.env.RUNNING_E2E_TESTS ? 60000 : 3000, // default 3s, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Critical: Fix operator precedence bug in duration calculation.
The current expression evaluates as (notification.customTime || process.env.RUNNING_E2E_TESTS) ? 60000 : 3000 due to operator precedence. This means if notification.customTime is set to any truthy value (e.g., 5000ms), the duration will be 60000ms instead of the intended custom time.
Apply this diff to fix the precedence:
-duration: notification.customTime || process.env.RUNNING_E2E_TESTS ? 60000 : 3000, // default 3s,
+duration: notification.customTime || (process.env.RUNNING_E2E_TESTS ? 60000 : 3000), // default 3s📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| duration: notification.customTime || process.env.RUNNING_E2E_TESTS ? 60000 : 3000, // default 3s, | |
| duration: notification.customTime || (process.env.RUNNING_E2E_TESTS ? 60000 : 3000), // default 3s |
🤖 Prompt for AI Agents
In app/containers/InAppNotification/index.tsx around line 62, the duration
expression suffers from operator precedence so it currently evaluates as
(notification.customTime || process.env.RUNNING_E2E_TESTS) ? 60000 : 3000;
change it so the custom time is used when present and only fall back to the
E2E/test check otherwise: replace the expression with either
notification.customTime ?? (process.env.RUNNING_E2E_TESTS ? 60000 : 3000) or
notification.customTime || (process.env.RUNNING_E2E_TESTS ? 60000 : 3000) to
ensure the ternary applies only to the test flag.
Proposed changes
This pull request fixes two flaky tests that were failing intermittently:
Issues Observed and it's fix
Join From Directory:
When the test types the room name in the search bar and presses the Enter key, it triggers an additional request to fetch the list of rooms. This causes laggy behavior in the CI environment. To fix this, I removed the Enter key step.
In-App Notification:
The notification is displayed for only 3 seconds, which is too short for Maestro to detect and interact with it in the CI environment. By the time Maestro tries to tap the notification, it’s already dismissed, causing the test to fail.
I have increased the notification timeout to 1 minute for E2E builds to ensure stable detection.
Flaky Test Link: https://github.com/RocketChat/Rocket.Chat.ReactNative/actions/runs/19138279757/job/54698084402?pr=6766#step:9:266
Working Action Link
Issue(s)
How to test or reproduce
Screenshots
Types of changes
Checklist
Further comments
Depends on: #6752
Summary by CodeRabbit
Bug Fixes
Chores