Skip to content

Conversation

@Rohit3523
Copy link
Collaborator

@Rohit3523 Rohit3523 commented Nov 6, 2025

Proposed changes

This pull request fixes two flaky tests that were failing intermittently:

  • Join From Directory
  • In-App Notification

Issues Observed and it's fix

  • Join From Directory:
    When the test types the room name in the search bar and presses the Enter key, it triggers an additional request to fetch the list of rooms. This causes laggy behavior in the CI environment. To fix this, I removed the Enter key step.

  • In-App Notification:
    The notification is displayed for only 3 seconds, which is too short for Maestro to detect and interact with it in the CI environment. By the time Maestro tries to tap the notification, it’s already dismissed, causing the test to fail.
    I have increased the notification timeout to 1 minute for E2E builds to ensure stable detection.

Flaky Test Link: https://github.com/RocketChat/Rocket.Chat.ReactNative/actions/runs/19138279757/job/54698084402?pr=6766#step:9:266

Working Action Link

Issue(s)

How to test or reproduce

Screenshots

Types of changes

  • Bugfix (non-breaking change which fixes an issue)
  • Improvement (non-breaking change which improves a current function)
  • New feature (non-breaking change which adds functionality)
  • Documentation update (if none of the other choices apply)

Checklist

  • I have read the CONTRIBUTING doc
  • I have signed the CLA
  • Lint and unit tests pass locally with my changes
  • I have added tests that prove my fix is effective or that my feature works (if applicable)
  • I have added necessary documentation (if applicable)
  • Any dependent changes have been merged and published in downstream modules

Further comments

Depends on: #6752

Summary by CodeRabbit

  • Bug Fixes

    • Improved test reliability for iOS platform-specific key input handling.
    • Enhanced animation handling for settings view interactions to prevent timing issues.
  • Chores

    • Refined test selector strategies for more accurate element targeting.
    • Updated in-app notification duration configuration to support custom timing values.

@Rohit3523 Rohit3523 temporarily deployed to approve_e2e_testing November 6, 2025 14:02 — with GitHub Actions Inactive
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Nov 6, 2025

Walkthrough

These changes refine E2E test flows by implementing platform-specific iOS key input handling, improving element visibility wait conditions, switching tap selectors from text/index-based to id-based targeting, and adjusting in-app notification duration to extend timeouts during E2E test execution.

Changes

Cohort / File(s) Summary
E2E Test Flow Adjustments
.maestro/tests/assorted/join-from-directory.yaml, .maestro/tests/assorted/setting.yaml, .maestro/tests/room/jump-to-message.yaml
Modified test flows to use conditional platform-specific runFlow for iOS key input, replaced direct Enter key presses, switched element selector from text/index-based to id-based targeting, and added waitForAnimationToEnd with extendedWaitUntil for improved element visibility handling.
Notification Duration Logic
app/containers/InAppNotification/index.tsx
Updated duration determination to prioritize notification.customTime if provided, extend timeout to 60000 ms when RUNNING_E2E_TESTS environment variable is truthy, otherwise default to 3000 ms.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

  • Consider the conditional logic in InAppNotification/index.tsx and its interaction with the E2E test environment flag to ensure timeout behavior is correct across all test scenarios.
  • Verify that the id-based selector in jump-to-message.yaml consistently targets the intended element across different device states and UI layouts.
  • Validate that waitForAnimationToEnd in setting.yaml doesn't introduce unnecessary delays or race conditions in the test flow.

Possibly related PRs

Suggested reviewers

  • diegolmello

Poem

🐰 With iOS keys now running smooth,
And waits that honor the animation's groove,
ID-based taps find their way,
Notifications heed E2E's sway,
Testing flows now fit the bill!

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'chore: fix flaky test' accurately summarizes the main objective of the pull request, which is to fix intermittently failing end-to-end tests.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch join-from-directory-flaky-fix

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@Rohit3523 Rohit3523 temporarily deployed to approve_e2e_testing November 6, 2025 14:47 — with GitHub Actions Inactive
@Rohit3523 Rohit3523 had a problem deploying to experimental_android_build November 6, 2025 14:50 — with GitHub Actions Error
@Rohit3523 Rohit3523 had a problem deploying to official_android_build November 6, 2025 14:50 — with GitHub Actions Error
@Rohit3523 Rohit3523 had a problem deploying to experimental_ios_build November 6, 2025 14:50 — with GitHub Actions Error
@Rohit3523 Rohit3523 changed the title chore: fix flaky test for join-from-directory chore: fix flaky test Nov 6, 2025
@Rohit3523 Rohit3523 temporarily deployed to approve_e2e_testing November 6, 2025 16:46 — with GitHub Actions Inactive
@Rohit3523 Rohit3523 had a problem deploying to experimental_android_build November 6, 2025 16:49 — with GitHub Actions Error
@Rohit3523 Rohit3523 had a problem deploying to official_android_build November 6, 2025 16:49 — with GitHub Actions Error
@Rohit3523 Rohit3523 had a problem deploying to experimental_ios_build November 6, 2025 16:49 — with GitHub Actions Error
@Rohit3523 Rohit3523 temporarily deployed to approve_e2e_testing November 7, 2025 16:54 — with GitHub Actions Inactive
@Rohit3523 Rohit3523 had a problem deploying to experimental_android_build November 7, 2025 16:59 — with GitHub Actions Error
@Rohit3523 Rohit3523 had a problem deploying to official_android_build November 7, 2025 16:59 — with GitHub Actions Error
@Rohit3523 Rohit3523 had a problem deploying to experimental_ios_build November 7, 2025 16:59 — with GitHub Actions Error
@Rohit3523 Rohit3523 temporarily deployed to approve_e2e_testing November 7, 2025 17:39 — with GitHub Actions Inactive
@Rohit3523 Rohit3523 had a problem deploying to official_android_build November 7, 2025 17:42 — with GitHub Actions Error
@Rohit3523 Rohit3523 had a problem deploying to experimental_android_build November 7, 2025 17:42 — with GitHub Actions Error
@Rohit3523 Rohit3523 temporarily deployed to approve_e2e_testing November 8, 2025 15:09 — with GitHub Actions Inactive
@Rohit3523 Rohit3523 requested a deployment to official_android_build November 8, 2025 15:13 — with GitHub Actions Waiting
@Rohit3523 Rohit3523 requested a deployment to experimental_ios_build November 8, 2025 15:13 — with GitHub Actions Waiting
@Rohit3523 Rohit3523 requested a deployment to experimental_android_build November 8, 2025 15:13 — with GitHub Actions Waiting
@Rohit3523 Rohit3523 marked this pull request as ready for review November 10, 2025 18:19
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

Disabled knowledge base sources:

  • Jira integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 3532447 and 0fc3f8f.

📒 Files selected for processing (4)
  • .maestro/tests/assorted/join-from-directory.yaml (1 hunks)
  • .maestro/tests/assorted/setting.yaml (1 hunks)
  • .maestro/tests/room/jump-to-message.yaml (1 hunks)
  • app/containers/InAppNotification/index.tsx (1 hunks)
🔇 Additional comments (3)
.maestro/tests/room/jump-to-message.yaml (1)

111-111: Good improvement to selector strategy.

Switching from text/index-based to id-based selector improves test reliability by eliminating dependencies on text content or element position.

.maestro/tests/assorted/setting.yaml (1)

107-112: Good addition of explicit synchronization.

Adding waitForAnimationToEnd and extendedWaitUntil before tapping ensures the element is fully ready for interaction, reducing flakiness from timing issues in CI environments.

.maestro/tests/assorted/join-from-directory.yaml (1)

105-109: Verify iOS search behavior and test consistency across all directory search types.

The SearchBox component correctly configures returnKeyType='search' and onSubmitEditing={this.search}, which should trigger search submission on both iOS and Android. However, the test file shows inconsistent behavior: explicit pressKey: enter appears only for the iOS user search (line 109), but not for channel or team searches in the same file.

Additionally, no platform-specific iOS mitigations for the "extra request" issue mentioned in the PR objectives are present in the codebase. To confirm this change resolves the stated concerns:

  1. Clarify why only the user search requires explicit Enter on iOS, while channel and team searches do not
  2. Verify that returnKeyType='search' alone is sufficient to trigger submission on iOS, or if the explicit pressKey: enter is necessary as a workaround
  3. Confirm that the extra request and lag issues do not occur on iOS during search

notification
},
duration: notification.customTime || 3000, // default 3s,
duration: notification.customTime || process.env.RUNNING_E2E_TESTS ? 60000 : 3000, // default 3s,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Critical: Fix operator precedence bug in duration calculation.

The current expression evaluates as (notification.customTime || process.env.RUNNING_E2E_TESTS) ? 60000 : 3000 due to operator precedence. This means if notification.customTime is set to any truthy value (e.g., 5000ms), the duration will be 60000ms instead of the intended custom time.

Apply this diff to fix the precedence:

-duration: notification.customTime || process.env.RUNNING_E2E_TESTS ? 60000 : 3000, // default 3s,
+duration: notification.customTime || (process.env.RUNNING_E2E_TESTS ? 60000 : 3000), // default 3s
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
duration: notification.customTime || process.env.RUNNING_E2E_TESTS ? 60000 : 3000, // default 3s,
duration: notification.customTime || (process.env.RUNNING_E2E_TESTS ? 60000 : 3000), // default 3s
🤖 Prompt for AI Agents
In app/containers/InAppNotification/index.tsx around line 62, the duration
expression suffers from operator precedence so it currently evaluates as
(notification.customTime || process.env.RUNNING_E2E_TESTS) ? 60000 : 3000;
change it so the custom time is used when present and only fall back to the
E2E/test check otherwise: replace the expression with either
notification.customTime ?? (process.env.RUNNING_E2E_TESTS ? 60000 : 3000) or
notification.customTime || (process.env.RUNNING_E2E_TESTS ? 60000 : 3000) to
ensure the ternary applies only to the test flag.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants