Skip to content

Conversation

@jvJUCA
Copy link
Member

@jvJUCA jvJUCA commented Jul 17, 2025

This PR delivers the fully integrated Eye Tracking module for RuxaiLab, including browser-based iris capture using TensorFlow.js, calibration interface, backend predictions of gaze/fixation points, and real-time visualizations across three interactive modes as users perform tasks (including screen recording).


🔧 Scope & Key Features

  • Iris/Gaze Capture via TensorFlow.js

    • Implemented client-side iris / eyeball tracking directly in the browser using TensorFlow.js.
    • Calibration flow: new front-end screens redirecting users into the calibration phase before the main test.
    • Seamless transition from calibration → main test interface, so that gaze data is captured continuously and synced with the task.
  • Backend Prediction API Integration

    • Created or integrated an API endpoint that receives iris/gaze data and returns predictions (fixation points, gaze heatmaps, etc.).
    • The front-end now sends captured data in real-time (or near real-time) and receives processed results, enabling synchronized analysis of the user’s gaze during test tasks.
    • Screen recording of the task is captured and coordinated with gaze data, enabling playback + analytics.
  • Real-Time Visualization Modes

    • Three distinct modes for visualizing gaze/fixation data in real-time:
      1. Precision Points Mode (blue theme) — shows raw individual gaze/fixation points overlaid on the screen.
      2. Heatmap Mode (orange theme) — shows aggregated intensity of gaze/fixations over time.
      3. Free Eye Mode (red theme) — continuous live mode showing gaze path + point history.
    • Each mode features distinct color schemes and active-state UI indicators for clarity and switchability during the test session.
    • The visualization is synchronized with the screen/video recording of the task — enabling playback + heat/point replay after the test completes.

📈 UI/UX & Performance Improvements

  • Added full front-end flows: calibration, data capture, live test, visualization panel, playback screen.
  • Enhanced the UI: large buttons, full-width toggles for mode selection, clear visual feedback (loading states, active modes, error handling).
  • Screen recording integration: Task video recording is linked with gaze data so researchers can review both in sync.
  • Optimized performance: minimized latency between gaze data capture and backend predictions display; improved responsiveness of visualization component.
  • Ensured responsive design and layout consistency across desktop and mobile form-factors.

🧠 Impact

With this PR merged, RuxaiLab now supports a comprehensive eye-tracking workflow entirely in the browser + backend pipeline: calibration → data capture → prediction → visualization → playback.
This significantly expands the platform’s capabilities for usability testing and gaze analytics, making remote testing with eye-tracking accessible and powerful.


🚀 Next Steps

  • Export and download of gaze/fixation data + screen video for offline analysis.
  • Integration of gaze/fixation metrics with task analytics (task completion times, errors, affective states).
  • Further calibration refinement and accuracy improvements via user-study data.
  • Optional: deeper analytics overlays such as AOI (Areas of Interest) heatmap validation, gaze-based metrics (e.g., time to first fixation, revisit count).

@jvJUCA jvJUCA added the GSoC GSoC-specific work label Jul 17, 2025
@github-actions
Copy link

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

@jvJUCA jvJUCA requested a review from marcgc21 July 17, 2025 23:15
@jvJUCA jvJUCA assigned jvJUCA and marcgc21 and unassigned jvJUCA Jul 17, 2025
@github-actions
Copy link

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

1 similar comment
@github-actions
Copy link

github-actions bot commented Aug 6, 2025

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

@gitguardian
Copy link

gitguardian bot commented Aug 26, 2025

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secret in your pull request
GitGuardian id GitGuardian status Secret Commit Filename
16386405 Triggered Generic Password bc1e56b src/app/plugins/locales/es.json View secret
🛠 Guidelines to remediate hardcoded secrets
  1. Understand the implications of revoking this secret by investigating where it is used in your code.
  2. Replace and store your secret safely. Learn here the best practices.
  3. Revoke and rotate this secret.
  4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

@github-actions
Copy link

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

20 similar comments
@github-actions
Copy link

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

@github-actions
Copy link

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

@github-actions
Copy link

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

@github-actions
Copy link

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

@github-actions
Copy link

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

@github-actions
Copy link

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

@github-actions
Copy link

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

@github-actions
Copy link

github-actions bot commented Sep 2, 2025

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

@github-actions
Copy link

github-actions bot commented Sep 4, 2025

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

@github-actions
Copy link

github-actions bot commented Sep 4, 2025

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

@github-actions
Copy link

github-actions bot commented Sep 4, 2025

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

@github-actions
Copy link

github-actions bot commented Sep 4, 2025

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

@github-actions
Copy link

github-actions bot commented Sep 4, 2025

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

@github-actions
Copy link

github-actions bot commented Sep 8, 2025

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

@github-actions
Copy link

github-actions bot commented Sep 8, 2025

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

@github-actions
Copy link

github-actions bot commented Sep 8, 2025

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

@github-actions
Copy link

github-actions bot commented Sep 9, 2025

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

@github-actions
Copy link

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

@github-actions
Copy link

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

@github-actions
Copy link

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

@github-actions
Copy link

⚠️ The tests have failed, @KarinePistili Please review the proposed changes.

@github-actions
Copy link

⚠️ The tests have failed, @KarinePistili Please review the proposed changes.

@github-actions
Copy link

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

@github-actions
Copy link

⚠️ The tests have failed, @KarinePistili Please review the proposed changes.

@github-actions
Copy link

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

@github-actions
Copy link

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

@github-actions
Copy link

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

@github-actions
Copy link

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

@github-actions
Copy link

github-actions bot commented Nov 6, 2025

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

@github-actions
Copy link

github-actions bot commented Nov 9, 2025

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

@jvJUCA jvJUCA assigned jvJUCA and unassigned marcgc21 Nov 9, 2025
@jvJUCA jvJUCA changed the title [GSOC-25] Eye tracking per task implementation [GSoC-25] Eye Tracking Module - Full Capture, Calibration & Real-Time Visualization Integration Nov 9, 2025
@sonarqubecloud
Copy link

Quality Gate Failed Quality Gate failed

Failed conditions
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

@github-actions
Copy link

⚠️ The tests have failed, @jvJUCA Please review the proposed changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

GSoC GSoC-specific work

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants