Group tracks into streams

[AHGIJMKLKKZNPJKQR]

Description

Camera + mic and screenshare are now grouped into media streams. They are now placed into groups in the sdp (msid is set)

Motivation and Context

Why is this change required? What problem does it solve? If it fixes an open
issue, please link to the issue here.

Documentation impact

• Documentation update required
• Documentation updated in another PR
• No documentation update required

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to
  not work as expected)

[linear]

FCE-2935 Group tracks by stream id

[Copilot]

Pull request overview

This PR updates local publishing so camera+microphone and screenshare A/V tracks can be grouped into shared MediaStreams (so the SDP msid groups them), enabling the remote side to treat related tracks as coming from the same source.

Changes:

• Extend WebRTCEndpoint.addTrack to accept an optional MediaStream, and emit localTrackAdded with that shared stream.
• Introduce LocalStream in ts-client and use it in FishjamClient.addTrack to group camera/mic and screenshare tracks into two stable shared streams.
• Adjust local track replacement stream-mutation behavior to remove only the replaced track (instead of clearing the whole stream).

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
packages/webrtc-client/src/webRTCEndpoint.ts	Allows passing a shared MediaStream into addTrack for msid grouping.
packages/webrtc-client/src/tracks/LocalTrack.ts	Changes how replaceTrack updates the associated stream’s track list.
packages/ts-client/src/index.ts	Exports the new LocalStream utility.
packages/ts-client/src/LocalStream.ts	Adds a helper to manage a shared stream with at most one audio + one video track.
packages/ts-client/src/FishjamClient.ts	Uses shared LocalStreams for camera/mic and screenshare when publishing tracks.

Comments suppressed due to low confidence (2)

packages/webrtc-client/src/tracks/LocalTrack.ts:161

• replaceTrack mutates the shared MediaStream (removing oldTrack / adding newTrack) before calling RTCRtpSender.replaceTrack. If replaceTrack throws, the catch block restores trackContext.track but does not restore the stream’s track list, leaving the stream out of sync with the actual sender/track state. This can break subsequent operations and, with shared streams, can desync A/V grouping. Consider either updating the stream only after sender.replaceTrack succeeds, or performing a full rollback in the catch (remove newTrack from the stream and re-add oldTrack when present).

    if (oldTrack) {
      stream?.removeTrack(oldTrack);
    }

    if (newTrack) {
      stream?.addTrack(newTrack);
    }

    const action = getActionType(this.trackContext.track, newTrack);

    this.trackContext.track = newTrack;
    this.mediaStreamTrackId = newTrack?.id ?? null;

    if (action === 'mute' || action === 'unmute') {
      emitMutableEvents(action, webrtc, trackId);
    }

    try {
      await this.sender.replaceTrack(newTrack);
    } catch (_error) {
      // rollback: emit opposite events and revert internal state
      if (action === 'mute') {
        emitMutableEvents('unmute', webrtc, trackId);
      } else if (action === 'unmute') {
        emitMutableEvents('mute', webrtc, trackId);
      }

      this.trackContext.track = oldTrack;
      this.mediaStreamTrackId = oldTrack?.id ?? null;
    }

packages/ts-client/src/FishjamClient.ts:429

• The localTrackStreamMap is used to find a MediaStreamTrack when handling localTrackRemoved (since the event only contains trackId). After a localTrackReplaced, this map is not updated, so a later localTrackRemoved will attempt to remove the old track from localCameraStream / localScreenShareStream, leaving the current track in the shared stream. Update the map in the localTrackReplaced handler (set to event.track or delete when event.track is null) so stream cleanup stays correct after replacements.

    this.webrtc?.on('localTrackRemoved', (event) => {
      const track = this.localTrackStreamMap.get(event.trackId);
      if (track) {
        this.localCameraStream.removeTrack(track);
        this.localScreenShareStream.removeTrack(track);
        this.localTrackStreamMap.delete(event.trackId);
      }
      this.emit('localTrackRemoved', event);
    });
    this.webrtc?.on('localTrackReplaced', (event) => {
      this.emit('localTrackReplaced', event);
    });

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.