Unexpected freezes with VP9 SVC and packetloss

[geirbakke]

Bug Report

Your environment

• Chrome 140,
• MacOS 15.6.1
• Mediasoup online demo

Issue description

We see lots of freezes when introducing packetloss on reception of VP9 SVC. Seems to only happen when using SVC - not K-SVC.

Reproducible with mediasoup demo when forcing VP9 and overriding default scalabilitymode:
https://v3demo.mediasoup.org/?roomId=vp9svcpacketloss&forceVP9=true&webcamScalabilityMode=L3T3

I think this might be a browser/lib issue, so I have reported it there:
https://issues.webrtc.org/issues/449408585
But if i'm wrong and it is a mediasoup issue, hopefully you can use this info to track it down

I narrowed it down to the freezes happening when dropping a single packet and then dropping the first re-transmit of that packet. Somehow we only receive a new nack (re-nack same seq number) if it is the first packet in a picture (first packet after a packet with marker bit being sent). Since it doesn't get the re-nack, the receiver never gets what it needs and gives up after a while and sends a PLI which unfreezes the image.

I can reproduce by:

• running latest release and testing on localhost (no network lag or packetloss)
• dropping every Nth (i.e 200th) packet in WebRtcTransport (by not calling this->iceServer->GetSelectedTuple()->Send(data, len, cb);)
• drop the first RTX packet after this

Logging packets i see it only freezes when not re-nacking the packet i stopped retransmission of. And it only does that when it isn't the first packet of picture.

Maybe related to:
#1536

[ibc]

Thanks for the report. We will take a look when we get some time. But I just wonder how this can be an issue in mediasoup if the browser doesn't retransmit a NACK if it didn't receive a packet retranmission...

[jmillan]

Very well detailed @geirbakke, I'm following the chrome issue 👍

[geirbakke]

I'm not 100% sure the browser doesn't retransmit nack. Didn't wireshark, only custom logged in mediasoup and didn't see any nack there. I'm not familiar with mediasoup code, but i thought in theory maybe some VP9 or RTP rewriting could make the browser not NACK? I don't know, it doesn't make sense either way to me, browser or mediasoup, why i don't see that nack when there is a gap in sequence and image freezes only to recover after a PLI coming long after.

[jmillan]

@geirbakke, it would be really useful if you could sniff and check with wireshark that the NACK indeed is not sent by the browser.

[jmillan]

I'll investigate this.

[geirbakke]

I have confirmed it with wireshark. It never sends the second nack. If i tweak the code to only drop when the previous packet had market bit set, it never freezes and always re-nacks. If previous packet marker bit isn't set, it will only nack once and freeze

[jmillan]

Thanks @geirbakke, could you please share this info in the chrome issue, sharing the details and the .pcap too? This will likely bring their attention.

[jmillan]

@geirbakke, PR done #1620. It's working nicely in my tests. Can you please give it a try?

[geirbakke]

thx @jmillan, but i'm still afraid it might not be a mediasoup issue. As far as i understand it the marker bit was already correctly set. Now, if there is a bug running in the browser preventing repeated nacks happening when in the middle of a picture, then there will of course be less of this happening if adding a lot of those marker bits. but I fear this workaround can have side effects. Anyway i will test it next week, and if it was indeed a bug in how mediasoup sets marker bit, this PR should reduce freezes to 0 in my targeted test. Also commented in webrtc.org thread.

[jmillan]

The spec says that the marker must be set:

This bit MUST be set to one for the final packet of the highest spatial-layer frame (the final packet of the picture); otherwise, it is zero. Unless spatial scalability is in use for this picture, this bit will have the same value as the E bit described in [Section 4.2](https://www.rfc-editor.org/rfc/rfc9628.html#VP9payloadDescriptor). Note this bit MUST be set to one for the target spatial-layer frame if a stream is being rewritten to remove higher spatial layers.

This: " Note this bit MUST be set to one for the target spatial-layer frame if a stream is being rewritten to remove higher spatial layers."

We are not rewritting anything in VP9. So we are now being compliant, and it's actually working perfectly. Since we are not signaling by any means (inband our out of band) to the decoder which is the target spatial layer, it's impossible it knows, and hence we were getting the freezes before.

[geirbakke]

When you subscribe to a lower layer, you are rewriting, as you are dropping, and at least altering the marker bit, afaiu.

I think

This bit MUST be set to one for the target spatial layer frame if a stream i being rewritten

because unless you do that, you might never get the marker bit (if filtering out the highest spatial layer). And you did that properly before, at least since you last fixed this (before that there was problems subscribing to a lower layer SVC stream afaiu in some cases, probably because of missing marker bit).

We already have the E bit, so why would we need a marker bit that is just mirroring this? The marker bit should signal the end of the picture, not the end of the frame. The E bit is end of frame. The maker bit is end of picture.

I can imagine it is valuable for the decoder to know there more frames coming expanding on the details of the current picture, by looking at (the missing) marker bit.

(the final packet of the picture); otherwise, it is zero
This means afaiu that there should be only 1 packet with marker bit in a picture. and a picture can have many frames. With the merged PR every frame gets the marker bit right? and it is just a mirror of e bit.