Use TranscodingStream for converting to UTF-8

[erwinkramer]

This leverages CreateTranscodingStream to change the encoding to UTF-8 when required. Because this only exists on .NET 5 and up, I made it conditional and kept the old logic for lower .NET versions.

[jskeet]

This feels like it's reasonable - it's entirely an implementation detail, so a simple #if would be fine IMO.

[erwinkramer]

@jskeet sure, I made it conditional now. Should be good to go.

[erwinkramer]

@jskeet letting you know this one's still waiting for you

[jskeet]

Thanks, thought I'd reviewed it yesterday - will have another look when I get a chance

[jskeet]

Ah, no - I reviewed a different one before. Will look at this next weekend. (Sorry to be slow on these.)

[jskeet]

Change looks good. I'm slightly worried that we're not testing the old path, as our tests only run on .NET 8 and .NET 10.

If you have any thoughts on how we can address that in this PR, great - otherwise I'll merge in 48 hours. Potentially we should have tests that run on .NET Framework, with suitable "only on Windows" checks etc...

[erwinkramer]

we should have tests that run on .NET Framework, with suitable "only on Windows" checks etc...

Yeah, in general we target 4.x so makes sense to also build and run the tests during the GitHub workflow for that target. I guess we can do this in a separate PR.

[jskeet]

Adding testing in a separate PR works for me. Merging.