MelonDS System Bus

[SuuperW]

This PR allows access to 16-bit and 32-bit reads/writes on the system bus domains for melonDS, resolving #3121. This is done in 2 parts:

1. Create a new type of waterbox memory domain. It is similar to the function hook domain, except this one can expose up to 8 function hooks for various sizes of read and write operations.
2. Change the system bus domains to the new type.

To enable bulk access, I have also given MemoryDomain , IMemoryApi, and MemoryLuaLibrary some new functions.

The behavior of bulk byte operations remains unchanged compared to 2.11.

This bit left in from the original version of this PR:
Regarding hex editor, I don't think there are any great solutions. With this PR, access size in hex editor can be controlled by changing the viewed data size. This is not intuitive. Using sized system bus domains would require the user to change domains, and also leads to the question of what should be done if the user writes a single byte in 1-byte data size mode, to a 32-bit domain? I guess this would have to first read the relevant 4 bytes, change the one byte, then write 4 bytes again. Fortunately, none of this matters for the vast majority of addresses so it's probably not very important as long as it is possible to perform sized access.

Check if completed:

• I have run any relevant test suites
• I, the commit author, have read the licensing terms for contributors (last updated 2024-06-22) and am compliant

[CasualPokePlayer]

This does not give a way to specify to waterbox function hook domains the write size to use. MelonDS will now infer it.

This is very much the wrong way to do it. If I'm specifying a bulk read of multiple bytes (e.g. memory.read_bytes_as_array), the behavior shouldn't change just because I decided to read a different amount or depending on where I decided to start the read. Byte at address X returns what address X has when you peek that byte with PeekByte.

Lua users cannot obviously specify access size

They can, there are a ton of lua functions specifically for this.

C# users cannot obviously specify write size (as MemoryDomain has no such functions)

They can, MemoryDomain does have such functions (that's the entire point of the different poke functions).

Another option would be to provide new function hook domains for 16 and 32 bit access.

You don't need new domains, the function hook just needs to be modified to specify access size and that access size must be sent over for the core to understand how to read/write things.

Ideally different widths shouldn't be changing results in the first place, but the System Bus is inherently quirky to the console (since it is more or less native bus reads), so if different widths change results those are arguably just system quirks which are inherent to the System Bus domain, so under that logic it's acceptable for System Bus domains.

[SuuperW]

the behavior shouldn't change just because I decided to read a different amount or depending on where I decided to start the read.

Just to be clear - for the vast majority of addresses, the behavior does not change depending on alignment or number of bytes read.

They can, there are a ton of lua functions specifically for this.

With the current implementation memory.read_u32_le(0x02000001) will internally do 4 single-byte reads while memory.read_u32_le(0x02000000) will do 1 4-byte read. Similar for read_bytes_as_array and for the C# API. This doesn't matter for most addresses. Except, if we do a 4-byte read with read_u32_le then reading from address 0x02000001 would ALWAYS return the same value as reading from address 0x02000000 since the system bus aligns multi-byte reads and writes. Not a bad thing, but surprising to most people.

You don't need new domains, the function hook just needs to be modified to specify access size and that access size must be sent over for the core to understand how to read/write things.

I didn't want to try modifying every other core that uses function hooks. Although I suppose it could instead be a second kind of function hook. I could do that.

And if we do it this way, the Lua bulk read functions could similarly have a parameter for access size. And corresponding C# API functions...? There's already BulkPeekUshort and BulkPeekUint but no multi-byte bulk poke functions. Maybe Lua should follow suit and have multiple functions instead of adding a parameter to the existing one. That probably makes more sense actually, a new read_u32_le_array would return a table with 32-bit numbers instead of separating each byte.

those are arguably just system quirks which are inherent to the System Bus domain, so under that logic it's acceptable for System Bus domains

Yes, that is my view. IMO there should also be a way to perform system bus reads that have side effects. Maybe this could be accomplished with a new API method that enables read side effects. Making read side effects opt-in would protect users from accidentally causing desyncs.

[YoshiRulz]

Can I veto any new peek or poke APIs until #3472 and/or #3738? At least in ApiHawk; I guess if there's an asymmetry in the Lua API that should be filled.

[SuuperW]

The first one I will look at later. I don't see the relevance of the second, nor would I wish to attempt to implement it in the near future.

[SuuperW]

While reviewing Yoshi's PR, I found this in BaseImplementations/MemoryDomain.cs lines 115-116:

			if ((start & 1) != 0 || (end & 1) != 0)
				throw new InvalidOperationException("The API contract doesn't define what to do for unaligned reads and writes!");

I didn't know this was here, and it doesn't make sense. Why would the bulk read option require aligned access when individual reads do not? And this is for generic memory domains too, where we don't even know if (a) the emulated system cares about memory alignment or (b) 16/32-bit values are always or even usually aligned.

The commit message indicates it's related to Hex Editor ("the only consumer of the API"), which I assume can never perform bulk operations un-aligned because there just isn't a UI option to do that. So this code probably belongs in Hex Editor logic, and not in memory domain logic.

[YoshiRulz]

Someone probably noticed that one of the cores' implementations choked on unaligned access and, rather than add the necessary guards and logic there, forced alignment on everyone.

[CasualPokePlayer]

Unaligned access is complete UB on many platforms. That includes even dereferencing with like *(int*) natively. In practice on modern machines unaligned access usually "just works" with the most UB involved just being on the language constructs (although this typically comes with a minor speed penalty and of course actual raw I/O probably won't like unaligned access). For the machines we emulate many different behaviors could occur (especially relevant for the System Bus): you could just have the same result as if you read 1 byte at a time ("just works"), you could have force alignment (low address bits masked off), sometimes you get some rotated result, sometimes you get garbage, sometimes the machine would just crash / throw a processor exception.

[SuuperW]

Force pushed with a new approach.