fix: Correct dpad axis mapping and add support for start/select/home buttons (iOS)

[wantroba]

I rewrote GamepadsIosPlugin.swift following Apple's documentation https://developer.apple.com/documentation/gamecontroller and also keeping the plugin consistent and working.

With this I fixed the dpad axis mapping issue that appeared in #64. In addition, now the buttons buttonMenu, buttonOptions and buttonHome are also mapped when available.

Note: The only thing I couldn't keep was the timestamp that used to come from the event, now the time is generated by the plugin. If there is a need and a way to retrieve this value please let me know or correct my code for it.

[luanpotter]

LGTM, I think genering the timestamp is totally fine if one is not provided by the platform.

[gyenesvi]

Hi,

It seems to me that this fix introduced some incorrect behaviour that was working correctly before. I recently updated the gamepads package to the latest version and noticed that all button keys have been renamed on iOS, which in itself is a breaking change, but I can understand that with this package being under development. So I did a remapping, and I realised that the left/right triggers on an Xbox Series controller are now acting as non-analog buttons, whereas previously were acting as analog ones. Also, when I tested my controller with other apps, the triggers were read as analog ones. Then I saw this update, in which it seems the whole status update logic has been reworked with the button keys remapped. In this code, the triggers are explicitly treated as non-analog buttons, so no wonder for the observed behaviour. I had to go back to version 0.1.2+2 of the gamepads_ios package to get back the analog behaviour, with which it is indeed working again.

Another missing feature I noticed is that the 'home' button on the Xbox controller isn't getting mapped with the latest version, while it did get mapped with the older version of the code (the 'options' and 'menu' buttons do work).

Also, the d-pad is now mapped as an analog button, whereas previously it was non-analog, which seems to be a better fit. Are there controllers on which the signal is actually analog? Also, the older version is working fine for me for the dead, getting -1, 0, 1 values across both axes.

I'm not sure what the required change would be for these issues, but from the older version working properly I guess it should be doable, and it should be fixed in order not to lose previously existing functionality.

By the way, can I ask what was the fundamental problem with the previous code logic that warranted the substantial reworking?

@wantroba could you please look into these issues?

[wantroba]

@gyenesvi Thank you for testing my fix and finding these strange behaviors. Did you take a look at my initial issue #64 ?

If possible, test with the previous version before the PR, and see if that behavior also happened to you? For me it was incorrect and I explained why there.

The only thing I changed was this file: packages/gamepads_ios/ios/Classes/GamepadsIosPlugin.swift
and following the instructions in Apple's documentation to perform the correct mapping: https://developer.apple.com/documentation/gamecontroller. After that, the behavior became correct for me. But if you notice anything wrong, just let me know and I'll fix it.

I don't know anything about the Xbox problem because I haven't worked on that part, but when I test my Windows app using a wired Xbox 360 controller, it works normally.

By the way, I'm not one of the developers of this library, just a user who made this small fix on the iOS side, but I'm willing to help in any way I can.

[gyenesvi]

Thanks for the response @wantroba.

Yes, I did see the original issue, and I did test with the previous version, and it was working fine for me, the dpad was giving -1, 0, 1 values in both axes. Anyways, it might be that it was not working with other versions of iOS or other controllers. Indeed, it seems from the documentation link that this is the right way to handle the game controller.

However, as I wrote above, I think the problem is the way you handle the trigger buttons. For all buttons in the list, you only acquire its pressed state, not the analog value, and you explicitly set isAnalog: false in sendEvent. This is okay for all other buttons, but the triggers should be handled separately, as they provide analog values.

To fix this, the GCControllerButtonValueChangedHandler has 3 arguments, button, value and pressed. For the triggers, you should be using the value instead of pressed, and set isAnalog: true in sendMessage.

Another minor issue I see is that the dpad is giving analog values, although they are non-analog buttons. Each GCControllerElement has an isAnalog property that tells whether it is providing analog input or not. You could query and use that property instead of the fixed true/false values when invoking sendEvent and passing the isAnalog flag.

What do you think? Could you implement these fixes?

[wantroba]

Hi @gyenesvi ! Thanks a lot for the detailed feedback and for taking the time to test the changes.

You were absolutely right about the trigger buttons. In the previous version they were handled the same way as the other digital buttons, using only the pressed state. As you pointed out, GCControllerButtonInput for the triggers provides an analog value in the range 0.0–1.0, so using only pressed was losing that information.
I’ve updated the implementation so that leftTrigger and rightTrigger are now handled separately and send their analog value (value) instead of a boolean state.

Regarding the D-pad, I kept it reporting axis values (dpad - xAxis and dpad - yAxis) as analog events. While the D-pad itself is not truly analog, the GCDirectionPad API exposes it as axis values (-1, 0, 1). Keeping it mapped as axes maintains compatibility with how the library already exposes directional inputs and how other platforms (and many game engines) treat the D-pad. For that reason I didn’t switch to dynamically checking element.isAnalog.

Please test my adjust changing your pubspec:

  gamepads:
    git:
      url: https://github.com/wantroba/gamepads.git
      path: packages/gamepads
      ref: 3199d7123b5adb4e873676a1021134143c29d3f0

Note: I don't have a mac here right now to test so please let me know if any errors popup ;)

[spydon]

@wantroba can you open a PR with the fix? :)

[wantroba]

Sure @spydon. Once @gyenesvi validates and sees that everything is correct, I'll do some more tests in my device and open the PR ;)

[gyenesvi]

Thanks for the quick fix @wantroba! Regarding testing, I'm not sure I know how to do that directly from your git fork, I have never done that in flutter/dart..

About the dpad values, I think being directional and being analog are two orthogonal concepts, and they are not mutually exclusive. So a directional value can be analog (-1.0 to 1.0) or discrete (-1, 0, 1), the same way as a non-directional button can be analog (0.0 to 1.0), or discrete (0, 1). So a dpad would be directional but non-analog. In the previous version it was exposed like that (the code was using element.isAnalog), so I don't think it would go against the library concepts. But maybe we could ask @spydon or @luanpotter to chime in on this question, what's the design philosophy about this in the library? Is it to adhere to how the underlying platform APIs report the button types, or is it to impose a fixed/unified mapping across platforms? I think this should be kept consistent one way or the other.

Other than these, I found the answer to another issue as well. The current implementation is missing (not reporting) one more button on my controller that was also working with the older version. On the Xbox controller, there is also a shareButton, which is not exposed by the GCExtendedGamepad explicitly, rather it has various subclasses, such as the GCXboxGamepad, which exposes that extra button. So in order to get full coverage of the buttons (not lose ones that were working before), those subclasses should be handled as well (others subclasses are GCDualShockGamepad and GCDualSenseGamePad). The only unclear part for me is that the old implementation was also using GCExtendedGamepad, but through a different callback mechanism, and in that that extra share button was also reported somehow. So maybe there's a way to actually get the all the buttons' updates from the GCExtendedGamepad without going through all the possible derived classes (via runtime casts I guess); that might be a more reliable approach, but I could not find clear info on it in the docs. What I found is that the GCExtendedGamepad has a valueChangedHandler itself, maybe that reports all the buttons, including the ones in the derived classes.

[wantroba]

@gyenesvi To test using my Flutter repository, it's very simple; just replace the import of the libraries in your pubspec, which should look something like this:

gamepads: ^0.1.9

replace to:

  gamepads:
    git:
      url: https://github.com/wantroba/gamepads.git
      path: packages/gamepads
      ref: 3199d7123b5adb4e873676a1021134143c29d3f0

Regarding your other questions, I think it's best to check with the library maintainers ;)

[gyenesvi]

@wantroba okay, testing your repo may be easier than I thought :) However, there's a catch, you gave me a link to your version of the gamepads package, but the actual changes are in the gamepads_ios package. This git version of your gamepads package depends on the official version 0.1.3+1 of the gamepads_ios package, which does not have those changes, so nothing has changed on my side this way. I think you need to give me link to your version of the gamepads_ios package. Is that also this simple to point to, or that being a platform implementation will require some kind of prior recompilation?

[gyenesvi]

By the way, @wantroba, it would be interesting to understand what happens on your device if you use the generic valueChangedHandler method of the GCExtendedGamepad object instead of the handlers of the individual buttons, as you do now. In that case, would you get the correct range for the dpad that was your original issue? That would help decide what would be better to get all button values: to handle all GCExtendedGamepad subclasses, or to use the generic valueChangedHandler method of the GCExtendedGamepad.

Would you maybe give it a quick try just to test that locally on your side, whether the dpad issue (and all other buttons) works that way? No need to make a proper implementation now, just to debug print some event handling on your device and tell us what happens.

[spydon]

@gyenesvi you can use a pubspec_override with a git dependency on gamepads_ios and still use the gamepads package from pub`

[spydon]

But maybe we could ask @spydon or @luanpotter to chime in on this question, what's the design philosophy about this in the library? Is it to adhere to how the underlying platform APIs report the button types, or is it to impose a fixed/unified mapping across platforms? I think this should be kept consistent one way or the other.

At some point we should try to unify the output, see #11.

@wantroba I think we're ready for a PR now? :)

[gyenesvi]

@wantroba sorry for the delay, I was on holiday, I am trying to test your fix. In order to get your fix for the gamepads_ios package instead of the gamepads package, I am now using

  gamepads_ios:
    git:
      url: https://github.com/wantroba/gamepads.git
      path: packages/gamepads_ios
      ref: 3199d7123b5adb4e873676a1021134143c29d3f0

Is that the right way to do it? I am getting weird compile errors for the modified GamepadsIosPlugin.swift :

Swift Compiler Error (Xcode): Cannot use optional chaining on non-optional value of type 'GCControllerButtonInput'
.../.pub-cache/git/gamepads-3199d7123b5adb4e873676a1021134143c29d3f0/packages/gamepads_ios/ios/Classes/GamepadsIosPlugin.swift:89:23

The weird part is that line 89 contains a comment only.. so I'm not sure what's going on.

@spydon about unifying the outputs, I see that it happened recently, but not sure I understand what that actually includes if this fix is not included yet. Is it only the button/axis names, or is it mean to unify the values as well? I mean the trigger is known to be giving the wrong output for now, so it's probably not unified :) Furthermore, the documentation of normalization refers to ios names like a.circle but this implementation does not actually use such names any more, rather a custom one (buttonA).
About the dpad values, this implementation returns an analog range for the two axes, but in the normalized version it seems to be remapped to non-analog set of buttons, is that right?

What about the shareButton on Xbox controller? Is that not part of the unified button list? I don't see that listed in the documentation.

I have the feeling that this ios implementation currently adds a middle layer of remapping, which does not match neither the original ios mapping, nor the normalized one, which could be confusing.

[spydon]

Do live activities persist longer than the application?

Both names and values should now be unified on the normalized events.

About the dpad values, this implementation returns an analog range for the two axes, but in the normalized version it seems to be remapped to non-analog set of buttons, is that right?

Correct.

What about the shareButton on Xbox controller?

That one would have to come from the raw event, since it's specific for that controller and not something generic.

[wantroba]

@wantroba sorry for the delay, I was on holiday, I am trying to test your fix. In order to get your fix for the gamepads_ios package instead of the gamepads package, I am now using

  gamepads_ios:
    git:
      url: https://github.com/wantroba/gamepads.git
      path: packages/gamepads_ios
      ref: 3199d7123b5adb4e873676a1021134143c29d3f0

Is that the right way to do it? I am getting weird compile errors for the modified GamepadsIosPlugin.swift :

Swift Compiler Error (Xcode): Cannot use optional chaining on non-optional value of type 'GCControllerButtonInput'
.../.pub-cache/git/gamepads-3199d7123b5adb4e873676a1021134143c29d3f0/packages/gamepads_ios/ios/Classes/GamepadsIosPlugin.swift:89:23

The weird part is that line 89 contains a comment only.. so I'm not sure what's going on.

@spydon about unifying the outputs, I see that it happened recently, but not sure I understand what that actually includes if this fix is not included yet. Is it only the button/axis names, or is it mean to unify the values as well? I mean the trigger is known to be giving the wrong output for now, so it's probably not unified :) Furthermore, the documentation of normalization refers to ios names like a.circle but this implementation does not actually use such names any more, rather a custom one (buttonA). About the dpad values, this implementation returns an analog range for the two axes, but in the normalized version it seems to be remapped to non-analog set of buttons, is that right?

What about the shareButton on Xbox controller? Is that not part of the unified button list? I don't see that listed in the documentation.

I have the feeling that this ios implementation currently adds a middle layer of remapping, which does not match neither the original ios mapping, nor the normalized one, which could be confusing.

Sorry, but I accidentally put a "?" in an element that wasn't optional. I checked the documentation here and corrected it. Could you test again using my last commit?

ref: f3b0e9947156c23140d05fd40526b80d2397e500

I don't have a Mac to debug, so I'll wait for your confirmation ;)

[gyenesvi]

@wantroba I have checked it and is working fine now, I am getting the continuous trigger outputs.

How do you test these modifications if you don't have iOS? I mean the original issue was that it wasn't working on iOS, no?

[wantroba]

@wantroba I have checked it and is working fine now, I am getting the continuous trigger outputs.

How do you test these modifications if you don't have iOS? I mean the original issue was that it wasn't working on iOS, no?

Great!! So it's working as expected? Is there anything else that needs to be done?

Regarding how I made the modification without owning a Mac: I rewrote the logic following Apple's documentation at the time, generated the build using Code Magic, installed the app via Testflight, and observed its behavior. Since the entire library was working perfectly, and only the iOS part was exhibiting strange behavior, I thought it was worthwhile to make the modifications and try to improve it.

This time I asked you to test it because I wasn't working on the project that uses this library and couldn't do all the work to validate it.

[gyenesvi]

@wantroba Well the trigger is working now, but as I wrote above there are other things that could be fixed (extra buttons that are currently not mapped but were mapped before, and the mapping names are not what iOS specifies). But maybe I will open a new issue about those, as this way they will be more visible (this PR is already merged so it is probably not tracked).

Are you going to create a PR with this fix?

[wantroba]

@wantroba Well the trigger is working now, but as I wrote above there are other things that could be fixed (extra buttons that are currently not mapped but were mapped before, and the mapping names are not what iOS specifies). But maybe I will open a new issue about those, as this way they will be more visible (this PR is already merged so it is probably not tracked).

Are you going to create a PR with this fix?

Perfect. I think it would be better if you opened another issue to discuss the other things. I'll create a PR that fixes the triggers with exactly those 2 commits that I made and we tested.

[gyenesvi]

Okay, thanks!

[wantroba]

Hello @gyenesvi and @spydon, PR Created: #94