Prefer positional mapping for controllers with a diamond button configuration

[SuperSamus]

Description

The issue linked below should explain everything about the motivation. To paragraph:

On Nintendo controllers, the A/B and X/Y buttons are in a swapped position, compared to the standard on PC that is the Xbox controller.
This means that when playing games with a Nintendo controllers, the prompts that you will see in the games won't match the button you press.
SDL (and Steam) decided that this is very bad, and thus by default the mapping on these controllers needs to be swapped in order to match the prompts on screen.
While I agree for this to be an option, I absolutely disagree for it to be the default, because it creates more problems than it solves, which is why I'm making this PR to change the default to a positional mapping.

These problems are:

• Human fingers don't remember: "I jump with A and attack with X", they remember instead: "I jump with the south button and attack with the west button". When changing controllers, the forces the player to readjust the muscle memory.
• When playing the same game on a Nintendo console, the mapping isn't different: only the prompts are. With a label mapping, if someone is playing a game on a Nintendo console, then uses the same controller for the same game on PC, they'll have to deal with a different mapping.
  • Some games on PC do support Nintendo controllers prompts. However, with a label mapping, this will make games have the wrong mappings and the wrong prompts. Meaning that you shouldn't enable Nintendo prompts on Nintendo controllers.
• Some games are also designed to have buttons in a specific layout. For example, in the Binding of Isaac, the south button shoots down and the east button shoots right. With a label mapping, the south button shoots right and the east button shoots down.
  • Other games do this with prompts: instead of showing the button label, they show the controller diamond and highlight the button to press.
• No other program defaults to a label mapping.
  • When you use an XInput wrapper for a Nintendo controller (like BetterJoy or DS4Windows), it defaults to a positional mapping.
  • When you set the default mapping for a Wii U/Switch emulator with an Xbox controller, it defaults to a positional mapping.

Now, a likely FAQ:

Just set SDL_GAMECONTROLLER_USE_BUTTON_LABELS=0

This isn't an easy to discover option. In fact, even some developers who use SDL aren't aware of this option and use hacky workarounds to have a positional mapping on all controllers (1, 2).

It's also the default on Steam

On Steam the situation is different, since the option is exposed in the UI, and it's impossible not to see it when enabling Switch Pro Controller support.

It's a breaking change

I know, but so was 9a76beb which introduced this to begin with.
Also, I think that the arrival of SDL3 is a good excuse for this.

Existing Issue(s)

Closes #6117.

[slouken]

This is a well thought out pull request, thanks!

Have you tried the new testcontroller? It shows the correct button labels for Nintendo Switch controllers, and allows you to change the type of your controller.

[SuperSamus]

testcontroller was lighting up the wrong buttons. This was only visual, and is now fixed.

(The buttons that visually light up are the same, but the buttons that SDL considers pressed (on the left) are different)

A question, though: after a change like, do you think that a function like SDL_GetGamepadButtonLabel is necessary?

That is, a function that has the gamepad and a button as arguments, and returns what button would be pressed if label mapping was active. That would be extremely useful for button prompts. (Because the alternative to changing the mapping is changing the prompts.)

Right now, even SDL itself has to implement some stuff from scratch in test/gamepadutils.c in order to achieve something similar for testcontroller.

Anyway, tell me if this PR should be drafted.

[smcv]

A question, though: after a change like, do you think that a function like SDL_GetGamepadButtonLabel is necessary?

What would it return on Playstation controllers? Unicode encoded in UTF-8? (The best I can find is U+2715 MULTIPLICATION X, U+25CB WHITE CIRCLE, U+25A1 WHITE SQUARE, U+25B3 WHITE UP-POINTING TRIANGLE, which comes out as ✕○□△ - note that the Playstation cross button is SDL's A button, and it's specifically a geometric cross with equal, right-angled arms, not a letter X)

Prior art: Steam Input has a function to get button labels, but it returns bitmap images, not text, so that it can give the diamond buttons on Xbox and Playstation gamepads their conventional colour-coding.

[slouken]

Since this is SDL3, I'm thinking that maybe we can change SDL's gamepad buttons from labeled ABXY to positional NSEW, and add a new gamepad face button style (Xbox, Switch, Playstation) that can be queried to map the positional buttons into button labels.

A full enumeration of all available inputs is more flexible, but then introduces a new mapping of labels to inputs and needs to be extended for each new type of controller. Since both Windows.Gaming.Input and Apple's GCController API both provide that, maybe that is the way to go, with the controller type above being a shortcut for a set of labels.

Hmmm... @libsdl-org/a-team, @flibitijibibo, thoughts?

[smcv]

Since this is SDL3, I'm thinking that maybe we can change SDL's gamepad buttons from labeled ABXY to positional NSEW

I like that plan - it seems much nicer than changing the meaning of "A".

In the spirit of that change, could we also change the Xbox Elite/Steam Deck/etc. back paddles to something like LEFT_PADDLE1, LEFT_PADDLE2, RIGHT_PADDLE1, RIGHT_PADDLE2, where "1" is documented to mean the upper or primary paddle if there's more than one? The fact that they're labelled in a different order by the Xbox Elite controller and SDL (according to comments in the code, the right-hand paddles are P1/P2 in Xbox world but P1/P3 in SDL) seems undesirable.

[slouken]

Since this is SDL3, I'm thinking that maybe we can change SDL's gamepad buttons from labeled ABXY to positional NSEW

I like that plan - it seems much nicer than changing the meaning of "A".

We still need to maintain mappings in abxy style for compatibility across different versions of SDL. It's probably a good idea to maintain the enumeration order (SOUTH, EAST, WEST, NORTH) so people who are indexing into lookup tables still have the values they expect for Xbox and Playstation controllers.

In the spirit of that change, could we also change the Xbox Elite/Steam Deck/etc. back paddles to something like LEFT_PADDLE1, LEFT_PADDLE2, RIGHT_PADDLE1, RIGHT_PADDLE2, where "1" is documented to mean the upper or primary paddle if there's more than one? The fact that they're labelled in a different order by the Xbox Elite controller and SDL (according to comments in the code, the right-hand paddles are P1/P2 in Xbox world but P1/P3 in SDL) seems undesirable.

Sure, that seems like a good change. :)

[flibitijibibo]

Definitely in favor of NSEW - the eastern layout override can be useful for end users wanting a specific layout but making the diamond button location explicit makes it much easier for situations like #7558 to work without having to untangle various layers swapping and unswapping the swapped buttons that may or may not be swapped!

[slouken]

So I started prototyping this and ran into the expected problem.

When prompting users, what buttons are the activate and cancel buttons? Do I need to now look up the face button style every time I want to act on a button press? With the existing system I can just use SDL_GAMEPAD_BUTTON_A as activate and SDL_GAMEPAD_BUTTON_B as cancel and be done.

[flibitijibibo]

I suppose this is where action sets come in - right now we're treating gamecontroller as both a joystick abstraction and an action set system with the labels acting as actions, but the diamond button actions may be context-sensitive based on which set is supposed to be active (UI vs. in-game), hence the overlapping issues with swapping on the library and client side. Moving this swap to the action set library would make this easier to avoid, in exchange for gamecontroller being strictly a way to get joysticks that always look like Xbox-ish controllers.

[flibitijibibo]

Really dumb idea, but if we need basic actions within GameController, do we just want to make a new

typedef enum
{
    SDL_CONTROLLER_UI_INVALID = -1,
    SDL_CONTROLLER_UI_ACCEPT,
    SDL_CONTROLLER_UI_CANCEL,
    /* etc */
    SDL_CONTROLLER_UI_MAX,
} SDL_GameControllerUIAction;

extern DECLSPEC Uint8 SDLCALL SDL_GameControllerGetUIAction(
    SDL_GameController *gamecontroller,
    SDL_GameControllerUIAction action
);

Instead of using ABXY, we would have NSEW as well as Accept/Cancel, and Accept/Cancel would pick which direction to use based on what SDL knows about the device's labels.

[nadiaholmquist]

Making the API more agnostic to the button labels is definitely a very good idea - having it be A/B/X/Y in specifically the Xbox layout regardless of controller (or as it is now, with the buttons sometimes swapped with no way to tell where they should be positionally) was never a good idea to begin with. It's not just the Nintendo layout where this is a problem, using A/B/X/Y for PlayStation buttons is also a bit unclear since they use different symbols and which one should confirm is a user/regional preference.

Having it simply be north/south/east/west with some separate field to get the button's menu purpose, and a way to get an accurate label to show the user would be a very welcome change, and I think this would solve most controller mapping weirdness people would encounter in a game using SDL.

For the button labels, instead of trying to give the user of SDL a string with the button name, which gets a bit tricky with PlayStation's glyphs, and also stuff like the Xbox and PlayStation home buttons, wouldn't a better approach be to return a value from some enum of button names that SDL knows about, perhaps with a helper function to provide a best-effort string representation of the name of the button for an entry in that enum?

Such an enum could look something like this - it would obviously be a lot of work for a client application to have glyphs for all the possible options, but it would allow for the extra flexibility if the developer of some game wanted to provide all of them in a style that fits their UI without needing to do string comparisons, as they'd have to now. And they could always fall back on showing a name given by the helper function.

(Also if you do this, please add a hint for the PlayStation menu button preference so those of us who grew up with the Nintendo layout can set it to use circle as confirm in SDL apps)

[flibitijibibo]

Definitely agree on the PS Eastern layout, I've been doing this with my games as well.

For the label/string representation, I do have a draft of this for action sets (#4464), not sure how it would work for gamecontroller though.

[SuperSamus]

please add a hint for the PlayStation menu button preference so those of us who grew up with the Nintendo layout can set it to use circle as confirm in SDL apps

I think that such a hint would be useful for all types of gamepads? For instance:

• Forcing South to confirm everywhere because you are used to Xbox/Western PlayStation
• Forcing East to confirm everywhere because you are used to Nintendo/Eastern PlayStation

I'm wondering if that hint should work globally, per vendor, per gamepad ID, or per player number. (Or maybe a combination of these: the narrower it is, the higher is the priority?)
And maybe it shouldn't even be forced to South/East: for instance, on many PS1 and PS2 games, the cancel button was Triangle (North).
So, yeah, wondering what the API should look like.
(If it should even support all of these use cases in the first place, I'm probably overengineering this.)

---

Anyway, all of these proposals are out of scope for what I was originally doing. I'll leave this open for visibility, and close it when superseded.

[slouken]

Just to throw more monkey business in, there are Nintendo Switch Pro controllers that use a GameCube form factor. Which means not only do they not have a NSEW button configuration, but the A button is actually the button you want for confirm, instead of B, which you'd expect when using Xbox style input.

How can you tell whether a Nintendo Switch Pro controller uses this form factor? As far as I know, you can't. You need to identify it via VID/PID.

[SuperSamus]

Is there still any interest in what was discussed here?

[slouken]

Coming back to this with fresh eyes...

The application cares about these things:

• Which button is primary, secondary, etc.?
• When is the primary, secondary, etc. button pressed?
• What do I show the user to prompt them to press that button?
• And sometimes, what is the physical layout of the controller, and where do those buttons go? This is needed for software that allows visual button remapping.

Microsoft (WGI) just provides buttons in hardware order and provides an API that allows you to query the symbol (A, Square, etc.) on those buttons.

Apple provides the buttons in logical order and provides an API that allows you to query the symbol on those buttons.

The way the SDL controller mapping software works, we show a diagram of the controller and prompt the user with the physical label of the button. So conceptually, the controller mapping is a map from hardware button to label.

The easiest way for applications to operate is to work in terms of primary, secondary, tertiary, etc. buttons, and query SDL for the button label.

Some considerations:

• Users need UI to change their primary buttons (Nintendo A <-> B, Sony South <-> East)
• The controller mapping currently represents hardware button mapped to physical button label, since that's how most of the mapping programs work.
• Applications and users both have worked around the current system, setting mappings that implement the behavior they want rather than what makes sense conceptually.
• Steam works in terms of physical layout and has its own UI and mapping to translate that into Steam Input

So it comes down to, what should SDL gamepad buttons represent?
Traditionally, they represent the physical labels on the controller, and then it's up to the application to map that to primary and secondary actions, plus a hack in SDL for Nintendo Switch controllers to swap the A and B buttons.
Maybe the right thing to do is for SDL to provide A/B/X/Y as primary/secondary/tertiary/quaternary buttons, and then provide the application and the user a way to map from the physical buttons to the logical ones. Or maybe the SDL gamepad button events should provide both, so the application can decide which it wants to use?

This comes back to @flibitijibibo's proposal of adding logical actions, where the current SDL events represent the physical state of the controller and the actions represent the intent of the button presses.

So, we have controller mapping UI that defines the physical layout of the buttons (Xbox A is south, Nintendo, A is east) and then we would have action mapping UI that allows defining actions and binding them to the physical layout. We would also have an API that allows you to query the labels for the physical buttons, and a set of predefined action mappings.

Speaking from experience it's easy to go out of control with actions and states and so forth, so maybe for the purposes of this discussion we should focus on primary and secondary actions, which, as you noted is both muscle memory and regional preference.

So, coming back to the diamond buttons, we have the labels on the buttons, the physical positions of those buttons, the intent of the user when pressing those buttons, and the monkey wrench of GameCube controllers, which aren't even in a diamond pattern.

[slouken]

So, if we come back to SDL's button representation matching the labelled layout of the controller (for visual mapping purposes, i.e. what is this controller?) and we have a simple switch to map from the physical layout to the logical intent, we basically come back to where we are today, with the request for the button events to signal logical intent and for that to default to having the primary button be the button labeled B on Nintendo Switch controllers.

[slouken]

So we have several different ways of interpreting the buttons...

Labeled:
A/B/X/Y

Positional:
N/S/E/W (plus GameCube and other unusual button layouts)

Intent:
Primary/Secondary/Tertiary/Quaternary

Really what applications and users care about is intent, and the mapping from intent to button label.

[nadiaholmquist]

Applications might need to care about both.

For example, say I want jumping in my game to be the south button, regardless of controller, but I also want the menu buttons to act like they are intended to be used on the given controller.

The current design, as well as just renaming them to reflect intent instead, doesn't allow for this as far as I can tell?

There are also somewhat special use cases like emulators, where we need to be able to know the physical position of the buttons as to not mess up controls in the guest software when automatically mapping controllers.

[flibitijibibo]

If it were me, I'd choose the physical positions, since the original code made that assumption in the first place - it's easy for applications to explicitly enable switching at the GameController level, while on the ActionSet side of things we want as clear and direct a controller layer as possible so that we're not having to untangle layers of button swapping when exposing hardware to high-level action descriptions. I guess the argument could be made that ActionSet could disable any swapping explicitly, but I dunno if I want to fight applications and/or users swapping variables when the point of ActionSet is to make case-by-case swapping unnecessary. The best compromise might be to allow flexibility for the scenario when apps interface directly with GameController, but for ActionSet we have to ensure that we completely ignore anything and everything that might interfere with us reading the physical device directly (short of going back to SDL_Joystick 😨).

For now that's my take, but I may have something different in January when I drop everything to get ActionSet across the finish line.

[slouken]

For ActionSet we have to ensure that we completely ignore anything and everything that might interfere with us reading the physical device directly (short of going back to SDL_Joystick 😨).

For a pure action mapping, you probably want to go back to SDL_Joystick, and use the gamepad mapping as a set of defaults for actions. Although reading back through your comments above, it sounds like positional buttons would be even better for your use case.

[slouken]

After mulling all this over, I think I'm landing on the position that games are going to have to do controller-style specific handling anyway, so positional mappings are the cleanest way to be clear about what SDL is providing the application.

Thanks for the feedback everybody, I'm going to continue forward with that and update this with a PR when it's closer, for feedback.

[slouken]

There's a draft of the change to positional buttons in #8489.

There's a bunch of additional complexity for applications now, having to look up button labels and primary actions and so forth, so feedback is welcome!