Add native trackpad input support#109
Conversation
Adds a new channel for trackpad inputs. This is similar to a touch device (i.e. has positional "fingers"), with an additional button press for pressing the trackpad.
|
How does this perform with different trackpad dimensions or aspect ratios? One of the downsides of our normalized x/y coordinates is that it basically assumes a direct mapping between the input device and the screen, which is fine for a touchscreen but not correct for a trackpad. We have the same issue with our gamepad touchpad support, but we get away with it because those devices aren't particularly precise anyway (and the emulated gamepad usually matches the client's gamepad, so the apps interpret the coordinates properly anyway). |
|
I unfortunately don't have the hardware to test this with different trackpads (I only have one device with a trackpad, with a relatively "standard" aspect ratio). However, I do agree that this is an issue. What we really want instead of resolution/aspect ratio is the physical dimensions of the trackpad (this is used by both libinput on Linux and Precision Trackpad on Windows). Once we have that, then using normalized coordinates in packets is OK, and also avoids any (rare I imagine) issues where the trackpad has different x- and y-axis pixel densities. In order to do this, we would have to add some metadata to set the height/width. Would a new packet type (i.e. Sidenote: I don't know how easy it would be for clients to access the dimensions to advertise them (especially on non-linux/non-windows clients) - right now, I just set a udev rule on my machine to overwrite the dimensions of the trackpad I use over sunshine (which is obviously not very user-friendly). At the very least, we could have some sensible defaults and allow users to overwrite them as options in the clients for platforms which won't let you access the physical dimensions (e.g. Android unless I missed something). |
|
One option is to include the dimensions of the trackpad in each event. This is what GameStream does for absolute mouse input. |
Current implementations (e.g. moonlight-android explicitly, and moonlight-qt implicitly) convert trackpad events into mouse actions, which inherently loses some information (e.g. gestures such as pinch-to-zoom and multi-finger swipes do not go through, scrolls are inprecise)
This PR adds a new channel for trackpad inputs. This is similar to a touch device (i.e. has positional "fingers"), with an additional button press for pressing the trackpad.
Similarly to touchscreen events, this is behind a feature-flag (which is currently implemented in my fork of sunshine, and my fork of moonlight-android, both of which I will be making a PR for).