Here’s a video of me driving a truck with an iPad Pro as the steering wheel in American Truck Simulator.
This is a quick hack project I made when I was playing American Truck Simulator. Since I didn’t have a suitable gaming controller, I was getting annoyed by how hard it was to control the steering wheel with precision.
I noticed that with Core Motion, the iPhone/iPad’s sensors provides exceptionally fine positional data. Maybe I could use an iPhone as a steering wheel & custom controller?
Breaking the problem down, I needed to read the iOS sensor data, transmit it to my laptop, and turn that signal into controller input. Best if this could be achieved with low latency.
The quick and dirty solution I came up with involves a couple of simple components:
- Core Motion
- Core MIDI via Bluetooth
In other words, I wrote an app that reads the device’s rotation, and presents itself as a bluetooth MIDI instrument. The signal is encoded as if they are music notes, sent over to the Mac, where ControllerMate turns the MIDI music notes into the input of a virtual controller.
CMMotionManager provides simple access to device’s physical motion sensor fusion data, and here I’m just using the yaw rotation.
The only complication here is that I need to keep track of full rotations and offset manually. The yaw’s range is from -π to π (half circle backward to half circle forward).
MIDI via Bluetooth
Turns out this is extremely easy. Just present a CABTMIDILocalPeripheralViewController on iOS, and the built-in Audio MIDI Setup on the Mac, and connect.
Bonus feature of communicating via Bluetooth MIDI is its low latency.
Sending Motion Data as MIDI Message
Core MIDI is a pain to use because of its C API, and usig Swift only make it harder.
Fortunately with the PGMidi library it’s again fairly painless. Though, some understanding of MIDI is required here.
Converting MIDI to Virtual Joystick
At this point you’re getting MIDI message on the Mac sent from iOS. The last super secret sauce is ControllerMate.
With El Capitan, it’s not a simple task to create a virtual HID. ControllerMate does it for you and more. ControllerMate listens to all the MIDI messages, and you can visually program it to map the MIDI messages to virtual joystick output axes.
At first I split the joystick value (0~4095) as two notes (C3 & C4) and re-combine them, as shown in the top half, since a note’s velocity is only 7-bit. Later it was changed to use the 14-bit modulation wheel instead. By the way, I’ve been using ControllerMate since 2007 and it’s one of my favorite application of all time.