Operating Emacs with a gamepad

Sometimes Emacs is accused of causing RSI. This isn't a fair accusation; the problem is not with Emacs, but with taking a typewriter-style keyboard (designed in the 19th century, for typing letters) a long way beyond what it is designed for, and using it as a controller.

It makes much more sense to control Emacs with something designed as a controller; furthermore, a controller designed for comfortable, fast, intensive use. In fact, something designed after, rather than before, the idea of ergonomics was widespread. A gamepad meets these requirements pretty well, when used with a suitable suite of commands, such as Versor.

Interface software

To interface the gamepad to Emacs, I wrote a little glue program in C, based on the test/sample program jstest.c. Of course, it's no longer such a little program; it grew a lot when I added a simple X display for demonstration purposes, and again when I extended it to combine multiple gamepads into one logical controller.

The process runs as a child process of Emacs, emitting s-expressions on its stdout which Emacs reads and evaluates in a process filter. These put events onto the input queue, much as though they were function keys, thus allowing the commands they run to run in the normal top-level loop environment instead of in the process filter.

Buttons and chording: modifiers and commands

When you press a button (say the button Trigger), an expression of the form

(jse 'Trigger-down)

is sent to stdout, and if that button is then released without any other buttons having been pressed,

(jse 'Trigger-up)

is output.

If a button is pressed and held while another button is pressed, the "down" event for the first button is sent as before (as we don't have the technology to predict whether another button will be pressed before the first is released), but the second button, say button TopBtn, gets some modifiers included, either as a string of abbreviated button names, or as an octal number embedded in the functor name:

(jse 'BaBt2-TopBtn-down)
(jse 'BaBt2-TopBtn-up)

The abbreviations are made of any upper-case letters in the full name of the button, any lower-case letters preceded by an upper-case letter, and any digits. This seems to be a reasonably simple way of making unique and fairly readable abbreviations, giving the collection of button names provided. For example, BaseBtn2 gets abbreviated to BaBt2, as used in the example above.

Any number of modifiers may be used at once. (There are potentially nearly 80 of them, given a suitable monstrous stick!)

When the BaseBtn2 button (in our example) is eventually released, because it has been used as a modifier, instead of an "up" event, it generates a "release" event.

(jse BaseBtn2-release)

This way, each button can be used both in its own right (press and release) or as a modifier (press and hold while pressing other buttons).

Also, note that if you're using "-up" codes for chording, it makes a difference which button you take your finger off last. This makes for a potentially extremely subtle chording keyboard!

Joystick axes

When you move a "stick" control, a stream of joystick events is sent repeatedly, at a rate determined by the displacement of that stick axis from its centered position. Each joystick event is sent as one of these (using the X axis as an example):

(jse 'X-previous)
(jse 'X-center)
(jse 'X-next)

Modifier buttons are applied as for command buttons, and it is remembered that those buttons have been used as modifiers, and hence generate "release" instead of "up" when released.

Modified gamepads

I found that the gamepad was excellent for controlling Emacs, but it was a real nuisance to have to go back to the conventional keyboard to enter text; I wanted to be able to use the gamepad as a chording keyboard for text entry, too. I already knew the braille alphabet, which is conveniently typable as chords, so I started to think about how to use a gamepad as a braille keyboard.

However, the arrangement of buttons wasn't suitable for that, and I started to think about duplicating some of the buttons on the underside. That would need to change state between typing and commanding (like vi, so that would never do). So, instead, I decided to get a second gamepad, take them apart, and build a double gamepad.

I made various mistakes while making it, and didn't manage to transfer all the part of the second joystick (OK, it was carnage). However, I got a working braille and control device out of it. Some time I might try again, learning from my mistakes.

Design

The double gamepad preserves the general shape of a normal gamepad, with the addition of a box at the front for the electronics of the second gamepad; I put the box vertically, placed such that it acts as a third leg when the gamepad is placed on a flat surface. I moved the face buttons group of the second gamepad to be underneath the centre of the body of the first gamepad, and the shoulder buttons to the inner faces of the handles of the first one, to where your little fingers naturally go when holding the device. Thus, moving your fingers down from the shoulder buttons of the first gamepad puts you in a position to type chords on eight buttons, arranged in two columns of four (like on a braille chording keyboard).

Successful points

The overall concept seems sound.

Mistakes

I desoldered connections from the gaempad board using desoldering wick, and had to drill some of the holes clear with miniature drill, which destroyed a couple of connections. Using a desoldering pump would be much better.
I may have overheated or scratched something, as one of the analog sticks of the second joystick got confused. With better desoldering tools and techniques, I would have liked to put them on the front electronics box, sticking out like bug eyes.

Unsure points

The transplanted buttons work OK, although separately-bought buttons might have been better; using real gamepad buttons was fiddly but at least the feel and appearance are consistent.

Other observations

The more of the surface of a device you cover with buttons, the harder it is to pick it up without accidentally sending commands.

John C. G. Sturdy

Last modified: Tue May 27 00:17:21 IST 2008