John's User Interface index

Whenever we use computers, was use them via a user interface. Often, we are manipulating data inside the computer, following the manipulations we wish to do on the information that the data represents.

To do this effectively, we must have a mental model of how computer handles the data, and a mental image corresponding to the data in the computer, and have channels for sending commands to the computer to manipulate the data and for receiving confirmation from the computer of what the data now is. (A WYSIWYG word processing system typically tries to hide the data model, and pretend to be a piece of paper. This may explain why so many users of such systems use them so ineffectively.)

Each of these channels has a width -- the amount of information that it carries at each step -- and is in a language (with definite grammar and semantics). An example of a narrow-channel input interface is the-two button interface often used for setting timers etc. Speech recognition is an example of a broad-channel input interface.

An interface may have several layers, of different channel widths. For example, a menu system in which you use up and down and select keys to choose one of many options use using a narrow channel interface to access an underlying broad-channel one.

Areas of the interface

When considering the user interface, we can consider input and output separately, but they may also combine in interesting ways.

Input

There are several areas of this, and several aims to be addressed, including conciseness and comprehensibility.

Data input

This includes the entry of text for documents

Commands

The main thing to address here is that a limited range of commands is available. The aims are to let the user choose the right command, and enter it with the minimum of effort.

Navigation

This is a specific area of commands: those that move the focus of attention within the data stored by the computer. It covers such things as which directory is ``current'' as well as cursors and pointers.

Output

This is often seen as simpler (perhaps because there is nothing for the computer to parse) but typically handles more data (and usually, more information too.) Conciseness and comprehensibility matter here, as they do for input, but I'm not sure they're often addressed as thoroughly -- after all, less human work seems to be involved.

Interactions between input and output

Output may be used during input, to give feedback (for example, drawing menus as the user selects from them).

When the output channel is narrow, such as voice (which is slow), it may be useful to model what the user may be assumed to know, to suppress output that merely duplicates what they know. If needed, such output can be requested explicitly by the user. (This was a philosophy notably followed in early days of Unix, on teletypes, which were very slow; for example, the line editor "ed" does not output the line resulting from an edit unless requested -- and, ingeniously, on of the commands to get the line confirmed to the terminal has negative length -- it is leaving the final delimiter off the previous command!)

The state of the system may also be useful in interpreting the input; for example, a voice input system may be able to recognize the input more accurately when it has been told (by the application outputting the information to the speech input system) what inputs are valid at that point.

Styles of interface

Command-line

This is straightforward and consistent. Newcomers to computing often use the term ``arcane commands'', which indicates that they have found that they need to learn a language but are not prepared to do so, or may have difficulty doing so. A possible drawback is that it can be long-winded, but choosing short command and option names reduces this problem. The output for command-line interfaces is typically character-based.

Keystroke -- e.g. emacs

Similar to command-line but more concise; with ``keymaps'' as used by emacs, this is very flexible and powerful. In the case of emacs, it also includes command-line input for commands that are not available on keys. This may have graphical or character-based output.

Pedals and other few-button devices

A variant on keystroke interfaces is to use a narrower channel (typically because of a physical limitation in the user -- I developed my version of this because of RSI) with the information structure arranged accordingly. This can be used as the sole input, or as a supplement to keyboard input. Menus, as used in point-and-click interfaces, can be useful here, as the narrow-channel commands can be used to navigate around the menu structure and make selections. I have made an implementation of this in emacs-lisp.

Similar approaches may be taken with other few-button devices such as:

• handheld keyers
• fabric keyboards built into clothing

Voice

This is a relatively recent arrival, and its potential is still being explored. It is naturally a broad-channel interface, but this is limited in practice by the quality of recognition. Its use so far seems to be mainly for input of text, but off-the-shelf it can be used for command input too. A voice recognition mode is available for use with the keystroke-based system Emacs, and I have started to work on an improved and extended version this.

Voice output is also used, particularly for the blind, and is also used as confirmation for voice input.

GUI

This is a broad-channel interface. It is popular with naive users, as it easy to learn to do a few simple actions. However, it has to be supplemented by other interfaces, such as menus (which can be graphical or keystroke-based). Naive users seem to like menus (more than other command-selection interfaces such as command lines) because all the possibilities are listed (and so it is not necessary to remember a command word, but only to recognize it on a list) and it is not possible to enter an invalid menu entry.

Mouse-based interfaces are an ergonomic disaster; many people who develop RSI find that they have problems with using the mouse well before they have any problems with the keyboard. I hope they will fade to be used only as an ``accessibility'' interface for the learning disabled and the mentally handicapped.

Joystick-based interfaces are used mostly for games, and sometimes for CAD, but may also be useable for software development; the multiple axes of modern joysticks may allow considerable expressive power for program manipulation.

Haptic output

As well as displaying on screen, it is possible to build a three-dimensional representation of a program (perhaps without the full text being physically modelled) for the user to explore by touch using a haptic interface. Since the user typically has to move the device around, this may also provide input to the system.

Web

RSI

Emacs

ASR33