Introduction

The concept of a keyboard emulating interface was first proposed by Dr. Gregg C. Vanderheiden (1980, 1981) in two IEEE articles in response to the need for a common communication protocol when using AAC systems as computer input devices. In the mid- 1980s, the Trace Center developed a series of keyboard emulating interfaces that followed a common protocol known as the Keyboard Emulating Interface (KEI) standard. Several variations of either hardware or software KEIs were developed and commercially available for IBM PC, XT, and AT computers.

KEIs, however, could not be developed for the Macintosh or PS/2 computers since keyboard access by itself would not provide full access to these computers. The Apple Macintosh and IBM PS/2 computers included non-keyboard input capability in the form of a pointing device, commonly referred to as a mouse. If AAC systems were to be used as input devices for Macintosh or PS/2 computers, then the KEI standard would have to be expanded to include a wider range of input devices.

In 1987, the GIDEI was proposed as the new interface protocol to handle the wider range of input devices, which were becoming a standard part of personal computers. While the GIDEI standard defines the connection and data communication protocol between the AAC system and the computer, SerialKeys was the name given to that feature which translates the GIDEI input into the appropriate keyboard and pointing device language on the computer to which the AAC system is attached. In other words, SerialKeys traps GIDEI messages arriving at the computer's serial port, translates the messages into the appropriate keyboard or mouse language, and then passes that information to the applications running on the computer. In many cases, SerialKeys is just special software running in the background on a computer the AAC system user wishes to access. However, SerialKeys could also be part of an external hardware device that connects to the computer and provide its own serial port for the attachment of the AAC system.

The GIDEI Standard

As part of the data communication protocol for sending keystroke and mouse information, the GIDEI standard has two modes of operation: Character Mode and Escape Mode. In Character Mode, SerialKeys, or other suitable emulating interface, translates ASCII characters received from the alternate input device directly into the keystrokes necessary to produce those same characters on the computer. Escape Mode is designed to provide a pathway for a person using an alternate input device to type keys that have no ASCII equivalent (e.g., home, page up, etc.), as well as to allow the user to hold down two or more keys at the same time, as is required by many computers (e.g., Ctrl+Alt+Del). Escape Mode also allows the user to specify mouse actions such as clicking, double clicking or dragging, and is used to change certain operational features such as the baud rate.

GIDEI Standard Utilization

Figure 1

The example in Figure 1 shows an AAC system connected directly to the computer using a serial cable. The serial cable from the AAC system connects directly to the computer serial or "com" port, which is usually located at the rear of the computer. The user in Figure 1 wishes to make selections on the AAC system and to transmit the information across the serial cable to the computer, and he or she expects the computer to ascertain that these selections represent keyboard or mouse actions.

In Figure 1, the connection protocol, as defined in the GIDEI standard, includes the serial cable, the internal wiring or pin-out arrangement, and the amount, speed, and characteristics of the information the user sends from the AAC system to the computer. As defined in the GIDEI standard, the data communication protocol deals with the contents of the information (e.g. data packets), which may be created or prestored by the user on the AAC system. The user might expect that once he or she has set up an AAC system to conform to both the connection and data communication protocols, the AAC system should be able to communicate with and control the computer. Unfortunately, the computer will not be able to understand what the AAC system is transmitting without one additional piece.

From a user input standpoint, however, most computers only understand the user when given input or instructions from the keyboard or the mouse. By design, keyboards and mice each use their own language to communicate with the computer. Therefore, once the AAC system is connected to the computer as shown in Figure 1 (e.g., either directly through the use of a serial cable or indirectly via some kind of wireless link), something must act as a go-between to translate the AAC system language into the keyboard or mouse language understood by the computer. Since the information transmitted from the AAC system is well defined by the GIDEI data communication protocol, a good portion of the translation work is already completed. In Figure 1, the missing piece is the software that functions as the emulating interface on the computer receiving the AAC system information. Emulating interface software can translate the information sent from the AAC system into keyboard and mouse language understood by the computer. The SerialKeys feature, provided as part of the AccessDOS software package, is an example of emulating interface software.

Figure 2

The AAC system-to-computer connections shown in Figure 2 are very similar to the example shown in Figure 1 except that the serial cable coming from the AAC system is now connected to a black box outside the computer, rather than directly to the computer. (Again, please note that the serial cable could also be a wireless serial link.) The black box has input cables coming from the keyboard and mouse and output cables exiting the black box which interface to the computer at the same location in which keyboard and mouse cables would have connected had the black box not been used. The black box also has a connection for a serial cable used to access the AAC system.

In this example, information is sent from the AAC system to the black box. Therefore, the black box needs to perform the necessary translation of the AAC system information prior to sending it to the computer via the keyboard and mouse cables. The black box becomes the missing go-between in this example. Since the black box is connected between the standard input devices and the computer, it understands the language used by the keyboard and mouse when talking with the computer and is therefore capable of allowing the AAC system to emulate the standard input devices. In this case, we would refer to the black box as an emulating interface device, since it is actually a piece of hardware which a user could take from computer to computer, as his or her needs may dictate. The Trace Transparent Access Module (T-TAM) is one example of an emulating interface device.

Figure 3

Figure 3 shows one more example of an AAC system-to-computer connection; however, the example in Figure 3 shows how a non-GIDEI alternate input device might connect to a computer. The AAC device in Figure 3 is connected directly to the computer using a keyboard cable. Since no serial cable nor wireless serial link is involved, this example does not use the GIDEI standard. The AAC system in Figure 3 does not use any kind of GIDEI connection, because this AAC system already has a built-in keyboard interface which understands the keyboard language of the computer. As technology advances, more AAC systems are expected to be designed in this manner, e.g., capable of connecting directly to the computer as substitute or additional keyboards, mice, etc.

GIDEI Operations

Character Mode is used for typing characters (e.g., to type a character means to emulate the press and release of a key, as if the user had actually typed on the keyboard). To type characters, simply send that character (sometimes referred to as an ASCII character) from the AAC system to the emulating interface. Some examples of ASCII characters are the lowercase letters a through z, uppercase letters A through Z, most common punctuation characters such as comma (,), period (.), etc., and the numbers 0 through 9. Typing characters does not require any special action on the part of the AAC system user. For example, to type an "a" key on the computer, the AAC system will send the ASCII character "a" to SerialKeys or other suitable emulating interface.

Escape Mode is used for activating specific keys or those keys which do not have an ASCII character name (e.g. page up, home, etc.), for keyboard and mouse commands, and for other miscellaneous GIDEI commands. The user can enter Escape Mode by sending an ESCAPE character from the AAC system to SerialKeys or other suitable emulating interface. Escape Mode ends when a PERIOD is sent from the AAC system to SerialKeys, or other suitable emulating interface. The entire string of characters from the ESCAPE character to the PERIOD character is known as an escape sequence. Therefore, an escape sequence is simply a special string of characters, starting with the ESCAPE character (ASCII 27, represented as <esc> in the text below), followed by a sequence or string of other ASCII characters, and ending with the PERIOD character (ASCII 46, [.]). Some AAC systems generate the ESCAPE character by choosing a control+ [ selection, while other AAC systems have the ESCAPE key labeled on a built-in overlay.

Since each key on a computer keyboard is designated by a GIDEI key name, you can activate any key on the computer by using that key name in an escape sequence. For example, to activate the page up key, you need to use an escape sequence with the key name for the page up key as follows: <esc> pageup followed by a PERIOD ".". The GIDEI key name for the page up key is "pageup".

Remember, the ESCAPE character is used to enter Escape Mode or begin the escape sequence, and the PERIOD is used to exit Escape Mode or end the escape sequence. If the user forgets to use the ESCAPE character, SerialKeys will assume he or she is in Character Mode and just type the key name letters into the computer. If the user forgets the PERIOD, SerialKeys will not be able to determine the proper end of the escape sequence, and the result may be that no key is activated and an error beep may come from the computer. Many AAC systems allow the user to preprogram entire character strings such as "<esc> pageup ." into a single selection, such that the entire string is sent from the AAC system to the computer when that particular selection is chosen.

Other keyboard actions may involve the typing of multiple keys simultaneously, or typing keys which are used to modify other keys when used together, such as the shift key. To perform these types of keyboard actions, within escape sequences, requires the use of a keyboard command. In general, the formula for escape sequences with keyboard commands is "<esc>, keyboard command, key name .". These keyboard command strings are typically preprogrammed into a single selection on the AAC system, so the user need only make one choice to perform the desired action, much the same as if the user had typed that same key on the keyboard. (Note the addition of a keyboard, mouse or GIDEI command to an escape sequence adds commas to the escape sequence to separate the ESCAPE character and the key name or action from the command. Therefore, whenever GIDEI, keyboard, or mouse commands are used in an escape sequence, they must be separated with commas.)

To Perform GIDEI Keyboard or Mouse Actions

combine: allows the typing of one to five keys simultaneously.

<esc> , combine , ctrl , alt , del .

hold: holds the specified key(s) down until the next key is typed.

<esc> , hold , shift .

lock: locks keys down until the rel command is used to release them (e.g., a key down is transmitted with no key up).

<esc> , lock , ctrl .

rel: releases all or only specified locked keys (e.g., sends key up) and clears any pending hold keys.

<esc> , rel .

Escape sequences are also used to send mouse commands from the AAC system to SerialKeys, or other suitable emulating interface. Mouse button actions are performed using escape sequences of the general form: <esc>, command, button identifier. Specific mouse button commands and corresponding escape sequences follow:

click: sends a button down immediately followed by a button up of the default or specified mouse button identifier.

<esc> , click .

dblclick: sends two click actions.

<esc> , dblclick .

moulock: locks down the specified mouse button identifier(s) (e.g., a button down is transmitted with no button up).

<esc> , moulock .

mourel: releases locked mouse button identifier(s) (e.g., sends button up).

<esc> , mourel .

Mouse movement actions are performed using the escape sequence formats as follows:

anchor (Macintosh SerialKeys only): remembers a specific location of the mouse cursor for subsequent return actions.

<esc> , anchor .

(A single lowercase letter a through z, following an anchor command becomes the name of the current mouse cursor screen position, thus allowing for 26 separate mouse cursor locations to be anchored or saved.)

move: moves the mouse cursor a specified distance in the specified direction(s).

<esc> , move , + X direction , + Y direction .

goto: A. moves the mouse cursor to a specified absolute screen location when both X and Y positions are indicated.

<esc> , goto , X position , Y position .

goto: B (Macintosh SerialKeys only). moves the mouse cursor to a specified screen location when only a single X value is entered, based on that location having been preassigned using the "anchor" command (see above).

<esc> , goto .

(A single lowercase letter a through z, following this goto command is the name of the screen position of the mouse cursor anchor to which the user wishes to return.)

moureset: initializes the mouse settings and mouse location or coordinates to (0,0) and moves the pointer to the upper left hand corner of the screen.

<esc> , moureset .

When the AAC user is communicating with the emulating interface at a known rate, the baud rate command can be used to change to another rate, should the user wish to do so.

baudrate: sets the baud rate of the GIDEI.

<esc> , baudrate , XXXXX .

(Where XXXXX refers to a baud rate of 300, 600, 1200, 2400, 4800, 9600, and 19200.)

GIDEI Connection

GIDEI Standard Status

The GIDEI standard was recently updated to provide improved keyboard and mouse support for non-United States AAC system users. A complete copy of the revised standard, known as GIDEI2, can be obtained by contacting the Trace Reprint Service, in care of the Trace Center, University of Wisconsin Madison, 2107 Engineering Centers Bldg., 1550 Engineering Dr. Madison, WI 53706, telephone (608) 262-6966.

References

Vanderheiden, G. (1981) "Microcomputer Aids for Individuals with Severe or Multiple Handicaps: Barriers and Approaches," in Proceedings of the IEEE Computer Society of the Application of Personal Computing to Aid the Handicapped, Johns Hopkins University.

* For an index of Trace Center information on SerialKeys go to Information on Macintosh SerialKeys.

This document is hosted on the Trace R&D Center Web site. Please visit our home page for the latest information about Designing a More Usable World - for All.