UIST 2009 Student Contest Entry

A silly poster.

Accessible Text Input Using Pressure-Based Selection and Word Frequency Heuristics

The Problem

Traditional “accessible” means of text input are slow, clunky, and inaccurate.  For those people with temporary or permanent loss of fine motor control, this can make interacting with a computer an arduous chore, whether it means fighting with flaky voice recognition software or struggling with ordinary keyboard input.

This device is chiefly intended for use by those with poor motor control.  This group includes people with amputated limbs as well as those who suffer from carpal tunnel, amyotrophic lateral sclerosis (ALS), and Parkinson’s disease.  The device could be very useful for individuals who can’t type with their hands, but can type using their fists, elbows, or feet.

The casual user may also benefit. Regular users are spending an increasing amount of time with personal computers.  Foot typing would enable these users to eat meals, practice musical instruments, and perform other chores while checking email, surfing the web or chatting with friends.

Our Solution

By using a pressure-sensitive keyboard to capture the most likely key events, it is possible to dramatically increase the chances of registering the intended keypress when multiple keys are pressed down on a keyboard at once.  By supplementing this with a predictive algorithm to determine which words were most likely intended, it is possible to compensate for a lack of accuracy and precision and allow those with restricted movement to use a computer more efficiently.

The Software

There are several different components involved in determining what the user means to type: keypress determination, a prediction engine that combines the results from a word frequency table and a trie, and a basic editor that presents the user with a choice of words that she may have intended to enter.

Keypress Determination

In order to judge which key the user intends to press, we construct a table of absolute key positions in a two-dimensional array.  We then compute the “center of pressure” of each stroke, which generally corresponds to the area of the keyboard that the user was targeting.  Any imprecision is mitigated by the word prediction applied as the user types more letters.

Prediction

Predictions are based on three considerations:

1) The frequency with which words appear in the training files.

2) How well a word fits the pressure profile of the entered keystrokes.

3) How well a word fits with the words preceding it (again, based on the training files).

By combining these factors, we avoid favoring the most common words too heavily while simultaneously avoiding the closest apparent matches that are very infrequently used.

Implementation

Word frequencies

Word frequencies are given to the program using the text files of several well-known public domain books. A dictionary file is also included in order to ensure that words not found within the used books will also appear in the frequency list.

Trie

Since delay between keyboard input and appearance of text on screen is unacceptable for general-case usage, we stored the words in a trie (see left). As the user enters characters, the predictor performs a breadth-first search through the trie. As more characters are entered, a list of likely possible words is presented to the user.

N-grams

The word predictor also looks back at previous words and by using learned information, it adds weight to the words that generally follow.

Acknowledgments

Faculty Adviser: Professor Christine Alvarado

Team Members: Gregory Fong, Ozzie Gooen, Ben Jones, Daniel Lubarov

Special thanks to: President Maria Klawe, Yvonne Coady

---

If you want to see all that text as seen at the conference, check out the original poster.