HCI Research Enabling Blind People to play Games on a Haptic Game Console

Designing Interaction Techniques To Enable Blind People To Play Doom On A Haptic Game Console

‍

In the course of my master thesis and a project seminar at HPI I designed interaction techniques that enable blind people to play Doom on the DualPanto. The DualPanto is a haptic device that enables blind users to track moving objects while acting in a virtual world.

How Does The DualPanto Work?

‍

As shown in above figure, the device features two handles, so called "pantographs". Users interact with DualPanto by actively moving the me handle with one hand and passively holding on to the it handle with the other. DualPanto applications generally use the me handle to represent the user’s avatar in the virtual world and theit handle to represent some other moving entity, such as the opponent in a soccer game.

How Did I Contribute?

For my master thesis my goal was to enable blind people to play for the first time a real-time interactive game with a high level of spatial sensemaking on a haptic device. Although there exist games for the blind, those are usually audio-based or not real-time interactive. We chose the 1993 first-person shooter "Doom", a fast-paced game designed for sighted people, that is a common benchmark for interactive devices. To enable blind people to play this complex game, I was designing a series of (haptic) interaction techniques for different non-trivial challenges that fell into one of the 4 categories:

1. In order to facilitate orientation and maximize spatial awareness, we let the objects in the field of view of the player announce themselves. We use a model of mutual awareness to determine when, how, and which objects should be announced. Second, when users first enter a room, we pause the game and play an auditory-haptic tour surveying the room to help the player building up a detailed mental model of the space.
2. We present 3 haptic techniques to facilitate navigation: to guide towards objects in the middle of the room, we use haptic rails which behave like walls on collision but can be passed if the player presses firmly against them. To prevent the player from going into dangerous areas we use haptic force fields, and to assist the player in way-finding, we dynamically mark the most important object around the player using the it handle.
3. During fights we use the dynamic importance model to multiplex between enemies and mark the most dangerous one. Additionally, we use spatial audio to convey their positions, enlarge the shooting angle, and render haptic cues on the end effector when the player is aiming at an enemy.
4. To enable that the player can move at different speeds, we apply a force against the movement direction of the player. To ensure that collisions remain noticeable under speed control, we separate the avatar from the handle, thus allowing the handle to move at a faster speed while the avatar follows the handle.

Eventually we brought all of these interaction techniques together and tested our version of "PantoDoom" with 3 blindfolded participants in a user study. All participants succeeded at reaching the exit, defeated at least 3 out of 4 enemies, and were able to find and collect an elusive item.

‍

‍