From Wearables To Thinkables: Biometrics Tap Brain To Play Games

A version of this story airs on NPR’s Here & Now. If you're a gamer, or have one in your household, odds are that there is a wedge of black plastic studded with joysticks and buttons nestled in the cushions of your couch. Buttons get pressed, and things on the TV go boom. That would be the humble video game controller. Humble, perhaps, for not much longer. “So what we're doing here is we've modified an Xbox 360 controller to help try and sense a player's emotions as they're playing video games,” said Corey McCall, a doctoral candidate in electrical engineering at Stanford University. He's created a way for game controllers to read your mind. Kind of. “What we want to do is estimate the player's autonomic nervous system activity,” said McCall. “Which is the part of the brain that responds when the player gets excited or they get stressed or they get focused. Excitement, stress, focus. The trifecta of gaming engagement. “So instead of measuring this directly, we try to measure it indirectly through things like their heart rate or their skin temperature. Say how much they're breathing or how quickly they're breathing.” To measure respiration, McCall placed a metal plate on the back of the controller and runs a small electrical current up one hand, measuring how long it takes to return. Air in the lungs slows the current down--that's impedance, for the technically inclined. From that, you can know if a player is breathing hard or holding their breath to focus. McCall's research team made the controller because they were interested in how a person's body reacted to a game. But another question is what can a game do once it is aware of a player's unconscious reactions. That's where Nevermind comes in. “The more scared you get the harder the game becomes,” said Erin Reynolds, the founder and lead designer at Flying Mollusk, creators of the psychological horror game NeverMind. “So you're going to subject yourself to these creepy scenarios and as you get more scared, more stressed the environments will try and get in your way. So you've got to learn how to calm down in the face of those.” NeverMind uses biofeedback to adjust the difficulty of the game. While an earlier version relied on a Garmin chest strap-- the same kind of device runners use to measure their heart rate--the commercial version of NeverMind will use a new camera from Intel that provides biometric data to the computer. “Basically there's a few different ways that camera tech can read biofeedback,” said Reynolds. “One is through very subtle fluctuations in skin tone, which sounds totally sci-fi and almost creepy to some. But it’s really cool because every time your heart beats, your skin slightly changes color. It gets a little bit more red as it gets more blood in it.” That’s the tech side. From a storytelling standpoint, biometrics opens up new possibilities in the way stories can be told. Let’s stay with the horror genre, which uses lulls in the action to build tension. “Imagine if the designer could tailor that downtime just perfectly,” said Garnett Lee is a veteran video game broadcaster, “and have you right there on the cusp of whatever moment they want you to have. That would be awesome.” He’s into the idea, with reservations. “Are gamers ready to have that sort of very exposing, personal experience with a game?” Before biometrically enhanced games become as popular as Candy Crush or Call of Duty. Lee says the product will have to user friendly, and not require a lot of gear. “I look to the inflection point of having biometric feedback in games and--really beyond gaming--as really the same place that we have with 3D televisions and that's ‘I don't want to wear something special. I don't want it to be an extra encumbrance.’ I want something that's easy, that I sit down, that it works, that it's a great comfortable experience and enhances what I'm doing.” That ease of use is the gold standard that hardware makers are chasing. If they can nail that, a whole new universe of gaming interfaces may be just a heartbeat away.