Virginia Tech researchers Doug Bowman and Brendan David-John are a part of a multi-institution Defense Advanced Research Projects Agency (DARPA) program to improve the security of the U.S. military against “cognitive attacks” in mixed-reality environments.
“Cognitive attacks mean that an adversary might do something in the real world to make the system misbehave or to confuse the operator,” said Bowman, director of the Center for Human-Computer Interaction, part of Virginia Tech’s Institute for Creativity, Arts, and Technology. “The experiments will tell us how these cognitive attacks can affect mixed reality users. Once we know that, we can design the system to mitigate those effects.”
Their work is a collaboration on Intrinsic Cognitive Security (ICS), a DARPA’s three-year, multi-million-dollar program that more than four teams across the country. The team is led by SRI International, a California-based independent nonprofit research organization, in collaboration with Virginia Tech, New York University, and the University of California Santa Barbara. The team’s total funding is $8.1 million with more than $1 million dedicated to Bowman and David-John’s work developing cognitive models and mathematical guarantees to protect military personnel.
The military and mixed-reality
Mixed reality integrates real-world and virtual elements, allowing users to interact with objects in both environments in real-time. According to the researchers, branches of the military have expanded the use of the technology in recent years as a way of enhancing situational, tactical awareness, as well as target acquisition. The technology can be used by soldiers to see around corners, look through walls, or communicate with other teams in real-time.
“Military personnel in the U.S. armed forces will increasingly rely on immersive technologies,” said Bowman, who is also a professor of computer science. “While handheld devices like computers or tablets can be used, they have the drawback of reduced situational awareness. Seamlessly overlaying information onto the real world, however, can significantly improve an operator’s ability to make decisions.”
Bowman researches augmented and virtual reality as well as collaboration across time, space, and reality. David-John’s research specializes in gaze-based interactions in virtual reality as well as emerging privacy and security concerns in extended reality.
“Extended reality is a broad term that includes virtual, augmented, and mixed reality technologies,” said David-John, assistant professor of computer science. “Virginia Tech has a strong reputation for its work in extended reality, which is why SRI International initially approached us.”
