What Apple's new Vision Pro headset might do to our brain

By Lauren Leffer for the Scientific American

Apple released its long-awaited Vision Pro mixed-reality headset this month. The company describes the device, which has a starting price of $3,500, as a “spatial computer,” an alternative to a standard laptop or desktop. Apple’s ads have shown people using the headset to send e-mails and conduct other mundane two-dimensional tasks, and a June 2023 press release from the company said, “Apple Vision Pro is designed for all-day use.” Enthusiastic early adopters have already recorded themselves using it for dozens of hours on end—and even wearing it while sleeping.

Yet many experts are skeptical that this kind of headset can—or should—replace our physical monitors, keyboards and mice. Some worry that using such a device for long periods could lead to motion sickness, new types of social isolation or other unintended consequences.

Virtual reality (VR) and augmented reality (AR) are incredible tools for creating unique and immersive experiences, says Jeremy Bailenson, founding director of Stanford University’s Virtual Human Interaction Lab, where he researches the psychology of VR and AR. But that doesn’t mean such a headset is always a useful tool. “We don’t use it for everyday stuff. You don’t need to put on a headset to read your e-mail,” he says, or to “enter numbers into a spreadsheet.”

Apple’s 1.4-pound goggles use sensors, including a lidar scanner and a camera array, to place people into what’s been called “mixed reality.” Outward-facing cameras offer a real-time view of users’ surroundings, while two small screens—situated directly in front of their eyeballs—simultaneously display an interactive digital realm. Meta’s Quest 3 headset, released in October 2023, also uses this style of “pass-through” video technology. Mixed reality is neither traditional VR, which completely blocks out the real world, nor AR, which presents a digital overlay on transparent lenses. Instead a pass-through device translates a digital representation of a person’s environment (their hands and nearby objects, for instance) into a completely virtual space.

This means the device mediates everything about a user’s experience. It’s “the dream of tech companies because you never [have to] take it off,” says sociocultural anthropologist Lisa Messeri of Yale University, who is author of an upcoming book on virtual reality, In the Land of the Unreal. “They can always have your attention. They always know where you’re looking. They always know what you’re doing.”

Pass-through’s pitfalls

To Bailenson, Vision Pro and Quest 3 are exciting technological advances that improve on VR’s graphics and AR’s limited field of view. Pass-through tech, however, also presents new risks. In a study published earlier this month, Bailenson and others assessed some of mixed reality’s short-term effects. They found that wearing a headset seriously restricts a user’s visual perception—and could also change social behavior and motor function. Plus, pass-through technology has quirks that often cause visual delays and other distortions. Color saturation varies frequently. Light dims. Some objects appear too close or blurry. Even though the Vision Pro has impressive specifications, its image resolution is still lower than what human eyes are used to perceiving.

The research team currently advises against spending hours per day with these goggles on. “We recommend caution and restraint for companies lobbying for daily use of these headsets,” the authors wrote, urging more rigorous study of the effects. Few long-term studies of VR or AR use exist. Bailenson is currently monitoring participants who regularly use mixed-reality headsets, but the results of that study are months away from being published.

“We don’t know what it means to walk around the world with reduced peripheral vision or visual distortions for hundreds of hours in a month,” says Rabindra Ratan, an associate professor of media and information at Michigan State University and a co-author of the recently published study. “This is purely speculative, but there could be effects on the way your eyes move around in space, and maybe that could make your vision worse,” Ratan suggests. “We don’t really know what that will do to our brain.”

Past research involving prismatic spectacles (glasses with stick-on overlays that cause distortions and displacement) suggests that people can adapt to significant visual disturbances, Ratan says. But easing into altered vision initially requires an adjustment period that can last for hours or even days, depending on the individual user and the disturbance’s strength. Reverting to normal vision—such as by taking off a headset—is usually quicker, on the order of minutes. Yet in each case, the gap between mind and body can make basic motor tasks such as pushing elevator buttons, high-fiving and navigating a crowd on foot much more challenging. They can also lead to serious safety concerns. When the researchers rode a bicycle while wearing a headset, they found it much more difficult. In a worst-case scenario, if someone were biking with a headset on and the battery died, their vision would suddenly be wholly obscured, Bailenson points out. And navigating with an Internet-connected device attached to your face can be downright distracting.

Already people are publicly operating moving vehicles, including cars, while wearing mixed-reality headsets. Just four days after Apple released the Vision Pro, the National Highway Traffic Safety Administration released a statement imploring people to not drive while wearing a VR device, in response to online clips of headset-wearing drivers. U.S. Transportation Secretary Pete Buttigieg also posted on social media, noting that all available consumer cars—even ones with advanced driver-assistance systems—still require completely engaged human drivers.

Simulation’s harsh reality

AR, VR and mixed-reality headsets also frequently cause “simulator sickness,” a suite of uncomfortable symptoms that include nausea, headache, dizziness and eye fatigue. Bailenson, Ratan and their co-authors encountered simulator sickness in the majority of their device sessions, even though the tests generally lasted less than an hour. Enduring even low levels of simulator sickness could impact people’s quality of life, activity level and productivity—which is one reason Bailenson worries that people might try to rely on these devices for their day-to-day work.

Then there are the potential impacts on memory. In one 2014 experiment, Frank Steinicke, a professor of human-computer interaction at the University of Hamburg in Germany, spent 24 hours alternating between two-hour bouts of VR use and 10-minute breaks. Throughout the study, Steinicke became unsure of was real and what wasn’t. “Several times during the experiment the participant was confused about being in the [virtual environment] or in the real world and mixed certain artifacts and events between both worlds,” the research paper says. Similarly, one 2009 study found that VR use can cause children to acquire false memories—even though image resolution and quality in VR was far worse 15 years ago.

“The audio-visual display is getting better and better; therefore, I am pretty sure that virtual and real content will continue to merge,” Steinicke says. Despite his findings in the 2014 experiment, he envisions a more positive computing future in which these tools beneficially replace keyboards and touchpads.

Immersive digital worlds may also affect how well users think and socialize, potentially influencing how they work or learn. In the presence of a virtual human character, people wearing augmented reality headsets perform better on simple cognitive tasks—but worse on more challenging ones—according to one 2019 study. In a separate experiment in that same study, researchers found that people wearing AR devices felt significantly less socially connected to the people around them who hadn’t donned headsets.

It's inherently isolating to wear a virtual or mixed-reality headset, says the 2019 paper’s lead study author, Mark Roman Miller, an assistant professor at Illinois Institute of Technology who studies the behavioral impacts of augmented and virtual reality. In-person collaboration becomes more challenging when workers wear headsets because it’s impossible to show anyone else what you’re looking at or to share a screen unless additional layers of software act as mediators, Miller adds.

Miller sees these devices as extraordinary tools, but he warns that they also carry enormous potential for counterproductive distraction. He says he “treats his smartphone like his shoes” by leaving it at the door when he comes home. Augmented and mixed-reality devices could further exacerbate the problem of divided attention that many smartphone users already encounter, he says.

Messeri agrees: every common complaint about how smartphones disrupt our real-life social interactions will only be amplified with mixed-reality headsets, she says. She has spent the past few years embedded with a Los Angeles–based group of tech experts and artists as they develop projects with the goal of expanding human empathy by creating experiences that wouldn’t be possible without VR. This techno-optimist community has its flaws, she says, but its work has been exciting and shows an openness to new possibilities. In contrast, Messeri describes Meta and Apple’s marketing of their mixed-reality devices as “predictable.”

“Meta is saying VR is just another gaming platform.... Apple is saying it’s your iPad but on your face—yet another productivity device,” she says. “If, in the next technological era, all we’re doing is the same thing we do on screens now but on a screen that’s a couple inches in front of our eyes, that’s not inspiring.”

(Lauren Leffer is a contributing writer and former tech reporting fellow at Scientific American.)