Google's 3D Video Calling Is the Closest Thing to a Hologram I've Ever Seen
Commentary: I tried out a 3D Project Starline call, and it felt like making eye contact with someone who wasn't actually there.
- Named a Tech Media Trailblazer by the Consumer Technology Association in 2019, a winner of SPJ NorCal's Excellence in Journalism Awards in 2022 and has three times been a finalist in the LA Press Club's National Arts & Entertainment Journalism Awards.
Project Starline is designed to make it feel like people are in the same room.
"Oh my God, stop," are the first words to come out of my mouth as I see Project Starline in action.
I'm sitting in a small booth at Google I/O in Mountain View, California, with a large TV-like display in front of me. Suddenly, the screen comes to life, and the most-3D rendering of a person I've ever seen jumps out at me. I'm now apparently face-to-face with Andrew Nartker, general manager of Project Starline, the immersive videoconferencing tool I'm trying out. He's streaming in from a separate booth, but it feels like he's sitting right across from me.
When he reaches his arm out, it looks like it's actually extending in my direction. At one point, he holds out an apple, and it almost feels like I can grab it. Most notably, I get the sense I'm making direct eye contact with him, which is simultaneously mind-blowing and unsettling.
Read more: AI Was Google's Biggest "Product" at I/O. Tech Keynotes Will Never Be the Same
What I got to demo is a prototype, but it showcases the futuristic vision behind Project Starline, which Google describes as a "magic window." Seven cameras are positioned around the displays in front of me and Nartker, respectively, which stream video between us. That footage is fed into an AI model, which reconstructs those 2D videos into 3D ones. A flat display that can sense depth, called a light field display, receives the 3D video stream, and depicts each of us at an accurate scale, complete with shadows and light that are "all generated in a way that makes it feel perceptually correct for you," Nartker says. The display can gauge where you're looking and orient images in a way that looks realistic from that vantage point.
The idea behind Project Starline, which was unveiled in 2021, is to make virtual meetings — whether with friends, family, colleagues or your doctor — feel more personal, impactful and memorable. My colleagues Scott Stein and David Lumb have also gotten to try it out over the years, and I'm thrilled to join their ranks.
The most recent update to the project came last week, when Google said it's teaming up with HP to start commercializing Project Starline, with the goal of making the tech available in workplaces next year. Exactly how and with what equipment still isn't clear, but Google noted in a blog post that it's "working to enable it directly from the video conferencing services you use today — such as Google Meet and Zoom."
What stood out to me about Starline was the high-quality stream with impressively no (or imperceptible) lag, separating it from the awkwardness of standard video calls, where it's easy to talk over people. This actually compelled me to chime in more often during my chat with Nartker, because it felt like a normal, in-person back-and-forth chat, rather than a virtual call where you have to pause to make sure the other person is done talking.
What's also unique about this experience is the ability to register someone's body language, as you would in the real world, compared with the flat, two-dimensional rendering you'd normally get on a video call. You can more fully take in gestures like shifting around or motioning with hands, which creates a more holistic and lifelike interaction. I tend to have a hard time focusing on Zoom calls because staring at people through a screen isn't very dynamic, so I can imagine being more engaged and attentive if this was how I was interacting with someone virtually.
In fact, one of Google's studies found that "participants were more animated, using significantly more hand gestures (+43%), head nods (+26%), and eyebrow movements (+49%)," compared with traditional videoconferencing. "Participants also reported a significantly better ability to perceive and convey nonverbal cues in Project Starline than in traditional video conferencing."
Google says this makes it easier to have meaningful conversations virtually, like a performance review or interview, instead of waiting to see someone face-to-face. Study participants also reported less video fatigue compared with traditional videoconferencing, and more attentiveness, thanks in part to the removal of other distracting browser windows. I can attest it's hard to look elsewhere when you're drawn into this entrancing display (though I wonder if the allure will inevitably wear off after multiple uses and my phone will suddenly be a tempting distraction again).
I'm curious to see what Google does with this technology, and how it rolls it out more widely — and if other people will be as stunned as I was when trying it out for the first time. For now Nartker and I give each other a virtual high five through the screen — but then redo it in real life when we reconvene after the demo, for good measure. After all, video calls can't replace everything.
