Imagine it’s time for a meeting. Instead of clicking on a Zoom or Google Meet link, you put on an AR/VR headset. And rather than stare at co-workers through tiny windows, you interact with their avatars in a virtual space and pass “objects” to one another.
That’s the vision for Mesh, a new cloud mixed-reality platform, officially announced Tuesday at Microsoft Ignite, the company’s annual developer conference.
Taking a meeting in Microsoft Mesh
“I can tell you’re sitting down,” Greg Sullivan, Microsoft’s director of mixed reality, told me despite the fact neither of us had our webcams on. With my HoloLens 2 strapped to my head, I stood up, walked closer to the virtual table, and looked at an avatar of Sullivan before he turned to me and said, “This is Microsoft Mesh.”
During my Mesh preview, Sullivan handed me a bunch of holograms, including the moon, a jellyfish, and a moving shark.
Thanks to the hand-tracking built into the headset, I simply reached out and grabbed them with my hand. From there, I pinched my fingers around the objects, and made them bigger or smaller by moving my hands apart or closer together.
Below is an example of what these holograms look like (please ignore my messy room, thank you very much):
As you can see, the detail is impressive. But the virtual space is pretty limited. The entire experience only consisted of a table and two avatars. Granted, Microsoft said the demo was specifically meant to show “all the core capabilities of Microsoft Mesh: Presence and 3D collaboration.” And, technically, the company delivered on that.
The experience will vary depending on what developers do with it and the type of device you use. Like I said before, I tried it out with HoloLens 2 which allows you to see your surroundings, so a VR headset might feel more all-encompassing.
And, speaking of VR headsets, Microsoft Mesh isn’t limited to the HoloLens 2. It’s also compatible with all Windows mixed-reality headsets (including those from HP, Lenovo, Acer, Samsung, and Asus), and the Oculus Quest and Quest 2.
What remains constant regardless of the device however, are the avatars — which look super cartoonish. They’re the same ones you’ll find in in AltSpace, a social VR platform that launched in 2015 and was acquired by Microsoft in 2017.
While creating my own avatar, which you can see below, the options felt super limited.
Still, thanks to that aforementioned hand-tracking, the meeting felt more personal watching Greg’s arms wave as he explained something, the same way he probably would in person. And, it also allowed him to see whether I was using the gestures correctly.
At one point, I had trouble opening the menu, which appears as soon as you raise your arm. In Mesh, Greg noticed that it wasn’t popping up for me because I had my fist closed instead of open, and he immediately corrected me.
But it does struggle when it comes to facial expressions, apart from the occasional smile. Microsoft is well aware of this, and is currently working on it. Considering that HoloLens 2 features eye tracking, it’s technically possible to show eye contact between avatars.
“The short answer is, yeah, that’s feasible,” Sullivan said. “And it is the type of thing that one could imagine we would iterate on to improve that, because we want to be able to convey more of the nonverbal communication clues. And that includes a lot of facial expressions.”
A collaborative experience, but not for everyone (yet)
Eventually, Mesh will work with smartphones, PCs, and Macs, so those without headsets can also join in — which truly opens up access to the software. For example, if you’re using Microsoft Teams, it might someday be possible to place the video conference window into the virtual meeting space. That would let you interact with avatars and view holograms from your computer screen.
“The way I view it is, in the same way that TV didn’t kill radio, I don’t think headsets are gonna make our other devices go away,” Sullivan said. “But in the right world, they’ll work in harmony. And we’ll have a constellation of devices that we use for the appropriate scenario.”
And, all that sounds really freakin’ cool, but I wouldn’t count on your workplace to adopt it anytime soon. At the moment, Microsoft Mesh is best for businesses that work with a lot of physical prototypes. In fact, the platform is currently being used to develop the HoloLens 3.
Okay, it doesn’t look like this at all yet but eventually it might.
Image: microsoft
Sullivan explained that in the past the HoloLens team would 3D-print a mock-up of a product and ship it out internally, before gathering around a conference table to discuss it. Then, whenever a part of it had to be changed, the entire process would repeat itself until they got it right.
“The power of not having to go create another clay model of a car, or a mock up 3D printed version of [HoloLens] is really impactful in terms of efficiency,” Sullivan told Mashable, “That’s why we’re already using [Mesh] to design the next HoloLens because that scenario is super compelling.”
Aside from tech giants like Microsoft, it can also be useful for those who don’t necessarily work in front of a computer all day.
“If you are someone who works on an assembly line, or repair complex machinery, or a bunch of other kind of first line worker scenarios, they don’t give you a laptop on that job, you don’t really directly benefit from the digital revolution,” Sullivan said.
He used civil engineers as an example. By uploading a really high-resolution image of a structure such as a bridge, engineers can gather in Mesh to see if there are any cracks or elements that need to be repaired right away.
But what about the rest of us?
Beyond work, Microsoft also sees Mesh as a way to attend social gatherings, such as concerts and even bars, with friends and family.
Okay, this looks a bit more true to what I experienced.
Image: microsoft
And, while that sounds lovely, it’s tough to ignore that VR and mixed-reality headsets are quite the investment. While the Oculus Quest 2 is considered the more affordable device, starting at $299, the HP Reverb 2 will set you back $600. And that’s without the proper PC and software you’d need to connect the headsets to. Meanwhile, the HoloLens 2 is listed for a whopping $3,500 on Microsoft’s site.
But Sullivan predicts that these expensive bulky headsets, with all their confusing wires and setup requirements, won’t be our only option forever.
He says “less intrusive” form factors — imagine something that looks like a normal pair of glasses — could come in the “not too distant future.”
It’s also worth noting that Microsoft isn’t the only company that offers this type of software. Collaborative workspaces in VR already exist, with the likes of Spatial, VrChat, Mootup, and more. So, with Mesh, it’s really up to Microsoft and developers to find ways to make the platform stand out.
It’s currently available in preview. Those with a HoloLens 2 can download the Microsoft Mesh app while others can request access to a Mesh-enabled version of AltSpaceVR — allowing users to hold meetings in VR. Developers, on the other hand, can join the Mixed Reality Developer Program for updates on when the SDK will be available.
I hate going to the doctor so, I’d be down with visits through Mesh instead.
Image: Microsoft
For those who own a HoloLens 2 and plan on downloading the app, I’m warning you: The state of Microsoft Mesh is currently super bare bones. While the ability to grab and manipulate 3D objects with just my hands was pretty cool, it’s still very much in its infancy. For a more immersive experience, you’ll want to experience it in a VR headset instead.
The future of Mesh
Where was this immersive experience during MY demo?!
Image: microsoft
Even if the technology works, people have to actually use it to reach the collaborative future that Microsoft envisions. Asked if the company is worried about lack of adoption for Mesh, Sullivan gave an honest answer.
“For such a successful company, we’re like, super, super worried all the time…but that helps inform our strategy. And so that’s what led to us thing, [that] we should make [Mesh] available to as many devices as humanly possible,” he said. “We’re not monetizing your behavior. We’re not monetizing your private information, we are going to build a service that we think is valuable enough that people will pay for it. And that’s how we’ll make our money.”
At this point, I’ll pay any amount of money for a virtual space if it means I no longer have to see my friends and co-workers in Brady Bunch-style windows. At this point, even an empty room with just a table and few holograms sounds far more entertaining than Zoom.
WATCH: Here are some private video chat alternatives to Zoom
