Offices and Job openings on Mixed Reality with HoloLens: Microsoft Mesh

We´ve all heard about this new Microsoft platform. Launched recently during the Ignite, Microsoft Mesh aims to be the next ultimate revolution for offices and collaborative work.

 

It is no news that since the pandemic the virtualization of workplaces together with the massification of Microsoft Teams, achieved a revolution never seen before.

Microsoft Teams converted into the epicenter of a lot of companies and the best tool for connection and Modern Work. Not short after the world was getting used to this reality, Microsoft brings us news that can change the whole paradigm.

 

Before mentioning Mesh, it is important to highlight the context: After 4 long years of waiting, in 2019 the HoloLens team launches the second version of this device. Also, in 2019 during the Inspire event, Microsoft presented through Julia White their own avatar in holographic form and spoke in Japanese in real-time. This project combined Mixed Reality and Artificial Intelligence and has supposed the first steps for Holoportation.

 

 

Today, 2 years later, with the boom in teams and these solutions, Microsoft launches Mesh the new platform for collaborative environments that enables you to have a meeting and virtual encounters with people or 3D objects. This platform which bets on being one of the greatest revolutions in working environments is built on Azure and is currently available for HoloLens. However, Microsoft plans for it to be available on much more devices. In addition, it’s fully compatible with Microsoft teams and Microsoft 365 suite.

Imagine highly effective and interactive work meetings with colleagues from all over the world where projects can be presented. Imagine the breakthrough this could mean for changing current educational models or medical assistance.

Microsoft’s overall aim is to develop numerous multi-device applications and cross-platform on Mixed Reality so they can be used freely in Mesh.

Facts about Mesh that you should know:

 

 

Multi-device: Compatible with many devices like Microsoft HoloLens, HP Reverb G2, or Oculus Quest 2, for virtual experiences, mobile devices, Android tablets, and Microsoft, Linux, and Mac computers.

A platform for developers: Azure is the central platform for any development with identity services like Azure Active Directory and Microsoft Accounts. Microsoft Graph allows the connections, contents, and preferences of both commercial and consumer spaces.

Beyond it is the main platform, it used AI that allows Mesh to address some of the most complicated  technical challenges when it comes to enabling massive online multi-user cases for Mixed Reality like immersive presence, spacial maps, holographic rendering, and multi-user synchronization

Immersive experiences: A fundamental aspect of Mesh is to be able to represent the users in different ways depending on the device they use to join. Mesh offers the most accessible 3D presence with representative avatars through the internal sensors of the devices. The platform has preloaded numerous avatars that are ready to use. Additionally, it can also modify these existing avatars with movement models powered with AI to capture movement, specific and coherent expressions with the action of the user.

Mesh also enables the most photorealistic 360° Holoporation with outdoor sensors. These external sensors can be a custom camera setup such as Mixed Reality Capture Studio, which helps capture in 3D with full fidelity, or it can be Azure Kinect, which captures depth-sensing images to help produce the holographic representations.

Mapping: The creation of applications with holographic content in the real world, requires a common perspective of the space that surrounds each participant as well as an understanding of the physical world.

Reliably placing holograms that can persist across time, space and devices is a common need.

All this is available in mesh through maps. Before Mesh, each device had its own local vision of the world. With Mesh, these local caches merge and optimize to have a global understanding of space/surroundings where they find themselves. This framework allows you to anchor content, share device views, and collaborate on 3D models.

Rendering: Mesh allows you to choose between independent local rendering or remote rendering connected to the cloud without any problem within its application for each case and model. In addition, holographic rendering supports most 3D file formats for native rendering in Mesh-enabled applications, solving the challenge of incorporating users’ existing 3D models for collaboration.

Multiuser: Within Mesh the shared context is enabled through multi-user synchronization. This is what illuminates any updates to participant poses, movements, and expressions or any holographic transformations that are occurring in the space. All of this happens in 100 milliseconds of latency, regardless of whether the user is in the same physical space or on the other side of the world. All of this is complemented by spatial audio in Mesh, which creates the sensation of being in the same physical space in a multi-user scenario.