In a breakthrough that could change the face of augmented reality (AR) and virtual reality (VR), a novel method of 3D capture called Gaussian splatting is gaining widespread adoption in the tech industry. This innovative approach has the potential to democratize 3D capture, enabling creatives and individuals to generate photorealistic 3D worlds with unprecedented ease and accuracy.

The technology has already been adopted by major players such as Niantic, Google, Snap, and Meta, which are leveraging Gaussian splatting to build out their respective mapping products and create immersive AR and VR experiences. According to AR/VR expert and investor Tipatat Chennavasin, Gaussian splatting is a "huge game changer" that could soon allow anyone to scan entire rooms and transport us to entirely new 3D worlds.

Unlike traditional polygon mesh-based 3D capture and reconstruction, which often results in plasticky and unreal renderings, Gaussian splatting captures 3D objects as a collection of fuzzy, translucent blobs, or Gaussians. Each blob is captured with exact information on its color, location, scale, rotation, and level of transparency, resulting in a much more detailed and realistic picture of a 3D object. This approach has been hailed as a "JPEG moment for spatial computing" by Chennavasin.

Niantic, the maker of Pokémon Go, has been at the forefront of Gaussian splatting adoption, acquiring the Scaniverse app in 2021 and adding Gaussian splatting as a capture technology last year. The company has also open-sourced its own file format for splats and expanded Scaniverse to VR, enabling users to view Gaussian splats in Meta's Quest headsets. Niantic's SVP of engineering, Brian McClendon, believes that Gaussian splatting will democratize 3D capture and is a "fundamental change" in the field of 3D graphics.

Gaussian splatting is not only limited to static content; it's also being used to record volumetric 3D videos. Computer vision startup Gracia AI has been using the technology to record 3D videos, which can be viewed on Meta Quest headsets. The company has been able to record scenes in almost complete darkness, with no restrictions on clothing or hair, and has achieved a level of creative flexibility that was previously unimaginable.

Meta has also been experimenting with Gaussian splatting, unveiling an app called Hyperscape at its Meta Connect conference this fall. The app allows users to explore photorealistic 3D renderings of scanned spaces, including artist studios and a conference room on Meta's campus that once served as Mark Zuckerberg's office. Meta envisions Gaussian splats as a tool for VR creators to build immersive worlds and experiences for its social VR platform, Horizon Worlds.

While the technology is still in its early stages, the potential implications are vast. With the combination of Gaussian splatting and generative AI, everyday people may soon be able to generate photorealistic 3D spaces with AI prompts, Gaussian splat captures, or a mixture of both, and then enter those spaces with VR headsets or AR glasses. As Chennavasin puts it, "The killer app of XR is a multiplayer holodeck. Generative AI and Gaussian splats is how we create it at a visual fidelity that's almost indistinguishable from reality."

As the development of Gaussian splatting tech continues to advance at a rapid pace, one thing is clear: we are on the cusp of a revolution in 3D capture and immersive experiences. The question is, what will you visit first in your own personal holodeck?