Google showcases light field capture tech

Advertising giant Google has announced the results of its experimentation with light field technology as a means of improving the sense of presence in virtual reality, while at the same time releasing its spatial audio software under a permissive licence.

While Google's first dip into the waters of virtual reality was both basic and extremely low-cost - the 2014 Google Cardboard accessory for smartphones, the spiritual precursor to Nintendo's upcoming Labo Toy-Con family for the Switch - the company has been getting increasingly serious about both augmented and virtual reality technology in the last year or two.

One result of this is the release of Welcome to Light Fields, a free package on Valve's SteamVR platform which is claimed to showcase the company's work on recoding light fields for an improved sense of presence in virtual reality.

'With light fields, nearby objects seem near to you - as you move your head, they appear to shift a lot. Far-away objects shift less and light reflects off objects differently, so you get a strong cue that you’re in a 3D space,' explains Google's Paul Debevec, senior researcher for virtual reality technology, in the company's blog post. 'And when viewed through a VR headset that supports positional tracking, light fields can enable some truly amazing VR experiences based on footage captured in the real world. This is possible because a light field records all the different rays of light coming into a volume of space. To record them, we modified a GoPro Odyssey Jump camera, bending it into a vertical arc of 16 cameras mounted on a rotating platform.

'It takes about a minute for the camera rig to swing around and record about a thousand outward-facing viewpoints on a 70cm sphere. This gives us a two-foot wide diameter volume of light rays, which determines the size of the headspace that users have to lean around in to explore the scenes once they are processed. To render views for the headset, rays of light are sampled from the camera positions on the surface of the sphere to construct novel views as seen from inside the sphere to match how the user moves their head. They’re aligned and compressed in a custom dataset file that’s read by special rendering software we’ve implemented as a plug-in for the Unity game engine.'

The demonstration package Welcome to Light Fields includes a series of environments Google has hand-picked to showcase the technology, including the Mosaic Tile House in Venice, St. Stephen's Church in Granada Hills, and the flight deck of the Space Shuttle Discovery - which, the company crows, has never been accessible to the public. Welcome to Light Fields is available to download now via Steam.

At the same time, Google has announced that it is releasing its Resonance spatial audio technology under the permissive Apache Licence 2.0, with software development kits for the Unity and Unreal game engines along with web use, Wwise, FMOD, and digital audio workstation utilities. The code is available now on the company's GitHub repository.