Live Immersive Video-Audio Interactive Multimedia
3D VIVANT: Live Immersive Video-Audio Interactive Multimedia
The aim of the project 3D VIVANT is to capture events automatically in 3D and deliver them for realistic, interactive and immersive play back to home users-viewers. This requires the project to create, develop and integrate a comprehensive range of components from the generic technology of 3D imaging. Real and virtual 3D content are homogeneously combined/mixed producing a novel form of rich and interactive content, defined as "3D Holoscopic content". 3D Holoscopic imaging is a technique for creating full colour 3D optical models that exist in space independently of the viewer. The images exhibit continuous parallax throughout the viewing zone.
Pic1. Brunel Holoscopic 3D Camera
The continuous nature of the images produced with this method eliminates the effect of cardboarding (flattening of objects into discrete depth planes) and flipping (a visible effect created by moving between image fields) present in multiview stereoscopic systems. Unlike multiview stereoscopy, information about a point is contained in many different parts of the image plane. In this respect, 3D Holoscopic imaging is akin to holography.
However, 3D Holoscopic imaging is more advantageous, since it can operate under incoherent illumination, which is in contrast with holography, and hence it allows more conventional live capture and display procedures to be adopted. Furthermore, in viewing the 3D optical model accommodation and convergence work in unison (i.e. the viewer's eyes focus and converge to the same point) to prevent eye-strain from occurring.
The 3D VIVANT project investigates the possibility of using different technologies for capture and display of 3D content. For the capture, 3D VIVANT takes full advantage of 3D Holoscopic imaging technology, where a single camera is required. Hence, in this project a world first single aperture ultra-high definition 3D Holoscopic imaging camera is constructed which will permit live capture of 3D content. For the display, the project will take advantage of Holography to provide immersive, ultra high resolution presentation of 3D content. To increase the immersive experience, the generation and playback of 3D spatial sound is also investigated. The developed system is tested using two user cases, namely, broadcasting and online hyperlinking. For this purpose, various 3D Holoscopic video processing algorithms, such as 3D codecs and 3D object segmentation, as well as search and retrieval techniques are developed.