Multimedia Information Processing Group


3D4YOU - Content Generation and Delivery for 3D Television

Project Description

The 3D4YOU project is a European research project to develop and establish standards for 3D Television in a wide range of aspects. Combining the expertise of different teams from across Europe, the goal of the project is to build a 3D Television production chain from content generation to end-consumer delivery.


Research Focus

The contribution of the Multimedia Information Processing Group of CAU University to the EU3D4YOU project is on the one hand in the calibration of the multi-camera capture setup as well as the generation of dense depth maps for 3D viewing devices at high definition quality.
To achieve reliable depth retrieval, a Time-of-Flight Depth Camera is integrated with about the same Field of View as the CCD Cameras to initially capture a low resolution depth map of the scenery. This low resolution depth map can then be refined with the help of high resolution color data.

The above image shows first results from a test sequence shot with the camera setup to the left. The low resolution depth map generated from the ToF-camera is warped into the left camera view and automatically refined to achieve a high resolution depth map.

One particular outcome of the project so far is a mixed reality systems exploiting ToF-Camera technology, using depth keying algorithms and depth layer segmentation.
The steps of the processing chain for the mixed reality system are:

  1. Environment model generation with Time-of-Flight camera.
  2. Camera pose determination
  3. Shadow computation
  4. Depth-keying and mixing
  5. Tracking of dynamic foreground objects and content alignment

This way mixed reality content with increased realism can be generated, while at the same time all necessary data for 3D-TV application is delivered.


Multi Camera System

Although under investigation the currently proposed camera system for capturing input data for the generation of 3D Television content consists of four CCD HDTV cameras in a multiview rig together with a Time-of-Flight depth camera, which can be used to initialize the computation of dense depth maps of the scenery at the resolution of the HDTV cameras. This seemingly complicated setup with additional satellite cameras is suitable for multi-view stereo approaches as well as for occlusion layer generation, yet staying as compact as possible.
The depth information will then be available for all the camera views in the multi-view rig as well as for interpolated views between them to allow for different 3D display types based on stereoscopic or autostereoscopic technology.


Further Information

For a more detailed overview on the 3D4YOU project and to visit our project partners, please visit the 3D4YOU Homepage.