Over the past three years, researchers in the Camera Culture group at the MIT Media Lab have steadily refined a design for a glasses-free, multi-perspective, 3D video screen, which they hope could provide a cheaper, more practical alternative to holographic video in the short term.
Now they’ve designed a projector that exploits the same technology. The projector can also improve the resolution and contrast of conventional video, which could make it an attractive transitional technology as content producers gradually learn to harness the potential of multi-perspective 3D.
Multi-perspective 3D differs from the stereoscopic 3D now common in cinemas in that the depicted objects disclose new perspectives as the viewer moves about them, just as real objects would. This means it might have applications in areas like collaborative design and medical imaging, as well as entertainment.
The MIT researchers — research scientist Gordon Wetzstein, graduate student Matthew Hirsch, and Ramesh Raskar, the NEC Career Development Associate Professor of Media Arts and Sciences and head of the Camera Culture group — built a prototype of their system using off-the-shelf components.
The heart of the projector is a pair of liquid-crystal modulators (tiny liquid-crystal displays) positioned between the light source and the lens. Patterns of light and dark on the first modulator effectively turn it into a bank of slightly angled light emitters — that is, light passing through it reaches the second modulator only at particular angles.
The combinations of the patterns displayed by the two modulators thus ensure that the viewer will see slightly different images from different angles.
The researchers also built a prototype of a new type of screen that widens the angle from which their projector’s images can be viewed. The screen combines two lenticular lenses — the type of striated transparent sheets used to create crude 3D effects in publications.
For every frame of video, each modulator displays six different patterns, which together produce eight different viewing angles: At high enough display rates, the human visual system will automatically combine information from different images.
The modulators can refresh their patterns at 240Hz, or 240 times a second, so even at six patterns per frame, the system could play video at a rate of 40Hz, which, while below the refresh rate common in today’s TVs, is still higher than the 24 frames per second standard in film.
With the technology that has historically been used to produce glasses-free 3D images — known as a parallax barrier — simultaneously projecting eight different viewing angles would mean allotting each angle one-eighth of the light emitted by the projector.
But like the researchers’ prototype monitors, the projector takes advantage of the fact that, as you move around an object, most of the visual change takes place at the edges.
If, for instance, you were looking at a blue mailbox as you walked past it, from one step to the next, much of your visual field would be taken up by a blue of approximately the same shade, even though different objects were coming into view behind it.
Algorithmically, the key to the researchers’ system is a technique for calculating how much information can be preserved between viewing angles and how much needs to be varied. Preserving as much information as possible enables the projector to produce a brighter image. The resulting set of light angles and intensities then has to be encoded into the patterns displayed by the modulators.
That’s a tall computational order, but by tailoring their algorithm to the architecture of the graphics processing units designed for video games, the MIT researchers have made it run almost in real time. Their system can receive data in the form of eight images per frame of video and translate it into modulator patterns with very little lag.