A new digital videocamera can see in all directions at once. Placed atop a concert stage or at midfield of a sports event, the Omnicamera could provide a 360-degree view—an entire hemisphere—to television or Internet viewers. With a joystick or mouse, they could bring any view to their screens, and see not hemispheric perspective, but normal, undistorted, linear perspective. They could even create onscreen windows to see several views at once.

  As the camera has no moving parts, the number of viewers who can see the view they wish to see is limited only by the bandwidth of the connecting network.

  Shree K. Nayar, inventor of the Omnicamera and professor of computer science, has spent six years researching his physics-based approach to artificial vision. His laboratory has developed four Omnicamera prototypes, with configurations for surveillance, teleconferencing, entertainment and robotic vision. Columbia has filed a U.S. patent application covering the Omnicamera videocamera and the Omnivideo software.

  

A still photograph taken on campus by Omnicam, which also works with video.

  The Omnicamera consists of a miniature digital videocamera mounted in a frame aimed directly at the apex of a parabolic mirror, a small inverted cup of polished metal enclosed within a transparent hemisphere. Columbia graduate student Venkat Peri has developed software that allows multiple Omnicamera images to be displayed on a computer screen in linear perspective at any magnification.

  "Computer vision research is attempting to construct systems that can perceive our environment using man-made sensors, such as cameras, in a way that is analogous to how we use our eyes," Nayar says. "The Omnicamera can view more of a scene than the human eye can, which makes it valuable for a number of applications."

  Two Omnicameras mounted back to back can produce views of 360 degrees, a complete sphere, for surveillance or security operations. In teleconferences, an Omnicamera can show simultaneously every participant seated around a table, in either hemispheric or linear perspective. It will allow a mobile robot to determine its location and direction of travel from local features. Because the camera views itself at the parabola's apex, there is one small blind spot in each hemisphere.

  The Omnicamera's parabolic optics ensure that it has a single effective center of projection, a single point through which all rays from a scene must pass on their way to the camera's lens. That design mimics a camera that takes in only linear perspective, and allows the Omnicamera's computer software to generate linear perspective images that are free of distortion.

  Other vision researchers have tried to create omnidirectional vision systems using fisheye lenses or planar, spherical, conical or pyramidal mirrors. Most of these do not yield the single viewpoint necessary to construct linear perspective images, or, if they do, use elaborate and hence expensive designs, Nayar said.

  Using his physics-based approach to vision research, Nayar has created mathematical models that represent the interaction of light with the scene and the sensor. Based on these models, he has developed vision sensors and algorithms that recover physical properties (shape, reflectance, etc.) of the scene from one or more images. These properties are then used by a computer system to reason about the physical world and perform intelligent tasks such as object recognition, manipulation, navigation and visual inspection.

Visual Recognition System

  His other innovations include a computer vision system that can recognize an object it has seen before, even if the previous encounter was from a different angle or in different lighting. Nayar has developed a comprehensive software package that can be used in conjunction with a personal computer and videocamera for real-time object recognition. The package has been distributed to a number of industrial and academic research institutions and is being used for real-time visual tracking of a moving object by a robot manipulator and inspection of manufactured parts.

  A major component of his research is dedicated to creating three-dimensional images from two-dimensional information. Nayar has given particular attention to shape from shading, photometric stereo, binocular stereo, shape from focus and depth from defocus. Recent results have led to the development of a real-time shape sensor that produces high-resolution depth maps—a simple three-dimensional image—at video frame rate. Such sensors are useful not only for vision tasks such as recognition and inspection but also automatic CAD model generation for vision and graphics, robotic assembly and remote visualization.

  For further Omnicamera information, or to see an online demonstration, visit the laboratory's web site: http://www.cs.columbia.edu/CAVE.