Video editing, especially laying computer-generated graphics on a videotaped image, is still a complicated process. The reason, says Alexandros Eleftheriadis, assistant professor of electrical engineering, is that existing editing terminals, even the Silicon Graphics or Avid machines used in Hollywood, run software that treats images like collections of pixels. Humans do not see or think in pixels, however. They see content—houses, cars, dogs, people—what video engineers call "objects." With new object-based editing software being developed by Eleftheriadis' group, users will soon be able to clip a moving, talking figure from one video sequence and paste it onto another, much the way computer graphics software such as Adobe Photoshop allows graphic designers to paste objects seamlessly into still photographs. "Users will become their own producers and will assemble video content in a natural and intuitive way," Eleftheriadis said. ZEST Eleftheriadis' content creation software, called ZEST, will enable users to manipulate existing audiovisual objects or build new ones, then position them and define how they will behave audibly and visually. The software is easier to use than video editing software now available, and uses less computer power. "Creating appealing and rich content becomes a point-and-click operation on a spatial and temporal canvas," Eleftheriadis says, an operation that even schoolchildren will be able to perform. He will be testing the ZEST tools in several K-12 public schools in a collaboration with the Institute for Learning Technologies at Teachers College.
Eleftheriadis is the editor of the MPEG-4 Systems standard and oversees development of the MPEG-4 specification. He has created a computer language, called FLAVOR (Formal Language for Visual Object Representation), that will define how objects are represented in computer code and will automate much of the process of creating new media editing tools. Flavor will provide the framework that allows computers to automatically encode objects so that they can be manipulated by MPEG-4 editing tools such as ZEST. He is also developing software for personal computers that will play back MPEG-4 content. The player will be embedded as a plug-in for Web browsers to facilitate seamless access to Web content, but could also operate in custom-designed platforms. Dynamic Rate Shaping In addition to object-oriented video, the Columbia engineer is researching ways to make often jerky Internet video more pleasing to viewers. The Internet has limited bandwidth at any one time for any individual user, but the amount of available bandwidth can vary from time to time. Instead of sending more or fewer frames per second to match available bandwidth, which is what existing Internet video software does, Eleftheriadis' new ALIVE program—for Adaptable Internet Video—moves the same number of frames at any time, but adds more or less detail to each frame by sending more or fewer bits to describe the frame. The result is a steadier picture. The technique, called dynamic rate shaping, provides a fast procedure for continuous modification of the rate at which compressed MPEG-1 and MPEG-2 video is sent. The result is extremely smooth playback of video at high frame rates without overloading Internet networks and without special hardware. In a related project, his research group is using automated techniques to detect areas of perceptual importance, such as the face, eyes and mouth, in video sequences. Once the software has decided which areas are important, it can allocate more bits to such areas for increased visual quality. This is particularly suitable for very low bit rate coding applications, such as those found on the Internet. The software allows different areas of a video sequence to have both different spatial resolutions as well as different temporal ones. In effect, the frame rate might be faster or slower in certain parts of the frame, and certain areas of each frame could display greater detail. For instance, a speaker's eyes might convey expression, but might not change much over time; so the eyes could show more detail but little movement. Because the mouth moves frequently but doesn't need much video detail, it could show greater movement, with less detail in each frame. Eleftheriadis' other research interests are in the areas of video signal processing and compression, video communication systems (including video-on-demand), distributed multimedia systems and the fundamentals of compression. For more information on Eleftheriadis' research, visit http://www.ee.columbia.edu/~eleft.
| |||||||
