Contact: Bob Nelson For immediate release
(212) 854-5573 June 5, 1997
Columbia Software Retrieves, Edits
Images and Videos on the Internet
Shih-Fu Chang, assistant professor of electrical engineering at Columbia
University, and his research team have created software that can search the
Internet for images or videos and then edit them, part of a new generation of
software that will allow users to create their own multimedia productions.
Graphic artists, journal editors, news professionals or anyone who keeps
large files of images or videos will be able to search their archives, or the World
Wide Web, by content. The software will also help students and teachers derive
new curriculum activities from the Web's global trove of information.
One of Professor Chang's new programs, WebSEEk, begins by downloading
files found by trolling the Web. It then attempts to locate file names containing
acronyms, such as GIF or MPEG, that designate graphics or video content. It
also looks for words in file names and associated tags that might identify the
Users can enter words describing the image they seek, or they can sketch
an image, assign color, texture or motion to it, and ask the software to match it.
A magazine editor looking for a picture of a black cat to illustrate a Halloween
layout could type the words "black cat" into the program, or find pictures of cats
under an "animals" heading in the system's picture classification, or could draw
a likeness of a cat and color it black.
When the software finds an image, it analyzes the prevalence of different
colors and where they are located. Using this information, it can distinguish
among photographs, graphics and black-and-white or gray images. WebSEEk
also compresses each picture so it can be represented as an icon for display with
other icons. For a video, it will extract key frames that begin new scenes.
So far, WebSEEk has downloaded and indexed more than 650,000 pictures
and 10,000 videos from tens of thousands of Web sites. The material has been
automatically classified into a taxonomic structure for browsing, with more than
2,000 divisions, such as architecture, arts, nature and people. Professor Chang is
continuing to refine the program, which will become far more powerful with the
eventual adoption of the MPEG-7 standard, which will allow producers of
multimedia to attach information about its content.
Professor Chang's VisualSEEk system gives users the option of sketching
and coloring in shapes corresponding to the image they are looking for. With his
VideoQ software, users can add motion trajectories to find video clips containing
motions in a certain direction, such as a high jumper clearing the high bar or a
racing car spinning out of control.
Another program, WebClip, is an editing tool for digital video, and can be
used either on a local hard disk or on the Internet, bypassing expensive video
editing facilities. Professor Chang's software performs editing functions in
compressed computer code, permitting rapid turnover since the video images do
not need to be fully decoded until they are viewed. Roving correspondents could
record scenes with a digital videocamera - such as the Omnicamera - and then
edit the material before sending it for broadcast.
Professor Chang has also developed technology to add a digital watermark
to video sequences. The watermark consists of a string of digital ones and zeroes
embedded in the digital video, and can be either visible, to prevent re-use of the
material, or invisible. The coding could help news editors authenticate images
they receive from field producers, or could add copyright information.
Software that accesses digital video in various ways will become vastly more
flexible with the adoption of MPEG-7, a comprehensive content description
standard. Information describing the content of a multimedia file will be
attached to the file in a way that any search engine can find. The attached
information could be at any level of abstraction, from language to computer code,
and is sometimes referred to as "the bits about the bits."
One objective of MPEG-7 is to enhance interoperability among different
image or video archives, most of which use proprietary content description
schemes. Information in a standardized format attached to multimedia files
might tell researchers when a video was recorded or give the names of artists who
wrote accompanying music or contributed in some other way. It could also
include codes to index primitive visual attributes, such as shapes, colors, and
motions, of video objects contained in the images or videos.
Professor Chang is among the researchers developing the new MPEG-7
standard, which according to an international timetable is to be implemented in
the year 2000. Users will be able to specify keywords or preliminary visual
information to search engines, which will then rely on the MPEG-7 content
attached to files to conduct fast and efficient multimedia searches.
The National Science Foundation, the AT&T Foundation, the NEC
Research Institute, IBM, Intel, Hewlett Packard, Columbia's Strategic Research
Initiative and sponsors of Columbia's ADVENT forum supported the research.
Professor Chang's image search and edit team includes graduate students
John R. Smith, for WebSEEk; Horace Meng, for WebClip; and William Chen, Hari
Sundaram and Di Zhong, for VideoQ.
Visit his site at http://www.ctr.columbia.edu/advent/demos.html.
This document is available at http://www.columbia.edu/cu/pr/. Working press may receive
science and technology press releases via e-mail by sending a message to firstname.lastname@example.org.