Augmented Reality in Architectural Construction,
Inspection, and Renovation
Anthony Webster (1), Steven Feiner (2), Blair MacIntyre (3), William Massie (4), Theodore Krueger (5).
1. Associate Professor of Architecture, Graduate
School of
Architecture, Columbia University. Associate
Member, ASCE.
2. Associate Professor of Computer Science,
School of Engineering
and Applied Science, Columbia University.
3. Graduate Research Assistant, Department
of Computer Science.
4. Adjunct Assistant Professor of Architecture,
Columbia University
5. Adjunct Associate Professor of Architecture,
Columbia University
Abstract:
We describe our preliminary work in
using "augmented reality" techniques to develop improved
methods for the construction, inspection and renovation of architectural
structures. "Augmented reality" refers to the use of
a head-worn interface to overlay graphics and sounds on a person's
naturally occurring visual and audial perceptions. As the person
moves about, the position and orientation of his or her head is
tracked, allowing the overlaid graphics to remain tied to the
physical world. We illustrate our existing augmented
reality system that shows the location of columns behind a finished
wall, the location of re-bars inside one of the columns, and a
structural analysis of it. Our preliminary work
in developing an augmented reality system for improving the construction
of spaceframes is discussed. We briefly describe our preliminary
work on a outdoor augmented reality system that will overlay information
about the nearby environment on a person's normal vision. Potential
uses of more advanced augmented reality systems are discussed.
Introduction
A variety of computer technologies and
computer science techniques are now used by researchers aiming
to improve aspects of architectural design, construction and maintenance.
Virtual reality systems are used to envision modified cityscapes,
and to assess the impact of proposed buildings (Novitski 1994).
Both virtual reality and desktop-based computer systems are currently
used in demonstration testbeds to simulate complex construction
operations. These systems promise to improve the optimization
of construction operations and to allow checks of constructability
and maintainability before building materials are ordered (Virtual
1995, Oloufa 1993); integrated structural, architectural, mechanical
building databases are being combined with engineering expertise
into knowledge-based systems for improving the design process
(Myers et. al. 1992). Robotics systems, mostly adapted from the
automotive industry, have also been used recently in experimental
and commercial attempts to automate various aspects of building
construction (Webster 1994, Richards 1994).
Augmented Reality Applications in
Architecture and Structural Engineering
Recent advances in computer interface
design, and the ever increasing power and shrinking size of computers,
have recently combined to make the use of "augmented reality"
possible in demonstration testbeds for building construction,
maintenance and rennovation. In the spirit of the first see-through
head-mounted display developed by Sutherland (Sutherland, 1968),
we and other researchers (e.g., (Robinett, 1992; Caudell &
Mizell, 1992; Bajura & Neumann, 1995)) use the term "augmented
reality" to refer to enrichment of the real world with a
complementary virtual world. We define an "augmented reality
system" as a head-worn interface that overlays graphics and
sound on a person's naturally occurring visual and audial perceptions.
Augmented reality systems also track users in space, so that visual
information provided by them can be tied to the physical environment.
We refer to "augmented realities" as the graphics and
sounds produced by the system. Unlike virtual realities, which
use virtual worlds to replace the real world, augmented realities
enhance the real world by superposing information onto it. The
spatial tracking capabilites of augmented reality systems distinguishes
them from the heads up displays featured in some wearable computer
systems (Jobsite 1993, Patents 1994).
As part of a program aimed at developing
a variety of high performance user interfaces, we have begun work
on three augmented reality systems for use in structural engineering
and architectural applications. The first, called "Architectural
Anatomy," creates an augmented reality that shows users portions
of Columbia's Schapiro Center for Engineering and Physical Science
Research that are hidden behind architectural or structural finishes,
and allows them to see additional information about the hidden
objects. We have built structural and architectural models of
parts of the Schapio building, including Professor Feiner's lab,
which provide data for use in this "x-ray vision" demonstration
testbed system. The model is based on the as-built construction
drawings provided by the building's architects. Our prototype
application overlays a graphical representation of portions of
the building's structural systems over a user's view of the room
in which they are standing. A head-mounted display/tracker provides
a user with monocular augmented graphics and tracks her head ultrasonically
(figure 1). Figure 2 is a view of a corner of Professor Feiner's
lab photographed through a version of our see-through head-mounted
display that is designed to be worn by a 35mm camera. A corner
of the ultrasonic tracker transmitter can be seen at the lower
left. The overlaid virtual world visible in this figure includes
the outlines of parts of three support columns and the space between
the structural concrete floor and the raised lab floor above it.
The middle, larger column is inside the protrusion in the corner.
The two other, smaller columns are actually located in nearby
rooms. The X11 cursor is visible near the desk. Our prototype
allows the user to select a column by clicking a mouse in order
to see more information about it. In Figure 3, the user has looked
down and slightly to the left and has selected the middle column
that contained the cursor in figure 2. This causes the outlines
of the other support structures to dim. (This project's display
hardware is one-bit deep, so dimming is accomplished through the
use of different line styles; in this case, dotted lines.) As
shown in figure 3, the re-bar inside the column is revealed and
a structural analysis of the column is presented to the user.
The analysis is provided by Dast, a commercially available structural
analysis and design program (Das, 1993).
The augmented reality system's monocular
graphics are provided by a Reflection Technology Private Eye display
and a mirror beam splitter. A Logitech ultrasonic tracker to
provides position and orientation tracking (the display and triangular
tracker are shown in figure 1). The display's graphics are rendered
at 720x280 resolution and, in the application described here,
include 3D vectors without hidden-line removal. We provide support
for 2D applications like the column structural analysis through
a full memory-mapped X11 Window System (Scheifler & Gettys,
1986) server. The X11 bitmap is treated as if it were projected
onto a portion of the surface of a virtual sphere surrounding
the user and is composited with the bitmap containing the 3D graphics
for presentation on the head-mounted display (Feiner et al., 1993).
Our augmented reality testbed allows an X11 window to be positioned
so that a selected point on the window is fixed at an arbitrary
position within the 3D world. We refer to such windows as world-
fixed windows to distinguish them from windows that are fixed
to the display itself or to the body- tracked virtual sphere.
Building on our work on knowledge-based augmented reality for
maintenance and repair (Feiner, MacIntyre, & Seligmann, 1993),
we are developing a knowledge-based system that will dynamically
control which parts of the structural system are displayed to
satisfy the user's goals in exploring the environment.
Our second augmented reality system
aims to improve aspects of spaceframe construction. Spaceframes
are typically made from a large number of similar size and shape
components (typically cylindrical struts and spherical nodes).
Although the exterior dimensions of all the members may be identical,
the forces they carry, and therefore their inner diameters, vary
with their position in the structure. Consequently it is relatively
easy to assemble pieces in the wrong position - which if undetected
could lead to structural failure. Our augmented reality construction
system is designed to guide workers through the assembly of a
spaceframe structure, to improve the probability that each member
is properly placed and fastened.
Our prototype spaceframe structure,
shown in figure 4, is a diamond shaped, full-scale aluminum system
manufactured by Starnet International (Starnet 1995). We have
created a geometric computer model of the spaceframe, an ordered
list of assembly steps, and a digitized set of audio files containing
instructions for each step. Our headworn augmented reality interface
includes a Virtual IO vga-compatible stereoscopic display
with integral headphones and orientation tracking, and an optical
target (figure 5). Position tracking is provided by a ______,
which tracks the head-mounted optical target. The interface also
includes a ________________ holstered bar code reader.
The spaceframe is assembled one component (strut or node) at a time. For each step of construction, the augmented reality system:
{Paragraph on system's hardware and
software feature sets here. [See the paragraph on the architectural
anatomy system. In my humble opinion it would also help to give
a brief accounting of the computer(s) and OS that the system runs
on, and the languages used to code everything - this info could
also be added to the description of the Architectural anatomy
system]. }
We are currently developing a flowchart
of spaceframe assembly steps and worker queries, which we will
use as the basis for a rule-based set of assembly instructions.
The rule-based system will also include context-sensative help,
and operate in "beginner" and "expert" modes
-- accommodating users with varying levels of experience. We
also plan to incorporate a tracking system that will track each
spaceframe component. This will allow better verification of
the installation of each piece, and adherence to the proper assembly
sequence.
[I have no problem with leaving the
following outdoor stuff completely out of this paper. Please
note, however, that it was mentioned in the abstract as accepted
by the ASCE. What do you think?].
Our third augmented reality system,
when complete, will help users located at portions of Columbia's
campus to navigate outdoors, to obtain historical information
about their environment, and to see the location of buried infrastructure.
This system features the see-through interface, sound system
and orientation tracking used in the spaceframe project, but relies
on a differential global positioning system for position tracking.
To date, we have mapped the location
of tunnels underneath the Columbia campus and created 3-dimensional
computer models of them. These models will be used to create
an augmented reality showing them superposed over a user's normal
view. This x-ray vision feature will also be used to show users
the locations of campus buildings obscured from view. Historical
information about the University and its buildings will be presented
to users on a hand-held tablet sized computer. The historical
information has been compiled and organized into an HTML - format,
hypertext database (figure 6), similar to databases we have previously
created for some major American architectural works (Webster 1996).
{Paragraph on system's hardware and
software feature sets here…}
We expect to have the complete demonstration-testbed
system working by the end of March, 1996, and to report more about
it when we deliver this paper.
Conclusions
We believe that the work described in
this paper demonstrates the potential of augmented reality's
x-ray vision and instructional guidance capabilities for improving
architectural construction, inspection, and renovation. Future
versions of systems with x-ray vision capabilities may allow maintenance
workers to see and avoid hidden features such as buried infrastructure,
electrical wiring, and structural elements. This promises to
both speed up maintenance operations and to reduce the amount
of accidental damage they currently cause. Future versions of
our space-frame system may guide construction workers through
the assembly of actual buildings and help to improve the quality
of their work. Inspectors with augmented reality interfaces may
be similarly guided through their jobs - allowing them to work
without reference to construction drawings and ensuring that every
item which needs to be checked is in fact inspected.
The potential impact of augmented reality
on architecture and structural engineering will increase as the
technology is tied to other emerging technologies. For example,
the addition of knowledge based expert systems (Myers 1992) to
the core augmented reality technology described here could yield
systems capable of training workers at actual construction sites
while they work to assemble a real building. Such real-time at-site
training systems could guide even completely inexperienced users
through complex construction operations. Over time, the continued
evolution and integration of these and other technologies may
yeild systems that improve both the efficiancy and the quality
of building construction, maintainence and rennovation.
[Paragraph on caveats, need for future
work, etc.??].
References:
Bajura, M., Fuchs, H., and Ohbuchi, R.
Merging Virtual Objects with the Real World: Seeing Ultrasound
Imagery within the Patient. In Computer Graphics (Proc. SIGGRAPH '92),
26(2), July 1992, 203-210.
Caudell, T., & Mizell, D. (1992). "Augmented reality: An application of heads-up display technology to manual manufacturing processes." Proceedings of the Hawaii International Conference on Systems Science, Vol. 2, 659J669.
Das Consulting, Inc. (1993). Dast user's manual. North Andover, MA: Das Consulting, Inc.
Feiner, S., Webster, A., Krueger, T., MacIntyre, B., and Keller, E.
"Architectural Anatomy." In Presence: Teleoperators and
Virtual Environments.
Summer 1995.
Feiner, S., MacIntyre, B., and Seligmann, D. "Knowledge-based
augmented reality." Communications of the ACM, 36(7), July 1993, 52-62.
"Jobsite Forcast: the Computerized Hardhat," Engineering News Record, March 29, 1993.
Myers, J. Snyder, J., and Chirca, L. "Database Usage in a Knowedgebase Environment for Building Design," Building and Environment, 27(2), 1992, 231-241.
Novitski, B. "Virtual Reality for Architects," Architecture, October 1994.
Oloufa, Amr. "Modeling and Simulation of Construction Operations." Automation in Construction, 1(4), 1993, 351-359.
"Patents: Portable No-hands Computer for Consulting Manuals," The New York Times, May 9, 1994, D2.
Richards, M. "Automation in Construction," The Industrial Robot, 21(4), 1994, 26-28.
Robinett, W. (1992). "Synthetic experience: A taxonomy." Presence, 1(2), 229J247.
Starnet International, Inc. 200 Hope Street Longwood, Fl 32750
Sutherland, I. "A Head-Mounted Three Dimensional Display."
Proc. Fall Joint Computer Conference 1968, Thompson Books, Washington,
DC, 1968, 757-764.
"Virtual Reality, by Ratheon," Engineering News Record, May 29, 1995, 27.
Webster, A. "Networked Multimedia Tools for Architectural Engineering." Journal of Architectural Engineering. In press, January 1996.
Webster, A. Technological Advance in Japanese Building Design and Construction, ASCE Press, New York, 1994.