Undergraduate Participation in the Research of Augmented Reality

Rod Freeman and Jenny Wu
completed fall 1995


INTRODUCTION

This report has been put together in order to introduce and help explain the Undergraduate Participation in the Research of Augmented Reality. This project aims to show how students from three different fields of study can collaborate in a research group and attain set goals. The students involved are all from Columbia University: Jenny Wu is in the Architecture department, Melvin Lew is a Computer Science student, and Rod Freeman is a Mechanical Engineering student. These three students were chosen with specific goals in mind and the following pages are to give information about these goals and the process followed to achieve them.

This interdisciplinary project is being run by professors Tony Webster of the Graduate School of Architecture, Steven Feiner of the Computer Science department and Bill Massie of the Graduate School of Architecture. It's starting date was September 1995 and the set date of completion was to be May 1996. This project was formally evaluated by Vanessa Stevens of the Teachers College of Columbia University throughout its implementation. The results of the evaluation will be published in report form describing the class, assessing its effectiveness, and analyzing some of the issues raised by this method of teaching.




Figure 1

PROJECT DESCRIPTION

The project of Undergraduate Participation in Research aims to show how students from different fields of study can collaborate in research beyond the traditional classroom setting. The topic of research used in this project was the development of an augmented reality testbed for improving spaceframe construction procedures. In this project augmented reality refers to the use of a head-worn computer interface (see figure 1), featuring an audio system and a see through display which superposes visual information over a user's natural view. It compares to virtual reality in that computer developed images that appear in front of the user stay in there respective places no matter where the user moves in three-dimensional space. Augmented reality goes one step further by programming a virtual image to act as part of the natural scene already in front of the user (see figure 2). This technology can give architects an idea of the structural design behind a wall without having to rip it down. A virtual image of a vertical beam will remain in one position no matter where the user moves just like the real beam would if it could be seen.

In this project augmented reality is being used to aid a worker in the construction of a simple spaceframe design. Through the audio and visual input of the computer, the worker is guided through a step-by-step process to build a complete model (see figure 3).The steps can only be carried out in one order and if they are not, the computer will ask the worker to correct the error before moving on to the next step. The complete system works as follows:


A worker will put on the head-worn interface in the area of the construction. Directly in front of him is a single spaceframe node installed in the floor of the lab and offto the side are all the components necessary to complete the model.


The augmented reality display will show the name and location of the first spaceframe member to be installed in a color different than that of the r eal spaceframe member.


An audio signal in the headphones of the head-gear will prompt the worker to locate the correct member and install it the way it is being shown to him through the display. The audio system will remind the worker to swipe the bar-code located on the member to indicate that he has completed the task.

If the proper piece was installed the computer will move to the next step and show a knew member to be installed and again prompt the worker to find the correct piece, install it and swipe the bar code.

If at anytime an incorrect piece is installed, the computer (which will know because of an incorrect order of bar-code swiping) will inform the worker that the wrong piece was installed and that he should correct that step. The correction will be made before further construction can proceed.

This process will continue until the spaceframe construction has been completed.


The testbed system was designed, constructed, and tested by the three undergraduate students involved in the project. Each student concentrated on an aspect that related to his or her field of study. The testbed system was developed by bringing many small elements together. Of these elements the three most important were: base design and construction, computer interface development, and computer programming. In developing the testbed, the students were guided along by the professors who set up this research.





Figure 4

THE BASE DESIGN

The base design is the first necessary step in this semester long project. The space frame needs to be secured on a base in order for it to be free standing. There are three basic criterias which Rod and Jenny followed in their design. First, the base must be compact and simple, in the case that one needs to exhibit it outside the building tech lab. Second, it must be stable and be able to hold the weight of the space frame. One also must account for the event that a worker may place the space frame in the wrong order which may put extra weight on one end of the entire frame. Lastly, the space frame must be at a certain height above the ground, so that the tracking device will not lose the worker if he/she has to bend far forward to work on the space frame. Yet, the space frame cannot be too high or else the worker cannot reach the top of the space frame comfortably. Of course, this base must be built by Rod and Jenny themselves, so practicality is also essential.

With these requirements in mind, Rod and Jenny worked separately on the design. Bill Massie, professor from graduate of architecture, critiqued their designs after it was presented. Jenny's first idea consisted of four legs interlocked in the center with a screw in the middle. This was criticized for it's lack of stability. Rod's design is a tripod like base. But it was complex in design and was very difficult to built. Jenny's second design consists of a tube inserted into a plane. The tube braced the bottom node of the space frame therefore provides much stability. Taking all the positive elements of both students' preliminary designs, they came up with the final design which is a culmination of these ideas.(see figure 4) It is a tube inserted into a box with screws which protrude out of the bottom of the box to balance the base. The screws are arranged in an equilateral triangle since three points make a plane. Buck shot is place inside the box to add weight to the whole base so the space frame will not tip over. The design of the tube is successful in that the height can be easily cut to the right dimensions. Screws are also inserted into the side of the tube to hold the node of the space frame in the tube.




CONSTRUCTION OF THE BASE

This was one of the most difficult part of this process. It was decided as a group to built the base out of aluminum since aluminum is soft and much easier to use than steel. Rod and Jenny spent an entire week in The Carleton lab cutting the pieces and putting them together. The most difficult part was drilling the holes into the main tube exactly. Both students learned to use the shop more effectively! The base was finished and presented, with much success. The base held the spaceframe very steadily even without the buckshot.



THE INTERFACE DESIGN

The core of the project is the computer interface which sends both audio and visual information to the worker so that he/she will better understand the construction sequence. The students were asked to develop an interface that would most efficiently provide a novice worker with the proper instructions.

The first part of the interface design is the display which visually informs the worker which piece of the spaceframe to locate and where to install it. The cylinders and spheres, otherwise known as struts and nodes, are color coded in red and green respectively. The worker would see through the goggles either a red strut or green node indicating which memeber is next in the construction sequence. The actual text, Strut 1 (or whichever number it is), is also shown next to the piece.




Worker using barcode reader.

IDEAS FOR FUTURE INTERFACE DESIGN

Rod and Jenny came up with many more features for the interface, but due to the restricting time, the interface was reduced to the essentials. One of them is rotating arrows indicating the direction in which the worker should screw the struts and nodes. Another is an help icon on the screen which will repeat the directions or link to a help database. Also, a bar code reader which is attached to the wrist so to free the hands is brought up.


CONSTRUCTION OF THE INTERFACE MOCKUP

In order to present the interface to the research group, Jenny and Rod used a Quicktake camera and Adobe Photoshop to illustrate the interface design. Pictures of every step were taken with the Quicktake camera then downloaded into the computer. The students learned to create spheres and cylinders and color them. They also learned the different functions in Photoshop. These images were also placed on the World Wide Web. The group was able to perform a mock presentation by bringing the images on the goggles and lining them up with the built spaceframe. There was a few distortions from the cropping and resizing that was done on Photoshop, but all in all, it was fairly successful.



WORLD WIDE WEB DOCUMENTATION

The documentation on the internet of the Augmented Reality Research Project is the main reason the project was such a success. The research team used the internet in a variety ways in order to ensure the completion of a working testbed system in the least amount of time. Not only was it used to record the class's history and accomplishments, but the building science researchers also used it to convert raw data into interactive three-dimensional viewers and storyboards so that the computer science researcher could program with far less difficulty.

The assembly sequence is the main part of the internet documentation. It describes in detail what is to be seen, heard and done by the user at each step. It is broken down into three parts: the order in which the space-frame members are to be installed, the scene description, and the storyboards. The great advantage of using the world wide web as an interface is that everything can be presented in a very broad perspective and if someone wants more information about a specific step, another page with greater detail can be accessed just by clicking on that step. For example, Mel could storyboard for steps 1 through 6 to get an idea of the order of steps. If however, he wants to know the exact location of strut 2 in three-dimensional space, he can just click on step 2 and a new page listing all the vital information about step two will appear.

The storyboard is a mockup of each step from step 1 to step 17 (figure 5 shows the storyboard from step 1 to 6). The member that is to be installed at each step is red for the struts and green for the nodes. White struts and gold nodes indicate members that were installed in previous steps. All the members are oriented by a diamond skeleton. The vertices of this diamond have been assigned a number for easy reference. Each storyboard step has a detailed description about it, which is accessed on the internet by clicking on the icon. The detailed description (see figure 7) gives the specifications of every member within the system. These specifications were recorded to aid Mel in defining an exact location in three-dimensional space for each member. By defining vertices on the diamond, each strut could be located between two vertices and each node centered at a vertex. Another important use for the internet was to test the computer interface mockups. Jenny and Rod created images of just the virtual part (i.e. just the red strut and its name) of what would be seen through the goggles for every step. These images were put on the world wide web as full size pictures. The virtual image in is on the screen of the computer to which the headgear is connected. The images created were put on a computer screen via the internet and seen through the goggles. The wearer of the goggles would stand in such a position that the image of a virtual member would line up with the actual model in front of him. This rough mock-up was done to test whether the struts and nodes in the virtual image were bright enough, the right color, the right size, etc.

Every member of the research team had to be assured they were using the same data and information for each step of the project. The documentation on the world wide web enabled the team to record information they developed or discovered and to also receive this information. Without this way of communicating the development process would have taken far longer to complete.






CONCLUSION

One of the goals for this project was to introduce and give undergraduate students a better understanding of advanced research. The way this was accomplished could not have been possible through the traditional classroom setting. Student were asked to solve a problem on their own through trail and error and independent research. For example, in order to find the best possible order for the assembly of the spaceframe, it had to be built and rebuilt until a satisfactory solution was discovered.

This project prepared the students in many ways for real-world experience. The need to develop a working testbed by the end of the semester pushed the students to work at levels which they may encounter during professional work. Another good preparation was the need to work as part of a team. In this project three students from different fields of study had to produce an outcome as one unit, which is why communication was very important. Also because each member was part of a team, they had to learn skills specific to the other fields of study. For example the building science researchers had to learn computer programming and vice versa. Projects with this format should be encouraged and undergraduates with the opportunity to participate in one should not give up the chance.


This page is maintained by Rod Freeman and Jenny Wu.