03 true to form
by Cate House

The FBI is onto him. And they're willing to pay big bucks for what he's got.

click to enlarge .: Lars Nyland with the DeltaSphere. Photo by Steve Exum; click to enlarge. :.

It's not just high-profile, it's high-tech. Lars Nyland, research associate professor of computer science, has developed a way to create realistic 3D models of rooms, buildings, historical landmarks, caves — just about anything tangible.

No, not virtual reality. Image-based rendering. Unlike virtual reality, in which people use software to build the models — for example, of a table or bookshelf — image-based rendering (IBR) uses photographs of real places to create computer-generated images. What Nyland has added to IBR is a way to gather accurate depth information — how far away each pixel is in the image — so that people can get a realistic experience of a place without physically being there. With help from numerous graduate students and colleagues Anselmo Lastra and Gary Bishop, Nyland created the technology — essentially a 3D scanner they named the DeltaSphere-3000 Laser 3D Scene Digitizer — using a laser rangefinder as its source.

"In virtual reality, you might see a bookshelf, but you won't see all of the different books on it with their colors and titles because nobody wants to take the time to build each book," Nyland says. "With the DeltaSphere, I can see where all of the books are, how many there are, and which books are lying on their sides."

So why are the Feds involved? For one thing, they'd like to use the technology to take 3D images of government buildings and embassies, so they'll have ready access to detailed floor plans in case of an attack or hostage situation. On a local level, fire departments, when they're not fighting fires, could collect scans of buildings such as restaurants that are likely to burn or have lots of occupants, so that when there is a fire, they'll know the location of all the walls and doors.

Nyland came up with the idea for the DeltaSphere after seeing how tedious it was for architecture students to create computer models based on measurements they'd taken of buildings. "They would go all over the world taking physical measurements and then come back and build these realistic computer models to help preserve places that were falling down or that were going to be destroyed," he says. "There was definitely a need to capture real scenes, but unfortunately no technology to support it."

So Nyland went to work building a huge cart of PCs, monitors, mirrors, and motor controllers with wires hanging everywhere. "Every time I wanted to go somewhere and scan something, it would take about forty minutes to set everything up, then another forty minutes to unplug everything and load the car," he says. "It also wasn't automatic. I was the only one who knew how to use it, and I had to sit there and watch it. If a mirror started going too fast, I had to hit the slower button."

Around the time Nyland was working out some of the bugs in his scanner, a company called 3rdTech opened on Franklin Street. Nick England, adjunct professor of computer science and founder of 3rdTech, started the company essentially as a holding company for new businesses because he was interested in taking new technology projects and turning them into products that could be used by the public. Nyland's 3D scanning machine was exactly the type of project England had in mind.

Nyland, wanting to be more than a one-man operation, decided to give 3rdTech a go. He split up his time, working part time at Carolina and part time at 3rdTech for a year and a half so he could see the project off the ground.

"During that time, there was a lot of trial and error," he says. But that was okay with Nyland because, as he puts it, "My idea of fun is taking an interesting piece of hardware and writing some software to make it do something cool."

Some "fun" projects he set up to test the scanner included mock car accidents to see how much detail he could get with each scan. He also tested the scanner out on a drainage pond to measure its volume, a deserted coal-processing plant in the middle-of-nowhere, Kentucky, and Laurel Caverns in Pennsylvania.

click to enlarge .: A grid based on the range data gathered by the DeltaSphere in Laurel Caverns. Image courtesy Lars Nyland; click to enlarge. :.

After about six months, Nyland, with help from some mechanical and electrical engineers, had transformed his huge cart into a one-and-a-half square-foot box that rests on a tripod. The streamlined version contains a laser rangefinder, a mirror, a motor, and a small computer. Another computer — a laptop attached to the scanner — runs software Nyland created to tell the scanner what to do: sweep here, take a scan, store the data, repeat.

To scan, a spinning mirror sends the laser out in a vertical plane, and each time the laser hits something, the DeltaSphere calculates how far away the object is (accurate to one centimeter). It does this 25,000 times a second, taking about 20 minutes to do one full revolution. Just after the laser measures the distances, a camera captures a full-color panoramic view.

The technology doesn't come cheap. To purchase just the scanner costs $45,000. Add the color camera, and it's $55,000. So before forking out the money, some interested parties are giving the DeltaSphere a trial run.

The New Orleans Museum of Art, for instance, is using the DeltaSphere to give visitors a 3D sense of what it is like to visit Thomas Jefferson's home of Monticello for an exhibit called Jefferson's America, Napoleon's France, in celebration of the 200th anniversary of the Louisiana Purchase. Nyland has taken several scans of the building and is now working on simplifying the data so that they're ready in April 2003 when the exhibit opens. The exhibit will run from April through August, and the museum is expecting over 400,000 visitors.

click to enlarge .: A 3D view of Thomas Jefferson's library at Monticello. The DeltaSphere acquired over 100 million range samples, along with 1,000 color photos in two evenings during the summer of 2000. The 3D measurements are colored with data from the photos. At the bottom, the color has been turned off, showing the simplified 3D model. Image courtesy Lars Nyland; click to enlarge. :.

To simplify the data, a software program collects the information from the scans and converts it from machine, or spherical, coordinates into Cartesian (X and Y) coordinates that people can understand. "The drawback to the DeltaSphere," Nyland says, "is that it takes five to ten times longer to turn the data into something that's understandable than it does to scan."

But Nyland enlisted some students to help him speed things up. Last summer, one undergraduate student, Kevin Dale, worked on a software program that weeds out extraneous details such as flat surfaces. Take a wall, for example. "What you really need to know about walls is where they end, so you can eliminate all those samples you don't need in between," Nyland says.

Another undergraduate student, Neeraj Kumar from Georgia Tech, spent his summer figuring out how far apart scans should be taken so that they're not overlapping, but also not missing anything. In other words, how much should the scanner rotate before taking another scan?

"We're trying to work it so that you never end up with these huge data files," Nyland says. Each scan requires about 200 megabytes, so a disk can fill up pretty quickly.

As he sorts out the technical difficulties, Nyland has been dreaming up all sorts of uses for the DeltaSphere. Say you want to use a real scene in a movie, but you want to have a computer-generated character walk in and make it look like he's actually walking on the floor. "You could give the computer animator a model complete with all the lighting and shadows in which to place his creation," Nyland says. "Shadows are one of the key things that give computer-generated imagery away. If they don't match, it's an instant clue."

In construction projects, the DeltaSphere would be useful to guide renovations in old buildings. After workers have torn down the old plaster, they could scan the open walls as a record of where all the studs and channels are. "You could get some information with a photograph," Nyland says, "but with a 3D model you can figure out exactly how deep a channel is or its actual spacing. It's hard to predict what you might need to know later on."

Similarly, the DeltaSphere might be helpful to airplane manufacturers. "Boeing owns more real estate than anybody else in Seattle," Nyland says. "That's because they have to store so many parts for all the old airplanes. What they'd really like to be able to do is measure all the old parts, so that when they need one, they could just build it instead of keeping them in stock."

The FBI is on to the DeltaSphere because they hope to use it to take scans of crime scenes before they are cleaned up and the evidence destroyed. With the DeltaSphere you can get a bird's eye view (or any other view) of, say, a bedroom, which would allow investigators to see the room from a point of view that's not possible with 2D images.

"The detail is incredible," Nyland says. "You can see the folds in the sheets and covers, and you can get very realistic lighting. In a virtual reality model, there might be a bed, but they wouldn't go to all the trouble of providing exact details."

If investigators want to figure out where a shooter may have been standing during the crime, they can find all of the bullet holes in the DeltaSphere image and then pull up a grid on the computer screen with exact measurements in inches. Where the lines intersect is probably where the gun was.

click to enlarge .: Top: Nyland used the DeltaSphere to scan a simulated car accident. In the bottom right corner, the grid is turned on, showing all of the triangular samples taken by the rangefinder. Solid black areas were not completely captured. Bottom: After staging a murder scene, Nyland and colleagues used the DeltaSphere to scan the room. "The detail is incredible," Nyland says. "You can see the folds in the sheets." Images courtesy Lars Nyland; click to enlarge. :.

 

While Nyland isn't the only one to come up with the idea of a 3D laser scanner — a company called Cyra Technologies has built its own version — he welcomes the competition. "It's always fun to talk about successful work," he says. "And there are pluses and minuses to both systems." Cyra's version has a longer range and higher accuracy going for it, but their system is slower, heavier, and costs three times as much.

"It would be difficult to take theirs on a plane," Nyland says. And that's important to Nyland, who took the time to make the DeltaSphere as compact as possible, so it could be taken anywhere — because he never knows when he might get the chance to go somewhere and scan something cool.

The development of the DeltaSphere was funded jointly by the National Science Foundation and the Defense Advanced Research Projects Agency. The Office of Technology Development (OTD) is the only UNC-Chapel Hill office authorized to execute license agreements with companies. For more information on reporting inventions, contact OTD at 966-3929 or visit their web site.

end of storyCate House is a freelance writer and staff writer for Carolina's Office of University Development.
email the author[Email Cate House.]

 

back to contents

 
related links:
3rdTech
new orleans museum of art
jefferson's america & napoleon's france
monticello
 
endeavors is the magazine of research and creative activity at the university of north carolina at chapel hill. copyright 2003 endeavors magazine, university of north carolina at chapel hill. no text or image may be reproduced without permission.  
winter 2003  .:  past issues  .:  browse  .:.  search  :.  discuss  :.  about us