Her watch reads 6 p.m. Exhausted from a full day of surgery, Elizabeth Bullitt places the boxes of Chinese takeout food onto the kitchen counter. She feels the house is a mess, but there’s never time. She greets her husband and three children and opens the food containers. The family eats, catches up on the news and events of the day, and separates for reading and homework before saying good night and heading to bed.

Now, the favorite part of her workday begins. She pulls the laptop computer from its case, brews a fresh pot of coffee, and kicks off her shoes. The hard drive whirs. For the next several hours, the computer screen illuminates the office in her dark home.

Elizabeth Bullitt, brain surgeon, is writing computer code. She will write into the wee hours, night after night.

Bullitt, associate professor of neurosurgery, belongs to a group that has existed at Carolina for a quarter-century without existing at all: MIDAG, the Medical Image Display and Analysis Group.

“We’re doing stuff nobody else in the world can do,” she says. “It’s gorgeous.”

With more than 90 people from a dozen departments at Carolina and Duke teaming up on several different projects, the “group” is abstruse, but the cutting-edge medical imaging research it’s responsible for is real and razor-sharp. Bullitt, for example, has been working with computer scientists and radiation oncologists developing a three-dimensional (3D) map of the blood vessels in the brain to assist surgeons like herself.

“MIDAG doesn’t have any official existence,” says Stephen Pizer, professor of computer science and leader of the group. “MIDAG is a construction of what everybody at UNC thinks it is. We have no paper existence whatsoever. We have grants, but those grants are to investigators and to individuals. We have software that we share; we have web sites. It means we can spend our time focusing on science and medicine and not worry about writing reports.”

The first manifestation of MIDAG at UNC-CH occurred in 1974 between Pizer and a medical physicist, Eugene Johnston, and a radiologist and M.D., Ed Staab. The next year Pizer became an adjunct professor of radiology and biomedical engineering, the first in a long line of multi-disciplinary appointments resulting from the work of MIDAG.

Today, under Pizer’s guidance, this multifaceted team of scientists, engineers, mathematicians, statisticians, and physicians operates with a common goal of improving patient care. The individual and overall progress made by the group is largely thanks to the open interactive spirit of MIDAG.

The essence of MIDAG emanates from Sitterson Hall, home of computer science, where the professors and graduate students see themselves as “toolsmiths.” Pizer says that just as the toolsmith is measured by the success of the people for whom he makes the tools, the computer science personnel who work on medical imaging measure themselves by how effectively a physician can incorporate their technology into the diagnosis and treatment of patients. A major motivation, according to Pizer, also stems from developing methods for the purpose of ultimately helping patients.

“Every time we’ve had site visits for grants, the granting agencies or the site visitors have always been very impressed with the close collaboration,” Johnston says.

When radiation oncology became its own department in the early 1980s, it was a “two-for-one split for MIDAG,” Pizer says. The partnership now included the department focused on treatment, in addition to the diagnostic field of radiology. Shortly thereafter, the department received funding for a large purchase of computer equipment, and Ed Chaney, professor of radiation oncology, made the decision to buy systems compatible with those in computer science. Since then, radiation oncology has played a major role in the development of MIDAG.

Much MIDAG research in the medical school has taken flight from the computer labs in the department. One of the group’s most successful applications, Plan UNC, is a radiation oncology brainchild that is in use in hospitals around the United States and Europe. Plan UNC, developed by Chaney, Julian Rosenman, and George Sherouse, is a 3D radiation-treatment planning program that allows physicians to see everything inside a patient through virtual simulation, an essential tool when planning to attack tumors with precise beams of radiation. The software, developed with computed tomography (CT) and magnetic resonance (MR) images, is open-ended, meaning researchers using Plan UNC at other universities such as Stanford and Chicago, can modify the program for use in related applications. As a result, countless research questions have been answered, at Carolina and elsewhere, using Plan UNC as a building block.

Diagnosing and treating tumors is the primary goal for Etta Pisano, whose work with contrast enhancement of digital mammographies is one of the group’s most acclaimed projects. She received her first research grant as part of MIDAG, and she still remembers the stifling summer day in 1989 when she almost cancelled her first meeting with Pizer on account of being “gigantically pregnant” with her second child.

“To me, this is a unique kind of resource that we have here at UNC. I’m not aware of too many institutions in the country where there are no walls between the departments in terms of collaboration,” Pisano says.

Pisano, associate professor of radiology and director of breast imaging at the Lineberger Comprehensive Cancer Center, has worked with numerous researchers to create a better method of detecting female breast cancer. By processing digital images on a computer, doctors can manipulate the brightness, contrast, and other characteristics of a mammography and possibly make a more accurate diagnosis. Since breast tumors and healthy breast tissue are similar in density, up to 20 percent of breast cancers cannot be seen on conventional X-ray films.

This research, which Pisano presented to the Radiologic Society of North America’s annual meeting in Chicago in December, included dozens of faculty, staff, and students from the U.S. and Canada, but the project originated with MIDAG collaboration.

This isn’t always easy. Often, clinicians and researchers have difficulty speaking the same language. Bullitt, who met Pizer through her collaboration with Rosenman in radiation oncology, was struggling to explain her ideas about three-dimensional imaging of the blood vessels in the brain. She had to invent words to describe computer terms. Pizer hesitated. But Bullitt knew the problem could be solved and refused to quite trying.

“She pounded on my door until she got my attention,” Pizer says. “I said it’s a pretty darn hard problem. She said, `I don’t care. I want to solve it.’”

A graduate student in computer science, who already had a Ph.D. in mathematics, had been hired to develop mathematics and an algorithm essential to the research. The mathematician made little progress and eventually quit, saying the problem couldn’t be solved by anyone at Carolina and the solution was decades away. Enter Stephen Aylward, a graduate student in computer science at the time. Within a month, the basic problem had been solved and the research could proceed.

The new technology combines two existing techniques, each with its own strengths and limitations: MR imaging, which renders in three dimensions but is relatively unrefined and lacks detail, possibly missing as much as 20 percent of the aneurysms in a brain; and X-ray angiography, which involves injecting a contrast material or dye into blood vessels and then taking X-rays of those vessels, but only provides a two-dimensional image.

Almost seven years later, parts of the original research objective are nearing completion. The group has already met with commercial companies interested in marketing a subset of the technology, but it is more likely to undergo extensive clinical application at Carolina before it is made available to other hospitals.

“I couldn’t have done it without the collaboration,” Bullitt says.

The brain is also the focus of some innovative collaboration taking place between psychiatry and computer science, thanks to the arrival of Guido Gerig from Switzerland. Gerig says that Carolina and MIDAG offered the best fit for his focus, combining psychiatry and computer science to measure the shape and volume of brain structures in an attempt to determine patients’ risk factors for certain diseases. The theory is that researchers will find volume and shape changes in brain structures over time in patients with schizophrenia, neurofibromatosis, and Alzheimer’s, among other diseases. Shape and volume measurements will be combined with measurements of local brain function and with patient exams. As the field of psychiatry increasingly turns to neuroimaging-based analysis of the brain, the need for powerful image-processing tools will also increase.

“This is the link to psychiatry,” Gerig says, “to quantify structures in the brain, to measure activities in the brain, to do patient studies, and drug trials, which will all lead to a better understanding of diseases and an improved treatment of patients.”

Since he arrived in August, Gerig has set up a new image-processing lab in the psychiatry department. The new lab is small, with four workstations and one PC, but offers software and system management for doing medical-image-analysis research. Gerig has installed software developed by the MIDAG team, an image-analysis package authored by his former group at the Swiss Federal Institute of Technology, and also a software package he helped develop in collaboration with the Harvard Medical School’s Brigham and Women’s Hospital and GE Medical Systems. All the software-fittingly-is compatible.

Pizer calls Gerig “a perfect MIDAG story” because his chaired professorship is shared equally between psychiatry and computer science. Gerig is currently seeking the funding necessary for a larger image processing lab that would be open to everyone on campus and research outside of psychiatry. But that’s MIDAG, where everyone is welcome to contribute. Futuristic technologies grounded in old-fashioned teamwork.

The sun has been hiding in the hillside for hours as Bullitt marvels at a three-dimensional image of blood vessels on her computer screen. She describes them as “pretty,” a word that conveys both the beauty of the image and its potential to help human beings. “It’s all part of the same day,” she says, “but this is the stuff that really makes me happy.”

Departments in MIDAG include: at UNC-CH-computer science, surgery, radiation oncology, family medicine, dentistry, mathematics, biostatistics, and statistics; at UNC-CH and Duke-radiology, psychiatry, and biomedical engineering; at Duke-the Institute for Statistics and Decision Sciences.

25 Years Linking Pixels and People

In the early 1970s, Steve Pizer paid a visit to the radiology department at Carolina and invited them to join him in research with image restoration.

“They said, `Research? What’s that?’” Pizer says. But 25 years later, computer science and radiology are still collaborating, and so are almost a dozen other departments.

Pizer teamed with radiologist Ed Staab and medical physicist Eugene Johnston in 1974 to form the first incarnation of the Medical Image Display and Analysis Group (MIDAG). Today, some 60 doctors and professors and some 34 graduate students conduct research under the MIDAG umbrella.

One of the earliest collaborations between Johnston and Pizer focused on the investigation of the display of medical images. Diane Rogers, a Ph.D. candidate in psychology, also served on the team as it studied the computer display of images as well as the way people viewed the images. The group experimented with the effects of human perception of images using two 1,000-line computer monitors, huge prototypes provided by Kratos.

“Nowadays, it would be pretty crude,” says Johnston, a professor of radiology, about the 27-inch monitors that had less resolution than today’s low-end PC.

Images of the human body contained many differing degrees of density, so distinguishing between shades of gray on a black-and-white monitor was an important step toward more accurate patient diagnosis.

“In those days it was considered that, well, you and I can’t see more than about eight or ten shades of gray,” Johnston says. “Our experiments showed that we could come up with about eighty shades of gray-depends on how you define it.”

The collaboration mushroomed. In 1975, Pizer, professor of computer science, received appointments in radiology and biomedical engineering, the first of numerous joint appointments produced by MIDAG research.

“That was sort of a formal recognition of the interdisciplinary relation, and essentially what happened was that things developed naturally without fanfare,” Pizer says of MIDAG’s evolution.

Pizer’s interest in the medical applications of computer science dates back to his days as a graduate student and a summer job in the medical physics department at Massachusetts General in 1962. He completed the master’s and doctorate programs in computer science in Harvard, eventually receiving his Ph.D. in 1967 after successfully defending his dissertation on medical imaging. He joined the faculty at Carolina that same year.

Staab and Johnston both came to UNC-CH from Vanderbilt, and their arrival significantly altered the environment in radiology, according to Pizer. Before their arrival in 1974, Pizer had found more interest in his ideas on medical imaging outside of UNC-CH. Consequently, he and Henry Fuchs, professor of computer science, began working with Ralph Heinz, a radiologist at Duke University, and there remains a “small but important fraction” of MIDAG at Duke today, Pizer says.

Although he denies ever conceiving the present scope of MIDAG during the early days, Pizer’s vision and spirit for collaboration have guided the group’s mission from black-and-white experiments in the 1970s to today’s three-dimensional, digital images, augmented reality, and much more.

“From the medical side of things,” Johnston says, “to have such an outstanding computer science department to put their expertise to solving real-life problems, that’s a real advantage.”