ALBUQUERQUE, N.M. — A surgeon in New York who wants the opinion quickly of a specialist in Los Angeles probably would send medical MRI [magnetic resonance imaging] files as e-mail attachments or make them accessible in Internet drop zones. Unfortunately for patients on operating tables, extremely large files may take a half-hour to transmit and require a very large computer — perhaps not available — to form images from the complicated data. Additionally, each rotation of the image for better viewing can take minutes to appear.
Now, interactive remote-visualization hardware that allows doctors to view and manipulate images based on very large data sets as though standing in the same room has been developed at Sandia National Laboratories.
The tool also will work for engineers, military generals, oil exploration teams, or anyone else with a need to interact with computer-generated images from remote locations.
“The niche for this product is when the data set you’re trying to visualize is so large you can’t move it, and yet you want to be collaborative, to share it without sending copies to separate locations,” says Sandia team leader Lyndon Pierson.
Stretching video cables
The Sandia hardware, for which a patent has been applied, allows the data to be kept at the main location but sends images to locations ready to receive them. The interactivity then available is similar to two people operating a game board.
The lag time between action and visible result is under 0.1 second even though the remote computer is thousands of miles away and the data sets, huge.
“We expect our method will interest oil companies, universities, the military — anywhere people have huge quantities of visualization data to transmit and be jointly studied,” says Pierson. “Significant commercial interest [in the new device] has been demonstrated by multiple companies.”
The Sandia hardware leverages without shame the advances in 3D commercial rendering technology “in order not to re-invent the wheel,” says Sandia researcher Perry Robertson.
Graphics cards for video games have extraordinary 2-D and even 3-D rendering capabilities within the cards themselves. But images from these cards, typically fed to nearby monitors, do not solve the problem of how to plug them into a network, says Robertson.
Fortunately, the Sandia extension hardware looks electronically just like a monitor to the graphics card, says Robertson. “So, to move an image across the Internet, as a first step our device grabs the image.“
Transmitting image and responses
The patented Sandia hardware squeezes the video data flooding in at nearly 2.5 gigabits a second into a network pipe that carries less than 0.5 gigabits/sec.
“While compression is not hard, it’s hard to do fast. And it has to be interactive, which streaming video typically is not,” says Pierson.
The Sandia compression minimizes data loss to ensure image fidelity. “Users need to be sure that the things they see on the screen are real, and not some artifact of image compression,” he says.
The group knew that a hardware solution was necessary to keep up with the incoming video stream. “Without it, the receiver’s frame rate would be unacceptably slow,” says Robertson. “We wanted the user to experience sitting right at the supercomputer from thousands of miles away.”
In an attempt to reduce the need for additional hardware,” says John Eldridge, a Sandia researcher who wrote the software applications, “we also created software versions of the encoder and decoder units for testing purposes. However, there is only so much you can do in software at these high resolutions and frame rates.”
The custom-built apparatus has two boards — one for compression, the other for expansion. The boards use standard low-cost SDRAM memory, like that found in most PCs, for video buffers. Four reprogrammable logic chips do the main body of work. A single-board PC running Linux is used for supervisory operations. “We turned to Linux because of its networking support and ease of use,” says Ron Olsberg, a Sandia project engineer.
“We built this apparatus for very complex ASCI visualizations. If we could have bought it off the shelf, we would have,” says Robertson.
Funded by ASCI’s [Advanced Scientific Computing Initiative] Problem-Solving environment, a pair of boards cost about $25,000, but are expected to cost much less when commercially available.
A successful demonstration took place in late October between Chicago and the Amsterdam Technology Center in the Netherlands. A second demonstration occurred between Sandia locations in Albuquerque and Livermore and the show floor of the Supercomputing 2002 convention in Baltimore in November.
“Now that this technology is out there, we expect other applications will begin to take advantage of it,” says Pierson. “Their experiences and improvements will eventually feed back into US military capability.”