BMW’s Z9 study car combines a haptic input device, on the console, with a display screen. With the long-range problem in view, BMW began speaking to the engineers at Immersion about the possibility of designing a mouse for the car. The target vehicle would be the 2001 7 series. An actual computer mouse in a car is one of those products that probably would cause a crash. According to an expert, strength of the mechatronics discipline is its notion of multivariable optimization, the idea of trying to solve the problem in the right place. Although the medical trainers for which Immersion provides tactile feedback look similar when seen from Schena’s favorite zoomed-out perspective, up close they are fundamentally distinct.
Here's the problem: With more data than ever at our disposal, it's only a matter of time before the information stored on our desktop computers jumps the interspecies barrier and begins replicating inside our cars. Nothing is going to stop it. Nothing stopped radios. Nothing stopped cell phones.
That's the thinking at BMW, anyhow. Three years ago, engineers there began realizing that the folks buying upscale BMWs--many of them fans of technology-might soon demand the same kind of access to the Web and to their contact lists that they now have on their desktop and lap top machines.
"All the functionality of the PC in your car-the notion is intriguing, but also scary," said Steve Vassallo of Immersion Corp. in San Jose, Calif. Vassallo, director of mechanical engineering, said BMW was aware that the demand for computer power from behind the wheel, while obviously a dangerous distraction, could also mean a barrage of buttons sprouting up from dashboards.
With the long-range problem in view, BMW began speaking to the engineers at Immersion about the possibility of designing a mouse for the car. The target vehicle would be the 2001 Seven series. "An actual computer mouse in a car is one of those products that probably would cause a crash," Vassallo cautioned. But a central controller-a knob, really, yet one with sophisticated switches for selecting modes could be a means of managing lists if it had sufficient range.
From this thinking emerged a simplified cursor control. It relies on tactile sensations to signal the user of discrete positions. For instance, the haptic knob can click a detent for every item on a l0-point list. Or, if the list is hundreds of items long, the device can make one sensation every tenth item, say, and another sensation for the first through ninth items within each group.
The feel of hitting a detent, though, is the particular magic of this device, Vassallo explained. "The technology is consistent with what Immersion has done in the last seven years," he said, explaining that the company started out building "flight simulators" for doctors-essentially, haptic trainers for developing that all-important surgeon's feel. Immersion makes gaming products, too: mechatronic joysticks that impart realism through touch.
The automobile mouse developed with one degree of freedom, using a relatively high-resolution sensor and encoder combined with a traditional de brush motor. "Planned cars have on the order of 30, to perhaps 50, motors in the interior to move lumbar supports, headrests, and so on," Vassallo said, "So the motor was no big surprise for BMW." The mouse uses a traditional microprocessor as well, he added.
"Where the magic happens is in the algorithms living in the microprocessor-the firmware," Vassallo continued. Host-side software in the "PC space" sends high level cOlT1l11ands to the mouse. The mouse generates the feel. "When the user turns the knob and feels a spring effect, or a detent, or a pop, or a damping effect-that's all controlled locally in the device itself," he said.
Because the device is programmable and because its underlying feel can change, the mouse interface is both flexible and scalable, Vassallo said. That gives automakers a "secret weapon" to combat the cycle of obsolescence that has desktop machines looking almost crude after a single year.
Cars are on a six- or seven-year life cycle, he said. "People who bought their cars six years ago will probably want some of the more interesting features of the new cars."
From a systems engineering perspective, Immersion's products all look very much alike, said Bruce Schena, the company's chief technology officer. "If you zoom back far enough, just about everything we do looks similar from a block diagram level," he said. Inside the blocks, things are very different, of course.
A strength of the mechatronics discipline is its notion of multivariable optimization, "the idea of trying to solve the problem in the right place," Schena said. The trained mechatronics engineer relies on knowledge from a quiver of technical specialties: mechanical and electrical engineering, software and firm ware design, microprocessor programming. Knowing what to use where keeps costs down. Added Schena: "You can look at a system as a whole, and say, 'We're ripping our hair out. It's going to be a much more expensive system if we have to solve it in hardware. Why don't we add another sensor and deal with it in firmware?'"
Indeed, that's just the kind of thing Schena and Vassallo learned to do while attending Stanford University's graduate program in mechatronics. Ed Carryer, who teaches the course that Schena and Vassallo affectionately call 218, emphasized the importance of "not getting locked into an M.E. point of view or an E.E. point of view," Schena said. Carryer taught the two to "look at things and optimize," Schena said.
Sometimes, he said, a mechatronics problem might call for using the best power amplifier possible, "the most linear black box power amp that's going to maximize performance of the system." Other times, a mechatronics designer might elect to use a cheaper power amplifier and compensate for it in firmware. "These are the kinds of trade-offs we make every day as engineers;' Schena added.
Yet, it doesn't sound as though mechatronics differs all that much from the daily regimen of practicing mechanical engineers. Depending on a person's particular specialty, would it be unusual for an M.E. to deal with mechanical systems, electrical and electronic systems, even software and firmware, as a matter of routine?
"Most mechanical engineering programs offer undergraduate M.E's the typical 'volts for dolts' courses," Vassallo said. "There is definitely a mysticism surrounding E.E. Most M.E's shake in their boots when they think about it," he said.
The program at Stanford breaks through the barrier of mystery, Vassallo said. "Because it's applied-you're making robots , and turning motors on, and making and breaking things-you break down the E.E. fear," he said. And it is that willingness to let boundaries between disciplines relax that's the hallmark of a good mechatronics designer, he added.
Another useful trait is a good mix of academic and hands on skills. "We have a fairly small team," Schena said. "Most of those people are quite capable with scope in hand or with a tap, drill, and file." As for himself, Schena said he studied mechanical engineering at MIT, then worked awhile in robotics. "You'll fin d a lot of people who are interested in robotics also interested in mechatronics," he said. "To be a specialist in robotics, you have to be a generalist in engineering to understand the system view."
Schena relates his early interest in robotics and his system- level perspective to his MIT days . "I remember a professor standing at the front of the room, saying, 'Don't be fooled; these systems are all the same.' That was a powerful thing to learn , that in modeling any system you could cook all this stuff down to a similar set using bond graph techniques and develop master level views."
Origins in Medicine
A Stanford spin-off, Immersion started in 1993. Both he and the founder, Louis Rosenberg, took course 218 in 1991, Schena said. From its beginning, the company focused on haptic aids for doctors.
For laparoscopic surgery, doctors use minimally invasive tools to enter a patient's body, Schena explained. Without a direct view of the surgical site, doctors rely on cameras to guide their way. But acquiring the hand-and-eye coordination necessary to twiddle end effectors while staring into a monitor takes lots of practice-which can get expensive when gained only on cadavers or in the pig lab. Plus, dead tissue often feels different from the living thing. "These procedures lend themselves well to simulation, since they're sort of virtual anyway," he said.
"Often these procedures use specialized tools-not blades per se, but cutters, scissors, grippers, and shavers," Schena continued. "We simulate the feel of the cutting interface, the tactile sensation of a real tool against the springiness of tissue."
A micro controller takes to the stage in all of Inul1ersion's systems, be they l11edical trainers, joysticks, or automotive mice. It plays as many as 30 sensations at once. Meanwhile, it can adjust each sensation 1,000 times a second.
Along with firmware and a transmission, sensors and actuators, the micro controller resides in the input device, where it limits the demand placed on the host computer. While some sensations depend on the high- level signal from the host computer, others, such as the simulation of a spring, respond directly to movement of the joystick or knob. The microcontroller, in this instance, increases the force the user feels as the handle moves farther off center-as the user compresses the "spring." Meanwhile, the microcontroller reports handle position to the host.
Actuators are motors. For the most part, the motors Immersion deploys in its haptic devices make few revolutions. Instead, they increase the torque on the handle or knob, responding to a conul1and for additional current by the microcontroller. This increases the resistance a user feels.
Like actuators, transmissions can vary between applications. Be they belts, cables, or gears, the transmissions link actuators and handles , delivering the feedback force from actuator to handle and on to the user. For the automobile mouse, with its single degree of freedom, the transmission turns out to be a simple shaft. For systems with several degrees of freedom, transmissions can evolve corresponding complexity.
Although the medical trainers for which Immersion provides tactile feedback look similar when seen from Schena's favorite zoomed-out perspective, up close they are fundamentally distinct. One simulates catheterization--actually, any procedure that involves inserting a needle into a patient. "That's where the haptics come in," Schena said. The trainer duplicates the feel of a needle going into a blood vessel. "You learn how hard to push. When to stop. It's a very tactile experience. If it's not done right, the 'patient' yells," he said.
Another system simulates the routing of catheters through major arteries . Placing pacemaker leads in the heart is one instance, he said. The procedure calls for the doctor to snake the leads through the vessels and into the right atrium. Sometimes, the doctor must also enter the left side of the heart. "The procedure is not done very often, and it's very tricky," Schena said. But that's where simulator practice can n1.ake perfect.
Yet a third medical trainer simulates bronchoscopy, where the surgeon runs a bronchoscope down the esophagus and into the lungs. Myriad small channels make the procedure dicey.
But the "holy grail" of medical simulation, said Schena, is when you've been diagnosed with a rare condition that requires complex treatment. "You go in for a tricky procedure, one where the doctor hasn't seen the case before," Schena suggested . "They take CT or MRI data, process it, and put it in one of these simulators so the doctor can practice on real-patient anatomy."
While the doctor is operating for real, you'll be comforted knowing she has left her spreadsheet of golf scores from last week's game safely stored away on her BMW's computer. You wouldn't want her distracted .