Alex Kipman, technical fellow at Microsoft, who led a team through years of research and development to produce the HoloLens, recaps the development journey of the mixed reality headset in this article.

Article

The Kinect camera, which Microsoft released in 2010, brought unprecedented levels of gesture and voice interaction to video games. “When we created Kinect for Xbox 360, we aimed to build a device capable of recognizing and understanding people so that computers could operate in ways that are more human,” said Alex Kipman, technical fellow at Microsoft, who helped invent the camera.

Microsoft wanted to build on the success of the Kinect’s sensor technology by parlaying it into a radically new device—a mixed reality headset that would project computergenerated images seemingly into real space. The company turned again to Kipman, who led a team through years of research and development to produce the HoloLens, which Microsoft began selling in 2016.

“It’s a device that understands people and environments, takes input in the form of gaze, gestures, and voice, and provides output in the form of 3-D holograms and immersive spatial sound,” Kipman said.

The wearable device—which is entirely self-contained and requires no cords or cables—combines a “holographic computing” processing system for determining where 3-D graphics should appear in the user’s field of vision with a display system that projects the graphics onto a lens. As a user walks around a room or turns his head, the position or orientation of the graphical image is altered so that it appears to the user as occupying a consistent location in space.

graphic

HoloLens has emerged as the goto device for engineers to integrate mixed reality into projects. Through its initial demonstrations, Microsoft showed the headset could be used to train medical students in surgery, operate robots, or to visualize, design and inspect complex structures. HoloLens is now improving productivity and efficiency for first-line workers.

“We understood that we are on the cusp of a transition in the way humans interact with digital information. Instead of peering into screens to access the digital world, we can now bring together the physical and digital worlds, giving us entirely new capabilities,” Kipman said.

As an example of current use of the headset, Thyssenkrupp is equipping elevator service technicians with HoloLens to visualize and identify problems ahead of a job, and have remote, hands-free access to technical and expert information when onsite. The company says that has helped improve service times. Ford is also using HoloLens, enabling their design and engineering teams to visualize full-scale models in 3-D. They’ve converted processes that used to take weeks down to days, and more easily and securely share ideas across the company and consider more concepts than before. And NASA has provided HoloLens to astronauts on the International Space Station as a holographic instruction manual and a tool for interactive remote support.

Any transformative technology or device encounters challenges to mainstream adoption in its early lifetime, such as cost, size, comfort, and technical barriers. These challenges often translate to initial skepticism.

“We’ve seen this all before with the very first computers, the internet, and mobile phones,” Kipman said. “Mixed reality computing is nascent, but we’ve made these investments because we believe it is the future of computing.”