A Mixed Reality Based Hybrid Swarm Control Architecture for Manned-Unmanned Teaming (MUM-T)

Recent advancements in robotics have established standard control and planning algorithms for robot localization, navigation, and manipulation, which extend the automation from skill-based to rule-based. Such automation approaches, however, are susceptible to environmental dynamics and the burden of corresponding event handling falls on the human operator. In multi-agent systems, any deviation from the otherwise inefficient one operator to one robot mapping can result in an exponential growth of system complexity, and, in the absence of some form of artificial intelligence supervisory control, the overall framework can quickly become unmanageable, counterproductive, and even hazardous. Therefore, for future manned-unmanned teaming, a knowledge-based cooperative control architecture is warranted that can process cognitive reasoning at the meta-level to autonomously carry out some or all tactical parts of the mission while maintaining constant connection with the human operator. Furthermore, in such a scenario, the human operator needs to be able to communicate with multiple robotic agents via natural language and gesture interface so that he/she can efficiently manage not just one robot but the entire swarm or at least a segment. This paper will discuss a hybrid swarm autonomy architecture to coordinate a diverse team of robots using an immersive and intuitive interface technology for cooperative control of unmanned platforms. This novel interactive interface will offer situational awareness and decision presentation capabilities. Implemented through a real time, networked, mixed reality environment, it will be designed to support rapid exploration and evaluation with the swarm as well as dynamic interaction among different human operators. One of the major objectives of this research is to reduce cognitive load on operators and enable trust among robots and humans. This paper will discuss the approach to design and evaluate a distributed trust control algorithm for high-throughput hybrid swarm autonomy, and implement it through a curated, controlled-access portal to integrate swarm algorithms and collective behavior. Major discussion points will include: customization of unmanned platforms for distributed control and sensor fusion, development and implementation of a mixed reality human robot interface portal, and incorporation of a neuro-cognitive dynamic trust controller for swarm autonomy. It is envisioned that through such interconnection between humans and robots the effectiveness of the swarm can be boosted to carry out the missions with unprecedented speed and accuracy at a fraction of the cost for complex systems. This paper presents experimental validation to the analytical models involving real and virtual platforms.