Development of an Autonomous Human Monitoring System for Preventative Safety in Sea Transportation

The purpose of this study is to develop a human monitoring system for preventative safety in sea transportation. The final goal of this research is to be able to detect even a small change in the physical condition of the crew such as inattention or lapse of concentration during a voyage and to call his/her attention to the navigation of the ship before accidents or dangerous minor incidents occur. This paper deals with autonomous facial expression monitoring system and ECG analysis using a mini physiological measuring system. Ship’s captain and/or chief officer is usually walking around in the navigation bridge so as to maintain ship’s safety. Human monitoring in sea transportation is more difficult than cases of aeroplane pilot, train or bus driver who are sitting on their own seat during operation. The authors have established an autonomous facial-expression monitoring system, named ‘KII (Kinetic Information Integrator) system.’ This system consists of multiple network cameras and Microsoft’s KINECT sensor. The KINECT sensor has an ability to detect subject’s posture and skeleton position in three dimensions. At the least two network cameras will be controlled autonomously so as to follow subject’s face from different directions even though he or she is walking around in the navigation bridge. And the rest network cameras are following his activities by zoomed out. Field experiment of the KII system has been conducted on board the training ship ‘Fukae-maru’ of Kobe University. Navigation officer’s face was followed by the system during operation, and the effectiveness and validity of the system has been demonstrated. It is thus thought to reach the stage of practical application. The heart is controlled by the autonomic nervous system, and the sympathetic nervous system and parasympathetic nervous system are well known to represent the mentally stressed or relaxed states, respectively. Heart rate variability was analysed and subject’s physiological and psychological changes during navigation were also discussed.