This article focuses on dynamics and control of connected and automated vehicles. The complexity and difficulty can grow significantly from low automation levels to higher levels. The paper briefly highlights three challenges, i.e., sensing, localization, and perception. The Mobility Transformation Center (MTC) is a public/private research and development partnership led by the University of Michigan. MTC aims to develop the foundations for a viable ecosystem of CAVs. A popular alternative to test high-automation-level AVs is the Naturalistic-Field Operational Test (N-FOT). In an N-FOT, a number of equipped vehicles are tested under naturalistic driving conditions over an extended period. In the near future, connected and automated vehicle technologies are expected to be deployed rapidly. While there has been a lot of research in, and attention to, the field of sensing, localization, and perception, this paper aims to point out a few areas related to the field of dynamics and control that are opportunities for further research.
Tomorrow's vehicles will be more electrified, connected, automated, and shared compared with vehicles today. In 2014, the University of Michigan launched the Mobility Transformation Center (MTC), a public-private partnership dedicated to study the new mobility trends. The near-term focus of the Center is to develop and deploy connected and automated vehicles (CAVs), and to study their societal impacts. Much of the content of this paper is drawn from the author's work at MTC over the last two and half years. This article is divided into three parts. First, the background and current status of connected vehicles and automated vehicles are presented. The MTC activities in these areas are then briefly described. Finally, research challenges, particularly those in the general areas of dynamics and control are highlighted to stimulate interest and thought in these areas.
Personal computers (PCs) first operated largely in isolation. They became a lot more useful and productive after they were connected through internet. Smart phones were subsequently invented, which keep people connected anytime and anywhere to each other and to the internet. The movement of Internet of Things (IOT) aims to connect all data sources and users to make them function better together. For ground vehicles, “function better together” could mean more convenient, safer, more energy efficient, less congested, etc.
Automotive companies and government transportation agencies soon realized that the “net neutrality” principle, universally embraced by internet and communication companies, may not work well for ground vehicles. Net neutrality is the principle that Internet service providers and governments should treat all data on the Internet the same, not discriminating or charging differentially . However, information related to safety (e.g., hard braking events, vehicle running red lights, etc.) should be communicated to road users nearby immediately. These communication needs should take priority over other non-critical needs, e.g., downloading a movie. Solution to this special, safety-first need, is dedicated short-range communication (DSRC). In the US, the Federal Communications Commission allocated 75MHz bandwidth near the 5.9GHz spectrum in 1999 for ground vehicle safety applications exclusively. Since then, professional societies such as SAE and IEEE have published standards to define channels, hardware specifications, communication protocols, and messaging formats and contents to ensure that DSRC devices from different OEMs and suppliers are inter-operable. Very importantly, DSRC was designed to ensure timely and lowlatency communications among hundreds of devices, which cannot be achieved by today's 4G/LTE systems.
Because Bluetooth, wifi and 4G/LTE technologies are mature, they have been deployed quickly and smoothly. However, DSRC is new and needs time to develop. While the technologies are getting mature, no production vehicle today in the US comes with DSRC. General Motors likely will be the first company to do so, with their 2017 Cadillac CTS. The US Department of Transportation has a new Federal Motor Vehicle Safety Standard (FMVSS) 150 under review. If passed, it will require DSRC on all light ground vehicles. In Japan, DSRC is already available under the name of ITS connect, using Japan's standardized ITS frequency of 760 MHz . Some other countries, e.g., China, might push for a cellphone based vehicle communication system, even though no standard has been defined.
For system and control engineers, the most significant impacts of vehicle connectivity are shown in Figure 1. The nature of control changes in three fundamental ways: (i) some of the unknown disturbance inputs become known; (ii) Instead of measuring the position of the lead vehicle, for example, its acceleration or even brake action can be known through communication. In certain cases, future information may also become available; (iii) multiple vehicles can become collaborative or coordinated. These three changes mean different control synthesis methods can be used, and better control performance is possible.
Many people use the terms “automated”, “autonomous” and “driverless” vehicles interchangeably. However, there are actually 5 levels of automated vehicles defined by SAE International . An automated vehicle (AV) is “driverless” or “autonomous” only when the vehicle (i) controls both steering and acceleration/deceleration, (ii) does not expect the human driver to monitor the driving environment, and (iii) does not rely on the human driver as the fallback for the driving task. The Autopilot system offered by Tesla Motors is a “Level-2” automated vehicle, and is not a driverless or autonomous vehicle. Many companies are already offering “Level-2” automated vehicles, or plan to do so in the near future. On the other hand, driverless vehicles are being actively pursued as the end-game by not only many automotive companies, but also outsiders such as Google, Apple, Intel, and NVidia. The google self-driving car is a “Level-4” automated vehicle and is a driverless car. When an AV can operate at highway speeds, and can operate in day/night, rain/snow, and in all driving scenarios, then it can be said to be a “Level-5” vehicle.
In general, there are five technical challenges for automated vehicles (see Figure 2). The complexity and difficulty can grow significantly from low automation levels to higher levels. For example, lane keeping assistance systems only need to focus on lane detection and keeping, and may not have full capability to detect other crash threats, e.g., a slowing lead vehicle, or a cross-path bicycle. A driverless vehicle, on the other hand, needs to detect all objects and understand the driving environment thoroughly to navigate through the traffic safely.
Since the focus of this paper is on dynamics and control, the last two technology challenges (control and validation) will be discussed later and in more depth in this paper. Here we will briefly talk about the first three challenges, i.e., sensing, localization and perception.
Sensing: Camera and radar are the two key types of sensors used in the Tesla Autopilot system. Lidar, on the other hand, is the major sensor used by the Google self-driving car project. Prices of cameras and radars have dropped to the range of hundreds of dollars while lidars are still 1-2 orders of magnitude more expensive. These three sensors all have their relative strengths and weaknesses. Lidars provide the best accuracy in detecting the position and shape of an object and can have 360-degree view, but are expensive, vulnerable to rain and snow, and might be interfered by other laser sources (including other driverless vehicles). While many believed that lidars cannot see the lanes, there were results showing the lanes can be seen by analyzing lane marker reflectivity. A radar sensor can best see through the fog, rain, or snow, and can detect the relative speeds of other vehicles accurately. However, it is not very good at detecting the size and shape of the object, thus is not very useful in understanding what it just saw. This sensing inaccuracy results in a major challenge: radars are not very good at differentiating stationary objects (e.g., a mailbox) and an obstacle with zero speed (e.g., a parked car). Radars are also pretty useless detecting the lanes. Due to these two major limitations, radars cannot be the primary sensor for autonomous driving. But radar can be a backup in inclement weather. Cameras can have adequate resolution and can see color, and stereo cameras can detect the range to the objects. Therefore, cameras have better situation awareness if the images are interpreted correctly. However, their performance is affected by light conditions and adverse weather. Since all the available sensors have limitations, there is significant effort to push down the price of lidars so that all three types of sensors are available on AVs. The test vehicle we recently acquired at the University of Michigan is equipped with 5 lidars, 5 radars and two cameras (see Figure 3), which allows researchers to study the synergy of these sensors in autonomous driving.
Localization: The concept of simultaneous localization and mapping (SLAM) was a hot topic in the robotics area -. For AVs, however, because GPS and navigation maps are already widely available, there is little need to solve the mapping problem, and GPS provides a reasonable localization measurement. However, the GPS accuracy can be poor, especially in urban environments, where fewer satellites are in direct line of sight, and significant multi-path problems sometimes exist. The SAE standard J2945/1  defines the recommended vehicle position accuracy in the horizontal direction to be 1.5 meters, so that the vehicle's lane position can be known reliably. If not corrected, standard GPS cannot achieve this accuracy requirement, mainly due to satellite and receiver clock drift, satellite orbit error, conditions in the Ionosphere and Troposphere, and multi-path. Fortunately, there are two characteristics that can be explored: (i) Slow-varyingness: many of these factors change slowly because they accumulate over time or change slowly with air temperature and season; and (ii) Commonality: they affect GPS receivers near each other largely in the same way, except the multi-path and receiver errors. Localization accuracy can be improved through on-board inertial motion sensors, high accuracy maps, and differential mode. The term differential mode means using a base station at a known location to broadcast correction signals for other GPS receivers in the neighborhood. Differential mode with coarser (code phase) correction is known as DGPS, those with real-time finer (carrier phase) correction are known as Real-Time Kinematic (RTK). Recently, time-separation signal processing techniques and choke ring antennae have been developed to reject far- and near-multipath errors, the major source of non-common position inaccuracy.
The benefits of accurate localization are two-fold: (i) It provides accurate measurement of the most important controlled output of the autonomous vehicle: its position; and (ii) If the vehicle location is accurately known and a high-definition navigation map is available, computation load in the perception phase (see next section) can be dramatically reduced. One can imagine that a future high-definition navigation map (HD map) will have not only streets and intersections, but also more details such as traffic signals, signs, and markers of major buildings, which are stationary objects that the perception software can use. As an example, it is much easier to differentiate between an obstacle in front on a flat surface and a hill using an HD map . Another example is that the HD map can be used to improve localization accuracy, especially in adverse weather. In addition, the stationary objects do not need to be the focus of the perception algorithm. The computation power can then be focused on moving objects for faster and more accurate perception.
Perception: Even though both sensing and localization problems are non-trivial, perception has been the weakest link in recent pursuit of autonomous driving. Cameras can produce millions of pixels, and lidars can generate millions of “hits” per second. The large amount of raw data then needs to be processed, clustered, and combined to form the basis for understanding the environment: the lane markers, parked and moving vehicles, pedestrians, cyclists, traffic controls (signals and cops), and special road users such as emergency vehicles. It is important to know not only their current positions, but speeds and accelerations as well. Knowing the obstacle or road user types is important because the knowledge enables using a model to better predict their behavior. In its June 2016 monthly report , Google announced significant progress in recognizing cyclists, and understanding their gestures. Cyclists are fast, agile, and vulnerable road users. They obviously should be treated differently from cars. Given the newness of this report from a top self-driving car project, and the fact that Google has not even started to deploy their fleet in cities with significant snow , it is fair to say that there is still a lot of work to do in perfecting their perception algorithms.
It is necessary to mention that there is significant interest in using the concept of “deep learning”, a fancy term for the long-standing field of machine learning, as a way to achieve model-free perception, or to combine the perception and the planning/control step into one stage. There are many deep learning approaches and algorithms -. They typically share two features: (i) multiple layers of nonlinear processing units to extract different features, and (ii) learning of feature representations in each layer and improving, e.g., through value evaluation and policy improvement through Monte Carlo tree search.
An example deep-learning-based autonomous driving system was recently demonstrated by NVIDIA . This system uses a convolutional neural network (CNN), and directly takes raw pixels from a single front-facing camera as the input and generates steering commands as the output. The results are significant in two ways: only tens of hours of driving data are used, and the system achieves perception and control simultaneously. However, there are also significant limitations: the system only achieves steering control and is far away from being a fully autonomous driving system. In addition, it is very hard to validate the performance of the black-box CNN-based driving system and practically impossible to ensure its performance in driving scenarios not used in the training data. The black-box nature of this approach could be a show-stopper for implementation on production vehicles for liability reasons. However, we do believe that CNN can play an important role in the perception process of AV, e.g., multiple CNNs can be trained very successfully for pedestrian detection, cyclist detection, gesture detection, etc.
Mobility Transformation Center
The Mobility Transformation Center (MTC) is a public/private R&D partnership led by the University of Michigan. MTC aims to develop the foundations for a viable ecosystem of CAVs. The major activities of MTC are described below.
Mcity: Mcity (see Figure 4) is a 32-acre test facility dedicated to research, development and testing of CAVs. Mcity was designed and developed by MTC, in partnership with the Michigan Department of Transportation (MDOT). The Mcity simulates a broad range of urban and suburban environments. It includes four lane-miles of roads with intersections, traffic signs and signals, simulated buildings, street lights, traffic circles, a railroad crossing, and a simulated underpass. It has been used to develop and test functions such as frontal collision warning, emergency electronic brake light, human factors, vehicle to pedestrian safety, autonomous driving, cybersecurity, etc. A notable recent test conducted at Mcity was the “Snowtonomy” system developed by Ford Motor Company . It was demonstrated that even when the road is heavily covered by snow, the AV can still drive safely using high-resolution 3D maps generated by Lidar before snow. Bicycle and pedestrian safety features have also been demonstrated at Mcity (see Figure 5).
Living laboratories: The University of Michigan Transportation Research Institute (UMTRI) was awarded the Safety Pilot Model Deployment grant  from the US Department of Transportation in 2012 to instrument 2,800 vehicles and 19 intersections with DSRC equipment. The results were used to understand real-world performance of DSRC, which in turn can be used to predict safety impacts of DSRC in reducing motor vehicle crashes. Under the support of MTC's 60 industrial members, a “connected Ann Arbor” living lab is being built, to instrument 9,000 connected vehicles. When the project is finished, roughly 10% of the vehicles in the city will be equipped with DSRC, the highest penetration-rate of DSRC vehicles anywhere in the world. In addition, 60 of the major intersections of the city will also be DSRC-equipped. The vision is that by 2018, the city of Ann Arbor will be a living laboratory, useful for the development and testing of advanced CAV functions such as pedestrian safety, eco-driving, adaptive traffic signal control, etc. The data collected from the living laboratory is being used to calibrate a macroscopic model  that can be used to accurately assess the impact of CAVs on energy consumption and congestion. We will also demonstrate how information collected from DSRC-equipped vehicles can be used to form the basis for a modern traffic control center without relying on traditional sensors such as magnetic loops, cameras and radars. This is possible because it was demonstrated  that traffic flow/congestion can be estimated accurately (within 10%) when just 5% of the vehicles’ positions and velocities are known (through DSRC).
Another living laboratory concept that just got launched is “Automated Ann Arbor”, which aims to deploy driverless vehicles in the city of Ann Arbor as a last-mile enhancement to today's public buses. This project is not only about development of artificial intelligence (AI) for AVs. In addition, the project will also identify and remove the legal, regulatory, liability and insurance roadblocks, and to learn how AVs can be smoothly integrated to become a useful element of the future mobility system.
Research Projects: To develop the talent pool and a strong pipeline for the future workforce, MTC also supports graduate students to work on long-term, pre-competitive research projects. Currently, there are 23 active MTC projects. These projects cover a wide range of CAV topics in engineering, legal issues, human factors, cybersecurity, user behavior, GPS accuracy, machine learning, image processing, accelerated evaluation, etc.
CAV Research Challenges
CAV research challenges in the areas of “Control” and “Validation” are discussed in this section.
Control: There are at least three areas related to the control for CAVs that need further research and development.
External HMI: This somewhat ambiguous term refers to the requirement that an AV should behave like a normal, undistracted human driver, and should communicate its attention and intention with people outside of the vehicle. Some people argue that AVs should (i) follow traffic laws, and (ii) should drive equally or more safely than human drivers. However, satisfying both requirements is far from enough. If an AV drives too conservatively and differently, it may encourage unsafe behavior from other drivers. The driverless cars from Google, Navya and Easymile all have a top speed around 25 miles/hour, and the cruising speed can be lower. In other words, they may be perfect for specific applications (e.g., mobility in a botanical garden or an industrial park), but are not yet ready to share the road with other vehicles in most cities today.***** In order to drive “more like a human driver”, it is important to collect large quantities of data to understand “driving etiquette”. Through the Safety Pilot Model Deployment project, more than 6 million trips, and more than 40 million miles of vehicle data have been collected, which provide vehicle position, speed, acceleration, yaw rate and brake pedal state information. A subset of the trips (400 thousand trips, 3 million miles) also have forward looking video images, headway to the lead vehicle, and lane position collected. We are working on analyzing these data to produce etiquette information for driverless vehicles for the city of Ann Arbor. In addition, mechanisms to replace eye-contact and gestures between the human driver and other road users need to be developed so that the intention of the driverless car can be communicated to people both inside and outside of the vehicle effectively and reliably. One such external HMI example was demonstrated by Mercedes-Benz F 015. Effective and standardized communication symbology must be developed to improve transparency and trust of other road users toward driverless vehicles.
Correct by construction: The concept of formal methods to ensure “correct by construction” was introduced for software development about three decades ago . Recently, this concept was applied to the synthesis of control algorithms -. The process started with clearlystated specifications followed by analysis of the behaviors of the dynamic system. If the bounds of exogenous disturbances are known, the invariant set that can be ensured by the bounded control signals can be computed, if the plant dynamics are given. There are two major reasons control algorithm development can be much more challenging than computer software: First, there can be significant exogenous disturbances. Secondly, the plant dynamics to be controlled can be highly uncertain. Nevertheless, there had been some initial successes reported in applying formal methods in the design of automated vehicles . In a recent paper , a “correct by construction” algorithm based on the barrier function concept was designed to guide a low-speed AV through a busy urban environment with seven pedestrians walking randomly, reaching the destination without any collision in 1,000 simulations. In comparison, traditional navigation methods are either too aggressive and experienced multiple collisions (e.g., potential field method ), or are too conservative and took a long time to reach the destination (e.g., the Hamilton-Jacobi method ). For safety-critical systems such as driverless cars, it is a good idea to apply formal methods for control synthesis, so that the final design has guaranteed performance and is much easier to validate.
Preview and coordinated control: When vehicles are connected, information can be communicated. CAVs can be superior to AVs that are not connected because (a) Connectivity is a better sensor. DSRC range is about 1,000 feet, much longer than those of the onboard sensors such as camera, radar or lidar. Position and velocity obtained through communication are much more accurate than those measured from the onboard sensors when the target is several hundred feet away. In addition, some of the vehicle states such as yaw rate and acceleration are much harder to measure using camera, radar or lidar, but can be easily obtained from a collaborative CV. Knowing driving conditions from 1,000 feet ahead enables preview or model predictive control for safer, smoother, and more efficient driving. It is also possible to learn what is around a corner and what is behind a bus using communication, both are scenarios challenging for onboard sensors to detect. (b) Connectivity adds an actuator. For example, an ambulance can request a traffic signal to change phase, to enable safer driving across intersections. Finally (c) Connectivity creates a collaborative traffic system. Automation makes individual vehicles smarter and safer, but connectivity can link them together to form an even safer and more efficient traffic system. A good analogy is that individual PCs can be great, but internet-connected PCs can be much better and more useful. It is also clear that CAVs can apply preview/ predictive control algorithms, can achieve collaboration among multiple vehicles, and can communicate intent and state. In comparison, AVs can only infer/guess the states of other vehicles, and has no opportunity to apply coordinated or preview controls. We have heard many argue that AVs do not need to be connected. The question is: if connected vehicle technologies are available and can enhance AV performance, why not use them?
Validation: It is necessary to thoroughly evaluate the safety of AVs before their release and deployment. Approaches for AV evaluation can be divided into three categories. Frequently, vehicles are tested through a “test matrix” approach. Low-level AVs such as automatic emergency braking (AEB) and lane departure warning are already widely available in both Europe and Asia, and their performance has been evaluated through test matrices. For driverless vehicles, this approach may not be appropriate, because the vehicle is intelligent and can be calibrated to excel in the predefined tests, while performance in other driving conditions has little assurance. In addition, driverless cars from different companies might use different sensors, with different challenging scenarios. A pre-defined test matrix may inadvertently bias against one vehicle vs. other vehicles.
A popular alternative to test high-automation-level AVs is the Naturalistic-Field Operational Test (N-FOT). In an N-FOT, a number of equipped vehicles are tested under naturalistic driving conditions over an extended period of time. An obvious problem of the N-FOT approach is the time needed. In the U.S., on average one needs to drive 530 thousand miles to experience a police-reported crash, and 100 million miles for a fatal crash. Based on the Google self-driving car project's July 2016 report , they have tested for more than 1.8 million miles, which is only 2% of the average “miles between fatal crashes”. Because of the low exposure to safety-critical events, N-FOT is unlikely to be used in government approval tests.
Researchers have also used Monte Carlo simulations - to emulate N-FOT tests by building stochastic models. Computer simulations can reduce the time and cost compared with field tests. However, low exposure to safety-critical scenarios is still an issue.
Finally, the Worst-Case Scenario Evaluation methodology  uses model-based optimization techniques to identify worst-case disturbances (e.g., lead vehicle motion) to evaluate control systems. It targets the weakness of a vehicle control system and does not consider the probability of occurrence of the worst-case scenarios. It has three major limitations: (i) high-fidelity vehicle models are needed, (ii) it is computation intensive, and (iii) the results do not relate to the risk in real-world driving.
After reviewing the literature, it becomes obvious that we do not yet have a thorough and efficient way to evaluate driverless cars (e.g., the Google car). Currently, there is no government-defined testing procedure anywhere in the world. If Google asks the National Highway Traffic Safety Administration (NHTSA) to allow it to sell cars, what should NHTSA do? Approve it without an official validation procedure (trust self-certification)? Test the vehicle for a million miles on public roads? Reject it? If the Google car is approved, and another company (with less experience) makes a similar request, what will be the decision then?
Recently, the concept of “accelerated evaluation” was developed and applied to evaluate AVs . The first step of this procedure is to collect large quantities of naturalistic driving data. The probability density function that describes the motion of the primary other vehicle (POV) is then manipulated through the importance sampling technique . The modified probability density function emphasizes the risky behavior of the POV and reduces simulation of benign scenarios. It was found that the safety benefit of AVs can be accurately estimated while the simulation time is reduced by 3-4 orders of magnitude (see example in Figure 6). While the initial results are encouraging, a lot of work needs to be done before the driverless car can be tested thoroughly and quickly. A knowledge gap exists in several areas: high-fidelity sensor models; large data requirement to build statistics and models for multiple other vehicles in a wide range of scenarios; behavior models for pedestrians and cyclists, and models for adverse weather conditions.
Vehicles over the last two decades have become noticeably more intelligent, with many driver assistance systems becoming widely available. In the near future, connected and automated vehicle technologies are expected to be deployed rapidly. While there has been a lot of research in, and attention to, the field of sensing, localization and perception, this paper aims to point out a few areas related to the field of dynamics and control that offer opportunities for further research.