This article focuses on efforts by automotive manufacturers and engineering students towards developing autonomous vehicles. The Society of Automotive Engineers (SAE) has defined different levels of autonomy (SAE J3016 standard), to describe how automated a vehicle is, which have also been adopted by the US Department of Transportation. The purpose of SAE and General Motors (GM) in designing and implementing hands-on engineering design and conducting technology-focused collegiate competition with an emphasis on autonomous driving and the associated technologies´ is to provide a professional development and educational experience for undergraduate and graduate students enrolled at selected universities. SAE and its sponsors are supporting the competition with training and mentoring. Students are also learning how to work in interdisciplinary teams, which has its own issues. Different academic disciplines approach problems differently, use different techniques, and sometimes even seem to speak a different language. Another important thing about the Challenge is that it lets them see how their courses impact real engineering problems. Students taking a controls course see plenty of Laplace transforms and all sorts of plots—root-locus plots, perhaps Nyquist plots or Bode plots, time domain response plots—but they may not always realize how this links up to real life.
The idea of a car driving itself isn't really new - news stories back in the 1950's predicted that sometime soon we'd be seeing cars drive themselves. "Soon" wasn't nearly as close as they thought, though, and for a long time self-driving cars have been seen as the subject of science fiction. Many people watched the 1980's TV show "Knight Rider", and even wished we could have a car like KITT, the nearly indestructible self-aware black TransAm, but it wasn't seen as a practical possibility.
While the concept of autonomous vehicles has been around for quite some time, it's gotten a lot of new attention recently. As technology has advanced, it's started to seem feasible that we may actually get cars that can drive themselves. Just about every automaker has people working on driver assist and autonomous systems, and many students in engineering and computer science are intensely interested in working in this area. Faculty members are pursuing research in different aspects of autonomous systems, and companies in automotive-related areas are looking for ways to get into autonomous systems and trying to hire engineering and computer science graduates who are ready to work on these things. The Society of Automotive Engineers (SAE) has defined different levels of autonomy (SAE J3016 standard), to describe how automated a vehicle is, which have also been adopted by the U.S. Department of Transportation.
Multiple Levels of Vehicle Autonomy
In the SAE classification, there are six levels, starting with level 0 and going up through 5. At Level 0, there is no automation at all, and everything is done by the human driver. There may be warning systems, as we see with backup cameras and lane-departure warnings, but it's still the driver who has to take the information from the warning, consider what it's telling him or her, and do everything required to drive the car.
The next two levels begin to turn tasks over to the car itself, but the human driver monitors conditions to determine when the automation can be used. In Level 1, there are driver assistance technologies present; steering and accelerating or decelerating could be done at times by the system, but the human has to monitor the environment and ultimately is responsible for every task. In Level 2, partial automation, the automated system does the steering and accelerating/ decelerating, but the human driver has to monitor conditions and take control when necessary. In Level 3 (conditional automation), a transition takes place to the automated system monitoring the environment; the system does the steering and accelerating/decelerating, and it also monitors conditions to determine when to turn control back over to the human driver. The human is then responsible for taking over and driving the car. The second-highest level, Level 4, is high automation; the automated system does everything, but only in some driving modes. There are some conditions-perhaps a snowstorm, or icy roads, or reduced visibility-when the automated system isn’t able to be used. Finally, in Level 5, those restrictions are removed. The system does everything, and in all conditions. The ultimate goal is to get to Level 5. Right now, there are Level 1 and Level 2 systems, and there is debate over Level 3. Some automakers want to leapfrog over it to Level 4 and Level 5, as they consider the difficulty of making sure the handoff to the human happens properly. A common concern is that, if the automated system is working well for long periods of time, the human driver will not be attentive and not notice when the system needs him or her to take over. If there is a hazard that needs to be handled, the driver might not realize right away what it is, which could also be a problem.
Genesis of the SAE/GM Autodrive Challenge
All of this leads up to a new collegiate student competition, the SAE/ GM AutoDrive Challenge. In 2016, SAE issued a Request for Proposals for this challenge; interested universities had to prepare a proposal explaining how they would recruit and manage their team, which faculty member(s) would advise the students, and what educational and physical resources the university could commit to the effort. In April 2017, at the SAE World Congress, the eight schools chosen to participate in the Challenge were announced-Kettering University, Michigan State University, Michigan Technological University, Virginia Tech, North Carolina A&T, Texas A&M, the University of Waterloo, and the University of Toronto. The goal of each team is to reach Level 4 automation by the end of the third and final year; the car needs to be able to drive itself in a complex environment, handling all the driving tasks itself-speed control, lane keeping, switching lanes to avoid hitting obstacles, and reacting appropriately to signs and signals. However, the competition won’t require this to happen in any and all driving conditions. For example, there won’t be a fog machine there to artificially reduce visibility, or icy roads to cope with, or snow machines creating artificial blizzards.
Creating a new collegiate competition is a major undertaking, which requires a strong motivation on the part of the organizers. According to SAE, the purpose of SAE International (SAE) and General Motors (GM) designing and implementing this “hands-on engineering design and technology focused collegiate competition with emphasis on autonomous driving and the associated technologies” is to provide “a professional development and educational experience for undergraduate and graduate students enrolled at selected universities.” They state that the goals of the competition are:
Providing a hands on engineering collegiate competition for university students to demonstrate a wide range of exciting and challenging opportunities in the rapidly expanding held of engineering systems for automated driving.
Building and implementing a STEM compatible competition that facilitates university student teams to demonstrate full autonomous driving through a phased approach including areas such as object detection (perception), categorization (classification) and full autonomous operation.
According to SAE, one of the “coolest” elements of the Challenge is that “no competition like this exists in the autonomous space. By joining the SAE Collegiate Design Series we get to facilitate connections between industry and education by providing experiential learning for engineers.” Each of the eight teams is receiving financial support, hardware, and software products to help in the development of their autonomous vehicle. Part of the financial support is designated to pay a dedicated graduate student at each university, with the idea that this will be that student’s graduate research project. The largest single donation item is the car itself; each team has received a Chevy Bolt from GM, which will be modified towards the goal of Level 4 autonomy by the end of the third and final year of the Challenge. Teams have also received a powerful computing platform from Intel, which will be the “brains” of their car. Other major donations that were announced early in the competition include long-range radar units from Continental and LiDAR from Velodyne. In the first year of the competition, other sponsors have signed on to provide discounted or free hardware and software, including Altair Engineering, DS Solidworks Corporation, HERE Technologies, Kistler Instrument Corporation, MSC Software Corporation, MathWorks Inc., Novatel Inc., On Semiconductor, Oxford Technical Solutions Ltd., and ZF.
In addition to all of these supplies, SAE and its sponsors are supporting the competition with training and mentoring. In the early stages of the competition, each team sent several people to a two-day workshop at SAE Headquarters in Pennsylvania covering a broad overview of the competition. Later, SAE hosted a two-day training course on Highly Automated Vehicles for representatives of each team, and GM sponsored driver’s training for the teams; each of the eight schools chose one faculty member and four students to be trained as a driver of an autonomous vehicle. On the face of it, the concept of a driver for an autonomous vehicle seems contradictory. However, it’s very important to have a trained person sitting in the driver’s seat during testing. If all goes well, he or she doesn’t have to do anything but put the vehicle in autonomous mode, and then return it to the garage after testing. If things go wrong, though, the driver needs to recognize a problem and take control, and that was the focus of the driver’s training. Intel also held a training session, where each team learned more about the computing platform, and had the opportunity for hands-on experience with some of the things they would need to do in the competition. The MathWorks assigned a mentor to each team, to help with any issues involving their software, and several of the sponsors have held webinars or offered to provide phone consultations with teams if they need advice on how to use their products.
Teams have gotten creative with their names; a few of the names chosen and shared with SAE are Bulldog Bolt (Kettering University), Prometheus Borealis (Michigan Tech), aUToronto (University of Toronto), WATonomous (University of Waterloo), A3 - Aggies' Autonomous Auto (North Carolina A&T), and Victor Tango (Virginia Tech).
Teams are establishing social media accounts to share what they're doing with the public. Four teams have publicized their Twitter accounts and five have public Facebook pages.
The Twitter accounts are:
The Facebook pages are:
First Year Challenge Events
So, with all these resources and training, what exactly do the teams have to do? SAE’s AutoDrive Challenge™ Rules Committee has set out a basic set of goals for the three years of the Challenge, with the expectation that technology will be changing over the next three years, so they will be issuing the detailed rules on a year-by-year basis. The first year will focus on architecture definition, the second on algorithm development, and the third on validating the design. Each year will have both static and dynamic events, so students will have a number of opportunities to showcase their work. Static events will include documentation, reports, and presentations-key skills for any engineer, since no matter how great the design is, if no one can understand it, maintain it, or duplicate it, there will be problems. On the dynamic side, the car will initially need to successfully complete tasks that we, as human drivers, consider to be simple-keeping to a given speed limit, staying in its lane, stopping appropriately at stop signs, and basic obstacle avoidance. (Of course, everyone with significant driving experience can think of at least a few human drivers we’ve seen who found those simple tasks to be beyond them.) In late April and early May of 2018, the teams will all gather in Yuma, Arizona for the first year’s competition, and will all see how they and their competitors are doing. The teams are from a diverse range of institutions, and are far from uniform. SAE notes that “From the number of people to their internal structure of graduate versus undergraduate students and the colleges or disciplines that they pull team members from is an interesting item to watch.”
One of the other interesting features of this competition is that, unlike so many other collegiate competitions, it focuses not on the physical car but on the control of the car. SAE and GM expect that students will learn a wide range of skills connected to the dynamic systems and controls area, with the list including:
Sensor Mounting and Wiring
Basic Controls—Brakes, Steering, Propulsion
Map Database Operation
Complex Algorithm Development
Data fusion from sensor set
The controls knowledge students need to learn and implement covers several different areas. To start with, there is classical control theory; if you know that the steering angle should be a certain value, a simple feedback loop could be used. Many automotive-focused controls faculty probably have similar examples in their classes. Of course, since all of this is being implemented on a digital computer, digital control is relevant. Discrete-event control comes into play as well. If the sensors detect a stop sign, then the car needs to change from simply maintaining its speed and staying in the lane to a state where it slows and comes to a stop. This is a lot more complex than simply “stomping on the brakes”, as the goal is to get the car to stop as close as possible to the line on the pavement without exceeding given limits on deceleration. The rules also indicate that teams should try to complete the course in the minimum possible time, so a team that decides to play it extremely safe and begin stopping long before reaching the sign may not do as well.
There are a lot of complex issues involved in achieving these goals. Some of them have to do with imperfect information; in a textbook problem, you typically have better defined problems than teams see in this Challenge. In real life, we don’t know exactly what the dynamic model of the car is, or exactly how the tires interact with the pavement. Students need to work with models that are limited in information, and they may not be accurate in all conditions. This means a controller could be designed using a model to be feasible and “pretty close”, but the final tuning of the controller can’t be accomplished purely through modeling. Other issues have to do with strategy; if you’re approaching an obstacle in the right lane of a three-lane road, there is only one choice-move to the middle lane to avoid it. But if you’re in the middle lane and see an obstacle, you could move to either the right or the left. Which do you choose? Do you always move to the right? Always to the left? Or perhaps flip a coin (or the computational equivalent, which would involve a random number generator)? In theory, if you had perfect knowledge of everything ahead, you could simulate and figure out some “optimal” choice-for example, the one that would have the fewest number of lane changes. That isn’t always possible, though, as any person who’s chosen a lane when driving on a crowded road and later cursed the choice knows. We don’t know what’s ahead in enough detail to be able to determine the optimal choices for a trip. And of course, once the overall strategy has been decided and a controller designed, there are issues of tuning gains, programming the controller, and testing it out. One of the big challenges early on in the effort is interfacing with the car-making sure that the computer gives the proper signals in a form the car can “understand”, so that it will do what’s intended. That’s one of the key things that has to be done right before any other goals can be met. As the competition goes on, no one can say for sure what the biggest hurdles will be in future years-especially since no team knows exactly what the rules will be or what tasks the car will have to execute. It’s reasonably certain, though, that in one of those future years, moving obstacles will be added to the course. What else could be present? Perhaps different types of signs that the car has to respond to, or traffic cones, or stoplights. A quick look at all the different things a typical driver encounters every day gives plenty of ideas for what might be incorporated.
Links to Undergraduate Engineering Education
Students are also learning how to work in interdisciplinary teams, which has its own issues. Different academic disciplines approach problems differently, use different techniques, and sometimes even seem to speak a different language. As students either go on to industrial careers, further study, or academia, they’ll need to work with people from different disciplines, and the Challenge gives them critical experience in doing this. Another great thing about the Challenge is that it lets them see how their courses impact real engineering problems. Students taking a controls course see plenty of Laplace transforms and all sorts of plots-root-locus plots, perhaps Nyquist plots or Bode plots, time domain response plots-but they may not always realize how this links up to real life. Lab exercises can help, but the ability to deploy a controller on a car and then watch it “do its thing” is a really powerful demonstration of the importance and applicability of the material. Classes in many different disciplines, both undergraduate and graduate, link up to the Challenge, as do automotive-focused classes and more general engineering courses. At the undergraduate level, there are ties to dynamic modeling and controls courses, mechatronics, signals and systems courses, programming of all types, signal processing, embedded systems, virtual reality, and network security. At a higher level, some of the courses that are linked are artificial intelligence, vehicle dynamics, automotive controls, and haptic systems. One of the other goals of the Challenge, which teams needed to address in their original proposals to join the Challenge, is to develop relevant classes that don’t yet exist. These could include courses in active safety, radar, LIDAR, or image processing, as well as more advanced controls courses. Students could also get independent study credit in some cases, and graduate students could easily get an entire thesis out of their work on the Challenge.
Non-engineering and non-computer science courses are also relevant. Everything that engineers and programmers do, all the products that we design and produce, exist in a larger context, and the Challenge includes this as well. All of the teams have to produce a Social Responsibility report and give a presentation on it. In this report, they need to discuss some of the issues raised by autonomous vehicles in society. These could be economic in nature, or social and ethical; so economics, philosophy, ethics, and various liberal arts courses could be relevant. Autonomous vehicles, for example, will give people who currently can’t drive control over their own mobility-the blind or vision-impaired, elderly, or people with conditions like uncontrolled epilepsy will be able to transport themselves. However, they will eliminate some jobs-cab drivers could become a thing of the past, as could truck drivers. The economics of paid parking will totally change, as paying more for close parking doesn’t really make sense when the car can drop you off and go park itself. The economics of who wants or needs to own cars, and how many per household, could change. Some people will really want or need autonomous vehicles, but others may resist them for various reasons. And, of course, there’s the famous “trolley problem” in ethics. If you’re programming a car to deal with situations where someone is going to be injured or killed, how do you decide who it will be? If the car could hit three people or ten, without any other information you would likely say to hit the smaller number of people, but there are a lot of different variations of this scenario. One of them focuses on whether to hit a pedestrian, who will be almost certain to die in some circumstances, or hit a tree and risk killing the driver. Of course, all of these types of scenarios assume that enough information is available to quickly know what the probable deaths are and to make any necessary calculations or decisions fast enough.
Over the next few years, the dynamic systems and control community is likely to hear a lot more about this particular competition. Some of the students will be joining our dynamic systems and control community, and we may very well find ourselves as passengers in cars they help to design.
About the Author
Diane L. Peters, PhD, PE, F.SWE is an Assistant Professor at Kettering University. Dr. Peters received the B. S. degree in mechanical engineering from the University of Notre Dame, South Bend, IN in 1993, the M.S. degree in mechanical engineering from the University of Illinois - Chicago, Chicago, IL in 2000, and the Ph.D. degree in mechanical engineering from the University of Michigan, Ann Arbor, MI in 2010.
From 1993 to 1995, she was an Engineer with A. B. Dick Company. She worked as a Designer/Senior Designer for Mid-West Automation Systems from 1995 to 1999, as a Project Engineer at Western Printing Machinery Company from 1999 to 2006, and as a Senior Control Systems Engineer for LMS International from 2011 to 2013. She is currently an Assistant Professor of mechanical engineering at Kettering University in Flint, MI. She has diverse research interests, including the interaction between design and control, autonomous vehicles, and the impact of work experience on graduate engineering education. She is the Faculty Advisor for the Kettering University AutoDrive team.