What “Self-Driving” Cars Will Really Look Like

Soon electronic chauffeurs will take us wherever we want to go, whenever we want, in complete safety—as long as we do not need to make any left turns across traffic. Changing road surfaces are a problem, too. So are snow and ice. It will be crucial to avoid traffic cops, crossing guards and emergency vehicles. And in an urban environment where pedestrians are likely to run out in front of the car, we should probably just walk or take the subway.

All these simple, everyday encounters for human drivers pose enormous problems for computers that will take time, money and effort to solve. Yet much of the public is becoming convinced that fully automated vehicles are just around the corner.

What created this disconnect? Part of the problem is terminology. The popular media applies the descriptors “autonomous,” “driverless” and “self-driving” indiscriminately to technologies that are very different from one another, blurring important distinctions. And the automotive industry has not helped clarify matters. Marketers working for vehicle manufacturers, equipment suppliers and technology companies carefully compose publicity materials to support a wide range of interpretations about the amount of driving their products automate. Journalists who cover the field have an incentive to adopt the most optimistic forecasts—they are simply more exciting. The result of this feedback loop is a spiral of increasingly unrealistic expectations.

On supporting science journalism

If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.

This confusion is unfortunate because automated driving is coming, and it could save lives, reduce pollution and conserve fuel. But it will not happen in quite the way you have been told.

Defining automated driving

Driving is a much more complex activity than most people appreciate. It involves a broad range of skills and actions, some of which are easier to automate than others. Maintaining speed on an open road is simple, which is why conventional cruise-control systems have been doing it automatically for decades. As technology has advanced, engineers have been able to automate additional driving subtasks. Widely available adaptive cruise-control systems now maintain proper speed and spacing behind other vehicles. Lane-keeping systems, such as those in new models from Mercedes-Benz and Infiniti, use cameras, sensors and steering control to keep a vehicle centered in its lane. Cars are pretty smart these days. Yet it is an enormous leap from such systems to fully automated driving.

Click or tap to enlarge

A five-level taxonomy defined by SAE International (formerly the Society of Automotive Engineers) is useful for clarifying our thinking about automated driving. The first three rungs on this ladder of increasing automation (excluding level zero, for no automation) are occupied by technologies that rely on humans for emergency backup. Adaptive cruise control, lane-keeping systems, and the like belong to level one. Level-two systems combine the functions of level-one technologies—the lateral and longitudinal controls of lane-keeping and adaptive cruise-control systems, for example—to automate more complex driving tasks. This is as far as commercially available vehicle automation goes today. Level-three systems would allow drivers to turn on autopilot in specific scenarios, such as freeway traffic jams.

The next two levels are profoundly different in that they operate entirely without human assistance. Level-four (high-automation) systems would handle all driving subtasks, but they would operate only in strictly defined scenarios—in enclosed parking garages, for example, or in dedicated lanes on the freeway. At the top of the ladder is level five—the fully automated car. Presumably, this is what many people have in mind when they hear someone such as Nissan CEO Carlos Ghosn confidently proclaim that automated cars will be on the road by 2020.

The truth is that no one expects level-five automation systems to be on the market by then. In all likelihood, they are a long way off. Level-three systems might be just as remote. But level four? Look for it within the next decade. To understand this confusing state of affairs, we have to talk about software.

Software Nightmare

Despite the popular perception, human drivers are remarkably capable of avoiding serious crashes. Based on the total U.S. traffic safety statistics for 2011, fatal crashes occurred about once for every 3.3 million hours of driving; crashes that resulted in injury happened approximately once for every 64,000 hours of driving. These numbers set an important safety target for automated driving systems, which should, at minimum, be no less safe than human drivers. Reaching this level of reliability will require vastly more development than automation enthusiasts want to admit.

Think about how often your laptop freezes up. If that software were responsible for driving a car, the “blue screen of death” would become more than a figure of speech. A delayed software response of as little as one tenth of a second is likely to be hazardous in traffic. Software for automated driving must therefore be designed and developed to dramatically different standards from anything currently found in consumer devices.

Achieving these standards will be profoundly difficult and require basic breakthroughs in software engineering and signal processing. Engineers need new methods for designing software that can be proved correct and safe even in complex and rapidly changing conditions. Formal methods for analyzing every possible failure mode for a piece of code before it is written exist—think of them as mathematical proofs for computer programs—but only for very simple applications. Scientists are only beginning to think about how to scale up these kinds of tests to validate the incredibly complex code required to control a fully automated vehicle.

Once that code has been written, software engineers will need new methods for debugging and verifying it. Existing methods are too cumbersome and costly for the job. To put this in perspective, consider that half of the cost of a new commercial or military aircraft goes toward software verification and validation. The software on aircraft is actually much less complex than what will be needed for automated road vehicles. An engineer can design an aircraft autopilot system knowing that it will rarely, if ever, have to deal with more than one or two other aircraft in its vicinity. It does not need to know the velocity and location of those aircraft with incredible precision, because they are far enough apart that they have time to act. Decisions must be made on the order of tens of seconds. An automated road vehicle will have to track dozens of other vehicles and obstacles and make decisions within fractions of a second. The code required will be orders of magnitude more complex than what it takes to fly an airplane.

Once the code is validated, manufacturers will need ways to “prove” the safety of a complete automated driving system to the satisfaction of company risk-management officers, insurance firms, safety advocates, regulators and, of course, potential customers. The kind of formal “acceptance tests” used today are completely impractical for this purpose. Testers would have to put hundreds of millions, if not billions, of miles on a vehicle to ensure that they have subjected it in a statistically significant way to the dangerous scenarios it will encounter when it is regularly used by thousands of customers. People have started to think about solutions to this problem—the German government and industry have launched a multimillion-dollar project with that goal—but those efforts have just begun.

The code that will control the vehicle—the brain, so to speak—is not the only thing that must be subjected to scrutiny. The sensors that provide that brain with the data it will use to make decisions must be subjected to equal scrutiny. Engineers must develop new sensor-signal processing and data-fusion algorithms that can discriminate between benign and hazardous objects in a vehicle's path with nearly zero false negatives (hazardous objects that were not identified) and extremely low false positives (benign objects that were misclassified, leading to inappropriate responses from vehicles, such as swerving or hard braking).

Engineers cannot resort to the kind of brute-force redundancy used in commercial aircraft systems to achieve these goals because an automated car is a consumer product: it must be affordable for the general public. Turning to artificial intelligence is not an obvious solution, either. Some people have suggested that machine-learning systems could enable automated driving systems to study millions of hours of driving data and then learn throughout the course of their life cycle. But machine learning introduces its own problems because it is nondeterministic. Two identical vehicles can roll off the assembly line, but after a year of encountering different traffic situations, their automation systems will behave very differently.

Level-Four Future

I used to tell people that level-five fully automated driving systems would not become feasible until after 2040. Somewhere along the way people started quoting me as saying level five would arrive in 2040. Now I say that fully automated vehicles capable of driving in every situation will not be here until 2075. Could it happen sooner than that? Certainly. But not by much.

The prospects for level-three automation are clouded, too, because of the very real problem of recapturing the attention, in an emergency, of a driver who has zoned out while watching the scenery go by or, worse, who has fallen asleep. I have heard representatives from some automakers say that this is such a hard problem that they simply will not attempt level three. Outside of traffic-jam assistants that take over in stop-and-go traffic, where speeds are so low that a worst-case collision would be a fender bender, it is conceivable that level-three automation will never happen.

And yet we will see highly automated cars soon, probably within the coming decade. Nearly every big automaker and many information technology companies are devoting serious resources to level-four automation: fully automated driving, restricted to specific environments, that does not rely on a fallible human for backup. When you limit the situations in which automated vehicle systems must operate, you greatly increase their feasibility. (Automated people movers have been operating in big airports for years—but they are on totally segregated tracks.)

In all probability, the next 10 years will bring automated valet-parking systems that will allow drivers to drop their cars at the entrance of a suitably equipped garage that excludes pedestrians and nonautomated vehicles. An onboard automation system will communicate with sensors placed throughout the garage to find out which parking spots are available and navigate to them. Because there will be no need to open the doors, parking spaces can be narrower than they are today, so more cars will be able to fit in garages in areas where space is expensive.

In urban pedestrian zones, business parks, university campuses and other places where high-speed vehicles can be excluded, low-speed passenger shuttles will operate without drivers. In such environments, limited-capability sensors should be adequate to detect pedestrians and bicyclists, and if a sensor detects a false positive and brakes unnecessarily, it will not harm anyone (although it will annoy the people in the vehicle). The CityMobil2 project of the European Commission has been demonstrating such technologies for several years, and its final demonstration is scheduled for this summer.

Segregated bus ways and truck-only lanes will soon enable commercial vehicles to operate at higher levels of automation. Physically segregating these vehicles from other users will greatly simplify threat detection and response systems. Eventually driverless trucks and buses will be able to follow a human-driven lead vehicle in fuel-saving platoons. Researchers worldwide, including the California Partners for Advanced Transportation Technology (PATH) program at the University of California, Berkeley, Japan's Energy ITS project, and the KONVOI and SARTRE projects in Europe, have already tested prototype bus- and truck-platoon systems.

Yet the most widespread implementation of level-four automation within the next decade will probably be automated freeway systems for personal passenger vehicles. These systems will permit automobiles to drive themselves under certain conditions on designated sections of freeway. The vehicles will have redundant components and subsystems so that if something goes wrong, they can “limp home” without human guidance. They will probably be restricted to fair weather on stretches of freeway that have been mapped in detail, down to the signage and lane markings. These sections of road might even have “safe harbor” locations where vehicles can go when they have problems. Most major vehicle manufacturers are hard at work developing these systems, and next year Volvo Cars plans to conduct a public field test of such capabilities with 100 prototype vehicles in Gothenburg, Sweden.

These scenarios might not sound as futuristic as having your own personal electronic chauffeur, but they have the benefit of being possible—even inevitable—and soon.