Who’s Winning in Self-Driving Car Tech

A week or so ago, I tweeted a pair of charts from a presentation I’m doing this week for a client. They showed some of the data from the California Department of Motor Vehicles on the subject of self-driving car technology. This is one of the best data sets we have on this topic so, even though it only concerns one state, it’s worth diving into a bit to see what it says about who’s winning among some of the major players.

The Data

The data I refer to is gathered by the California DMV as part of its oversight of the testing of self-driving technology in the state. California is far from the only state in which such testing is happening but it’s certainly where the tech companies in this space are doing the most testing and also one of two states I suspect are seeing the vast majority of testing of this technology in the US at present. The other state, Michigan, is obviously home to several of the big legacy carmakers and although it allows autonomous technology testing as well, it doesn’t require public reporting in quite the same way and so doesn’t provide similar data. As such, we’re left with the California DMV data as one of the best indicators of where testing on public roads has gotten to and who might be doing best in this area.

Miles Driven

The first chart shows the miles driven by self-driving cars in 2016 in California. I’m providing two versions of this chart, with the first using a linear scale and the second a logarithmic scale, for reasons that will be apparent:

Alphabet subsidiary Waymo is so clearly the leader here you need that second chart even to be able to see the rest of the set’s performance on this metric. Waymo’s self-driving cars drove over 600,000 miles in 2016, almost a hundred times more than the next highest company, GM’s Cruise Automation subsidiary, and over two hundred times more than any other company. The 2016 number didn’t come out of nowhere – Waymo drove over 400,000 miles in 2015, when most of these other companies weren’t doing any testing on California roads yet. It’s also worth noting the rest of the companies are a mix of different breeds, with several traditional automakers, a couple of component companies, and Tesla in the mix. Tesla, notably, drove just 550 miles in this autonomous mode in California last year, even though its production vehicles drove many times that number in Autopilot mode, which is better described as ADAS (Advanced Driver Assistance Systems) than autonomous technology.

Again, the major legacy carmakers are likely conducting significant testing on closed tracks and in other states and countries around the world, so we shouldn’t read these numbers as being comprehensive in this sense. I would still bet that Waymo is driving more miles than any other company by some margin.

Disengagement Data

The second data set companies are required to report to the California DMV is a metric known as “disengagements”, which refers to those times when human drivers have to take over from self-driving vehicles because they’re about to do something wrong. Eventually, in order for these cars to be ready to replace human drivers, they have to get that number down to zero. In the meantime, there’s a certain ratio of disengagements to miles driven which ends up being a decent indicator of how advanced the technology is. For consistency with the above charts, I’ve used a ratio of miles driven per disengagement so the strongest performers will show up the highest. Here’s that chart, again with both linear and logarithmic scales to account for the wide range in results:

Once again, you can easily see it’s Waymo that’s way out in front of the others in this category too. BMW’s numbers are so small on both miles driven and disengagements they’re probably worth taking with something of a pinch of salt. But Waymo’s performance is at a level that far surpasses anyone else, with roughly one disengagement per 5000 miles driven. A number of the other companies struggle to get 100 miles per disengagement and poor Bosch gets less than a single mile, though it’s likely Bosch is testing specific components in a broader system rather than an entire system.

It’s also notable that Tesla, which drives so many miles in Autopilot, seemingly without it causing any accidents, fares far worse when it comes to truly autonomous technology, managing only three miles per disengagement, with most of that likely driven on a clearly defined route when it was filming for a promotional video last year. Note, also, Uber doesn’t appear on any of these charts because it didn’t do any authorized testing on California roads last year. It has, however, begun testing self-driving cars with passengers in a couple of cities in the US and the results there haven’t been great. According to leaked data, these cars struggled to get a mile before a disengagement was necessary in a recent testing period.

All of these companies should see their technology get significantly better over time, especially if they begin to approach the scale Waymo is now operating at. Waymo itself improved from around 1000 miles per disengagement to 5000 from 2015 to 2016, suggesting that, even at very large volumes, there are still significant improvements to be made year on year. But only Uber shows signs of driving significantly higher numbers of miles in 2017 than 2016 in California. All of these companies, though, are still at the fairly early stages of being able to take this technology into a production environment – the numbers of disengagements, the conditions in which these cars perform as designed, the range of roads on which they have enough data to drive and so on, all limit the broader applicability of even the best versions of the technology. For now, though, Waymo looks to be quite a way ahead of the rest.

Published by

Jan Dawson

Jan Dawson is Founder and Chief Analyst at Jackdaw Research, a technology research and consulting firm focused on consumer technology. During his sixteen years as a technology analyst, Jan has covered everything from DSL to LTE, and from policy and regulation to smartphones and tablets. As such, he brings a unique perspective to the consumer technology space, pulling together insights on communications and content services, device hardware and software, and online services to provide big-picture market analysis and strategic advice to his clients. Jan has worked with many of the world’s largest operators, device and infrastructure vendors, online service providers and others to shape their strategies and help them understand the market. Prior to founding Jackdaw, Jan worked at Ovum for a number of years, most recently as Chief Telecoms Analyst, responsible for Ovum’s telecoms research agenda globally.

2 thoughts on “Who’s Winning in Self-Driving Car Tech”

Leave a Reply

Your email address will not be published. Required fields are marked *