Right now all we have is marketing. This leads many to believe that Tesla is ahead in self driving tech, for example. You may or may not perceive that as a problem.
Regardless, I agree with GP that there is a space for independent testing. It could spur development further in the right direction. There is lots of investment now, though the target varies. Developing some tests could help consumers see what types of autonomy they want to buy, and give developers a chance to succeed in different areas, earning more unique awards than "best self driving car". We haven't established a public understanding of SDC testing criteria yet. I'm not clear whether developers agree on a set of tests either.
It's an odd state of affairs. Testing ought to be the first thing software engineers design.
California DMV does testing, but the results aren't sexy or widely known. Companies participate to varying degrees depending on how they feel it would affect their brand. Uber left CA and Tesla's participation was minimal. Also, the reports are only annual which seems slow these days.
We could use something like the league of women voters did for presidential debates, or like Netflix did for machine learning. Get the SDC community to scientifically show who really has the chops, and not just rely on marketing and branding!
The California DMV doesn't do any testing, all the companies with a testing permit in California self-report.
I'm not sure how one would would even go about independent testing, now or ever. Gill Pratt, head of Toyota's SV research lab points out that to have statistically meaningful data about the safety of an autonomous vehicle, you'd need to do about 8.5 billion miles of real world testing on public roads. Of course, by the time one gets to the 8 billionth mile of testing, the early data will be irrelevant because whatever autonomous OS is being tested will have undergone considerable maturation and development in that time frame. I doubt there will ever be a huge amount of 3rd party auditing.
“You can apply for a permit to deploy when… you as a manufacturer believe the vehicles are ready to go,” says Bernard Soriano, Deputy Director of the California Department of Motor Vehicles (DMV). “The traditional system of not wanting to be sued crazy controls when manufacturers will do this.”[1]
> I'm not sure how one would would even go about independent testing, now or ever.
That's a bit short sighted isn't it? Certainly the companies themselves have internal testing in order to help them judge, for example, whether some sensor data is useful or not.
I disagree strongly that the only valid test comes from consumer-driving. Plenty of things on the market go through trials before being sold to the public, be it toys, drugs or cars.
I'm not calling for regulated testing by the government. Just something cool that can get more people, even the companies themselves, excited about self driving vehicles.
George Hotz could do it. He knows the tech and has the popularity to get people's attention. Or, Chris Urmson, any others who previously participated in DARPA, or any existing SDC company.
Right now all we have is marketing. This leads many to believe that Tesla is ahead in self driving tech, for example. You may or may not perceive that as a problem.
Regardless, I agree with GP that there is a space for independent testing. It could spur development further in the right direction. There is lots of investment now, though the target varies. Developing some tests could help consumers see what types of autonomy they want to buy, and give developers a chance to succeed in different areas, earning more unique awards than "best self driving car". We haven't established a public understanding of SDC testing criteria yet. I'm not clear whether developers agree on a set of tests either.
It's an odd state of affairs. Testing ought to be the first thing software engineers design.
California DMV does testing, but the results aren't sexy or widely known. Companies participate to varying degrees depending on how they feel it would affect their brand. Uber left CA and Tesla's participation was minimal. Also, the reports are only annual which seems slow these days.
We could use something like the league of women voters did for presidential debates, or like Netflix did for machine learning. Get the SDC community to scientifically show who really has the chops, and not just rely on marketing and branding!