'I felt like I was in heaven,' the cop said. 'It's like instant replay in the NFL, I can tell what happened.' The engineers looked at each other like, 'Aw, crap.'
As engineers, we have an obligation to think through privacy concerns.
These (and nearly all) devices should either not retain data or should encrypt it. Also my self-driving car better not be phoning home or getting updates over the air. I've seen what happens in Fast and the Furious 8… https://fsmedia.imgix.net/19/8a/19/0d/3e81/44e8/ab9d/679cbb2...
That quote is in the context of accident reconstruction. Putting aside the question of how many accidents there will be in our future autonomous world, accidents already leak personal information, especially if they lead to court cases, and it does not seem to me that improved crash investigation need necessarily leak any more. I think this is a qualitatively different situation than tracking people's location and who they communicate with, which is definitely a privacy issue, and not justified in the context of accident investigation.
Yeah it's about an accident, but if an ordinary police officer can view this, you know more powerful agencies are going to collect all of it (incidentally).
It is the people who want to expand general surveillance who would like to blur the distinction between that and accident investigation, in a bogus attempt to justify the former, and I don't think we should go along with it.
I eat popcorn watching the double-think these days. Laws simply weren't designed for perfect enforcement. We need a new legal framework that can handle omnipresence.
Many penalties have been scaled to compensate for the expected likelihood of getting caught (so as to keep the expected value of the crime negative.) Even without the radical overhaul it sounds like you're suggesting, we'll definitely have to update at least a few things.
You can use ML to summarize the video and you can store the most interesting bits in low quality - instead of actual video you can just store something like "red car crossed intersection going northbound, male with blue cap stopped at light, ..."
The quote is in the context of accident reconstruction, but the point of the article is that we're going to be filling our streets with mobile surveillance platforms that record everything everyone does around them and police won't need warrants to access this data.
I was rear ended last week. First accident I've been in. I suddenly have a photo of a 17 year old's driver's license. Name. Date of birth. Photo. Home address. Jeez...
Aircraft, even private aircraft, are tracked and recorded. Why should automobiles be any different? Far more people are killed and injured by vehicles than are killed by aircraft. In general I'm pretty pro-privacy but I've had way too many close calls and accidents with bad drivers who were found to be at fault. Maybe there's a better way to get to vision zero, and if so I'd love to hear it.
> Aircraft, even private aircraft, are tracked and recorded. Why should automobiles be any different?
FAA and NTSB agencies aren't known for abuses of power. Local law enforcement, on the other hand, routinely abuse power (civil forfeiture, unwarranted traffic stops, racial profiling, police brutality, falsifying evidence, lying under oath, illegal search and seizure...).
Self driving cars should reduce accidents, because computers > monkey brains, so granting the police unrestricted access to data shouldn't impact safety improvements.
Couldn’t you make the same argument about total surveillance?
Or, say, forced labor for the entire population. If people experience negative feelings while imprisoned, then maybe we should work on the real problem of those feelings, not challenge the nice neutral forced labor concept.
My point is that power over ordinary people generally corrupts, and the only real solution is to reduce and distribute that power so that individuals have more autonomy. Mass surveillance from AVs is a direct threat to your human autonomy.
I'd agree with you, except that the surveillors are themselves immune to surveillance, so it's very difficult for us to ensure they do not abuse power.
Either let us watch the watchers, or don't let them watch.
> FAA and NTSB agencies aren't known for abuses of power.
That suggests one possibility would be to separate law enforcement agencies responsible for the management of automotive violations from law enforcement agencies for everything else.
Highway patrol + DMV enforcement arm for registration, safety, and moving violations. Normal police only have jurisdiction for vehicle stops for things like stolen vehicles or reckless endangerment.
Might solve a bunch of other civil liberty issues.
Cars, especially in the US, is very closely linked to the location of a person. For private aircraft that is generally not the case I assume.
I work for a semi-governmental organization in the Netherlands and we track ships. Tracking company-owned commercial ships is no problem, but we are heavily restricted by law in what we can do with the location data of privately owned ships because people live on them.
In the US, the SCOTUS has ruled differently: that attaching a GPS tracking device to a car constitutes a search under the Fourth Amendment and requires a warrant. See US v. Jones (2012) and Grady v. North Carolina (2015).
I recently heard about people being fined for sailing in a prohibited zone (I believe an exclusion zone around a drilling platform).
How does that jive with the rules against keeping tracking information of personal vessels?
The tracking you can do for personal vessels is limited, not forbidden.
Enforcing an prohibited zone, which exists for safety, probably is well within parameters. If you only track and save the vessel that are very close to your drilling platform because you can't track a persons movements from just that data.
I'd rather think of roads(and national parks, and beaches, and mountains) as "ours" not as "government property". I have the right to be there because I'm the citizen of the country.
I guess I can understand the argument but I don't see what kind of world view thinks that you can go out into the world r have privacy. Roads are what connect the vast majority of all land. It's not like you can just trespass into other people's land and expect privacy either
The world view is that what happens in public space is not private; you can't expect privacy for things you do in public.
If you go to a public concert, people can look at you, note that you were here and tell that to others. That doesn't violate privacy because you had no privacy there to begin with. And in a similar manner, if you drove last thursday on a particular road, whoever bothered to look at you and your car is allowed to do so, note that you were there, write it down and tell that to others.
Privacy is about your private stuff - what you do in your private space, what you have in your private items. If you go out in the public, the things that you do (and where, and when, and with whom) are not private anymore.
In a public space a random person might 'see' you there - but they would not know who you were and may not even remember you correctly. A machine on the other hand remembers perfectly and records constantly. As far as your car-the random observer may be hard pressed to note the fact of your license plate number while trying to remember what kind and color your car was.
I highly doubt you are pro privacy even a little bit, and it is foolish endeavour to trade liberty for promised safety..
You start off on the assumption that everyone agrees that aircraft should be tracked, and that data should be open to law enforcement review with no warrant. To the extent the data can be justified for safety it should be limited to that purpose only and barred by law from being used in any other manner. That is not how the law is today, as these records are used in all kinds of legal cases (both civil and criminal) outside of need for safety.
People that are actually pro privacy understand how these records are used and do not desire to see it expand to the tracking of every car on the road as a car is much more personally identifiable than a plane, and is in much more common usage.
in short, Maybe automobiles should not be treated differently and maybe it is time to look at how aircraft are tracked and what that tracking is used for changing the law to be more privacy focused
I donate hundreds of dollars annually to the Electronic Frontier Foundation, so I have a vested interest in privacy protection. That said I ride public transit routinely. There are cameras in the buses and trains. There are smart cards that store a record of every station and bus line I tag in and out of. There are cops with bomb and drug sniffing dogs. The point is there is a strong and historic precedent that transportation systems are monitored. There are even creepy announcements urging riders to report odd acitivty to throw authorities, of which I've been on the wrong end of more than once.
> The point is there is a strong and historic precedent that transportation systems are monitored.
You do realize there is a difference between public and private transportation, right?
I have yet to see a precedent that supports the government getting as much data as TFA talks about while using my private transportation system. Or the Supreme Court deciding that making a person wait until "cops with bomb and drug sniffing dogs" show up to a traffic stop without probable cause is an "unreasonable search". I could go on and on...
>>I donate hundreds of dollars annually to the Electronic Frontier Foundation, so I have a vested interest in privacy protection.
Do you want a gold star? that in no way proves you have a "vested interest" in privacy, that just proves you have give some money to charity.
> The point is there is a strong and historic precedent that transportation systems are monitored
Yes and people that have a "vested interest" in privacy are concerned about and have problems with the current levels of surveillance on these systems,
You may be sarcastic, but I'm sure you know that the mobile phone companies already do this, as does Google if you have Location Services on. Not sure what Apple does.
No one is asking for historical location data, but merely what the car recorded as your input and other driving parameters before, during, after a crash. Where exactly is the privacy interest in that?
I am not sure where you believe "no one" is asking for, or collecting historical data. In fact everyone from the Manufacturers, Potential Advertisers, Law Enforcement, everyone is in fact asking for and getting historical data
so where where are you getting this myth that no one is
Great. To provide that, all the car will have to do is record all telemetry and upload it to a server farm. After all, it could be months before the cops realize they need a new angle on a crash you drove past.
This would actually be reasonable, and would match up with airplanes. However we're anticipating that companies will want access to this data en masse to keep training their ML.
I wonder what the police reaction will be to the same tracking on police cars. Using their lights to get though red lights and traffic jams when there is no emergency etc will be nicely logged.
Modern police dispatch systems already do this. The car automatically sends whenever the lights or siren are used, and it is flagged at the dispatch station if the car isn't on a call that authorizes that mode. I'm fairly sure Austin PD uses this.
> I wonder what the police reaction will be to the same tracking on police cars. Using their lights to get though red lights and traffic jams when there is no emergency etc will be nicely logged.
Body camera already demonstrate that (some) police will actively disable recording devices.
Love the sentiment, but that suggests that the public has access to these tracking data. (We also need access to police emergency calls and locations, which we might already have, IDK.) I'd be quite surprised if the vehicle data would be made public.
Hell, even if specifically requested, you'd be "amazed" at how many times, if the information looks negative it will be "lost" or the camera will be "non-functional".
One department tried to claim that their cameras were "malfunctioning" in the majority of FOI / criminal cases where they were requested.
The problem is that I do want the data to be stored because it could act as training data to make the vehicles better. But I also recognize the potential for abuse.
Also there's the issue of liability. Inevitably, autonomous vehicles will be involved in accidents. If the manufacturer has telemetry, they can prove their car was not at fault if the accident was caused by something else. Without stored telemetry, the legal risks for manufacturers seem much greater.
No, I'm not willing to compromise my privacy in any way. I haven't had a ticket since l996 and refuse to install the insurance company monitoring tool.
That's one major reason companies will want all this telemetry. (And will it be streaming video over cell networks, dumping once back on home wifi?) Perhaps homomorphic encryption can help achieve both aims, or sending back aggregated data only.
Retain all sensor data forever: absolutely not, and the storage requirements would be insane anyway, especially in a harsh automotive environment (think industrial SSD pricing)
Push back e.g. updated model data to help with updating routes, real-time construction developments, new driveways, ways humans make mistakes on the road, etc? I believe that every AV company will be doing this, and hopefully this data is useless to law enforcement.
OTA updates? Tesla already does. It won’t even be a choice, and that’s probably a good thing. Many people who don’t work with computers professionally don’t see the importance of updating things. In a situation where software updates could literally prevent deaths, I’m pretty sure that it will be mandatory to have vehicles receive OTA updates that can’t be disabled.
We allow you to drive on the public roads, in exchange for you giving up your driving privacy. This is why cops can stop you and give you a breathalyzer at will, because that's the trade-off you agreed to when you got your driver's license.
If you want to encrypt your driving data, you should only drive on private roads.
I never assume that driving is a right, instead of a privilege, and the best way to maintain privacy is to just not participate in public activities in the first place..
In a practical sense, people DO have their own expectation of privacy in being "lost in the crowd" when in public places, regardless of what the legal systems project & rationalize.
Regarding public roads, we need licensing and insurance because we can severely damage other people sharing the same space, but that's not necessarily linked to consent to eliminate privacy there. We expect as reasonable for license plates to be recorded when something bad or suspicious happens, and that things like comprehensive license plate tracking is an invasion of privacy.
> I never assume that driving is a right, instead of a privilege, and the best way to maintain privacy is to just not participate in public activities in the first place..
You can extend that reasoning to public sidewalks. I mean, walking is indeed a "public activity" that takes place in a public space, so I assume that following your line of logic you'd be ok with any public authority following you or me (or anyone else for that matter) without any warrant, wouldn't you?
> We allow you to drive on the public roads, in exchange for you giving up your driving privacy. This is why cops can stop you and give you a breathalyzer at will, because that's the trade-off you agreed to when you got your driver's license.
Uh-huh...
It's been a long time so perhaps I missed the part where I signed away my constitutional protections in exchange for my privilege to operate a motorized vehicle.
Unless by "at will" you mean "under reasonable suspicion" and/or "with probable cause"?
I think there is an aspect of privacy that folks may be ignoring. Privacy from commercial entities.
Consider the following scenario:
Driver A and driver B are involved in an accident. Driver C is in an autonomous vehicle that "witnesses" the accident, by being in close proximity to the event.
If driver A or B have the same insurance company as C, that company could get the data from C to determine fault in the accident that C's vehicle witnessed... or better yet, show that both drivers were at fault and they will not pay anyone anything.
As the number of autonomous vehicles rise, the amount of data available to companies will grow, too.
I would be surprised if there isn't language in the policies being underwritten today that would make such a thing possible... possibly even mandatory.
For automobile insurance, I don't find the idea of the insurance company getting detailed information about a collision disturbing at all.
There's probably reason to make sure that policies don't contain weird provisions related to that data collection, but there isn't really a strong case for hiding driver information, there is enough variation in drivers to justify varied premiums.
I would be surprised if there isn't language in the policies being underwritten today that would make such a thing possible... possibly even mandatory.
I think I'm misunderstanding, but when you say "make such a thing possible", what exactly are you referring to? The scenarios in your post I would think are heavily dependent on the considerations agreed upon in the policy between insurer and insured, so while Driver A may have a policy that sets limits on liability, Driver B may have a completely different set of considerations in the event of vehicle collision, what Driver C's car "sees" would just be supplementary, wouldn't it?
This is an aspect of autonomous vehicles that I had never considered before.
Given each AV has complete 360 degrees camera coverage (for detection of other vehicles, if Tesla, for debugging, if others which rely primarily on Lidar) and this camera data is very valuable for debugging and as a liability reducer, this data is very probably to be kept, then transferred to a datacenter. So each AV is also a bunch of mobile security cameras.
LE officers can access those recordings with the same ease they access fixed security cameras, once an investigation starts. Very easily, even without a warrant.
One increased difficulty is the need to ask AV makers which vehicles were near the incident at the time, so as to know which AVs camera recordings to ask for. That location data is protected, at least in some US states.
I don't think that data is likely to be kept, raw or otherwise, on anything other than a very small rolling window, if even that. What the author of this article missed is the boot full of server racks that have the storage and processing abilities to even consider storing the raw data from a multitude of cameras, LIDAR and associated sensor systems.
For development, sure, in test fleets, sure, but production cars won't bother with that.
In all the vehicle recording systems I've worked with (for a local municipality's police video recording) the only video data that would be stored is video data that is pertinent.
So you wouldn't record and store an officer's complete shift. When the officer turned on his lights or siren, the device would rewind 30-45 seconds and start recording. It would then record until the lights/siren were disengaged. That is the video that would be uploaded.
So I think a more likely scenario is that the autonomous cars would in fact permanently store and share all GPS data, but data from the sensors would only store what was recorded with a reasonable buffer around a fault event such as a collision.
Storage is cheap, this data will have a value. Precedent that gets set now is very important.
Also don’t forget Google’s substantial investments in both AVs and ad-surveillance. They have a strong incentive to make cars phone home the whole time, with as much as possible. And so will their lobbyists.
Imagine a world with less centralizing force, where owners of devices are encouraged to decentralize data storage. In such a world, autonomous vehicles would ask you to specify where to store the data they collect. The owner could select to choose on-vehicle storage, vendor-provided central storage, or a data server they personally own and operate. And in all cases, given an option to encrypt using a key selected by the user/owner.
I would prefer such a world. It provides a different legal context where law enforcement would need to compel each individual owner to turn over data. Such compulsion could certainly still be realized, but it's a more significant hurdle than forcing the manufacturer rather than the user, and it's more likely to see challenges.
I'd love that, but there are three problems with it, even if you put aside the commercial benefits:
1. Means people have to build and test two systems
2. It's confusing to the normal user
3. It's better for the product if it's shared as they can iterate and improve based on the data.
This actually applies to loads of products, like voice recognition (Google has recorded and stored everything you've ever said to 'ok Google's on their servers).
So I don't think we'll see a solution until we see a massive shift in privacy becoming a key election issue. Which it probably won't for decades or until a ruthless dictator takes over a developed country (sort of happening in Turkey).
In my ideal world, data should go to the vendor by default.
As you said, it helps the product improve by feeding it more data, and it's simpler for most users.
With that being said, privacy concious people should have the ability to opt out, and either not have their data stored permanently at all, or allow them to store it on a personal server. If you chose to do that, it's on you to make sure your server is setup in a way to ingest the data, so no additional work for the vendor.
I suspect in reality most vendors would not provide that option, because data is so valuable, which is why it's important to have laws that allow us to protect it.
They won‘t „pull the data off your car“ anyway, because all the interesting data will be elsewhere.
Also considering that all the big car manufacturers are working on business models where usually driver and owner are different entities this even gets more involved.
We had an interesting case in Germany where data ended up in the court room neither the manufacturer nor the car-sharing provider claimed to have.
Well, I think the essential mistake here is that it's not your car to begin with. You may fully or partially own the hardware, but the actual autonomous vehicle core is not your property - it's only licensed to you.
A man was sentenced to 33 months in prison for vehicular manslaughter. The verdict was based on location data obtained by
the court.
Both, the manufacturer (BMW) and the provider (DriveNow), maintain that they don't collect personally identifiable location data. The location profile of the convicted man was created by the court from the data it received from both companies independently.
OT: Original source is in German and I wanted to link to the Google Translate version, like I did in the past. Seems this is not possible anymore without a Google account:-(
For the more significant stuff -- where you go and when, who you communicate with and what you communicate about -- data from autonomous vehicles probably isn't going to be the main issue.
It will generally work a lot better -- cheaper, easier, more reliable, etc. -- if they get their surveillance data from a system designed for their purposes. And, of course, they don't have to wait for the widespread adoption of autonomous vehicles to build such a system. In fact, they haven't waited.
The big question is how our norms and laws around privacy reset after the explosion of information technology has wiped away all the assumptions on which our traditional notions rested.
They’ll need a warrant to pull the data from the manufacturer. Chances are no LEA is even going to ask your permission as you won’t have the tools nor know how to pull that data.
As the article says "As such, Tesla's terms and conditions—like those of other non-automotive tech companies, including Apple, Google, and more—say that the company will hand over data to law enforcement when legally compelled to do so."
"Legally compelled to do so" means a warrant or a court order or something like that; which in my mind seems reasonable; in general, we do want courts to get evidence to determine what actually happened in (potentially fatal) accidents, instead of having to do without and have a greater chance of making a wrong judgement because they didn't have that information.
The scariest thing is that the government could just ban non-autonomous cars completely, under the guise of "safety", meaning there would be no opting out of being tracked for any long-distance travel. I've never been a fan of self-driving cars, and everything I've seen hasn't convinced me to change my opinion.
"If you outlaw freedom only outlaws will have freedom."
Even if everyone were to opt out of autonomous vehicles, law enforcement bodies would just continue deploying and upgrading their own sensor networks. Cameras that read license plates are pretty much standard equipment now; face and gait recognition can't be far behind.
Yeah, but we already knew this. Of course they will collect all the data they can all the time and store is as long as they can. What I'm more interested in is how they plan to treat the vehicles themselves. In an autonomous vehicle world, there should be no more moving violations and being pulled over a thing of the past, especially if the vehicles are not owned by humans. Or not, who knows? That's the kind of information that would be much more useful and would potentially save many more lives from ruin than the accident avoidance technology.
This is interesting. A true comprehensive rethink is required.
When you rent a safe deposit box the law doesn't get automatic access (warrant etc is required). They law could consider buying just a ride from an AV service to be "renting" the car as you would a safe deposit box, house, or apartment and require a warrant for police access.
But you'd have no authority over the cameras and other systems of all the other cars around you -- those would still be available.
I wonder whether, along with the rollout of the infrastructure needed to support mass AVs, the govt will just be adding in traffic surveillance cameras as well? It seems to me that the infrastructure to support mass AV traffic would involve lots of sensors, cameras, and other electronic devices.
> I wonder whether, along with ... mass AVs, the govt will just be adding in traffic surveillance cameras as well?
I don't wonder -- it seems quite certain this will happen. It seems whenever new systems are put in place, surveillance is built in: Automatic bridge toll payment? Could have been done with a reloadable token but no, you have to tie it to a particular vehicle and identity, and that information has been used by law enforcement. Driver's licenses? They really need only a photo and an expiration date but instead they are belarded with all sorts of information (weight, address etc) which have nothing to do with driving.
In this case it will likely be shrouded by the overstretched and abused "third party doctrine" (if a third party has the info the govt can just go see it -- so the DEA doesn't have a license plate database, it can simply contract with a third party to make one instead). The government will simply require that AV fleets provide all their telemetry data including optical camera info.
> It seems to me that the infrastructure to support mass AV traffic would involve lots of sensors, cameras, and other electronic devices.
What sort of infrastructure should be built that is specific to AVs (and why should the government, rather than the fleet operators or other third parties implement it)? Why should they need even as much as humans require today?
How/why would a production car need to process it differently and why couldn't all processing be done in real time? Safety certification means the car's functionality has to be static, until the next certified software update. Data does not need to leave the car, since the purpose of the data is for the car to drive, which is a transient state.
If you're driving a production car version 1.23, then the manufacturer (obviously) wants data from that car to be used in development of production car version 1.24.
The technology needs improvements, and the process of making those improvements is hungry for data. Every manufacturer will be in development/debug mode of their self-driving systems at least for the coming decade, so sensor data from every mile of every car is wanted and useful, and manufacturers are configuring production cars to upload as much raw sensor data as is practical.
That logic applies to every product ever made in the history of technology. Somehow their R&D departments did not need 7x24 data feeds.
Each society can decide whether to permit this particular product to collect unlimited data. The presumption is that the data belongs to no one. Let's see what happens when more than one large corporate entity lays competing claim to the same data subset, embedded by a wide-ranging data dragnet.
I do think that the logic does not apply to every product ever made, that there's a major conceptual difference between products purely designed by humans and products where key features are driven by machine learning and availability of raw data.
A 24/7 data feed from your toaster is not going to make your toaster better. It might help an R&D department identify some ways how the next toaster model should be different, but that's about it.
However, a self driving car is data-starved and is still going to be data-starved years from now. At any moment of time, your car could drive better and safer simply if it had more "experience" - the v1.24 software release can be meaningfully better than the v1.23 software release even if R&D department does nothing else but simply import the data received from millions of other cars; if your car is allowed to learn from what other cars saw.
Ensuring that the quality of driving systems increases as fast as it can is important for the society, with a major impact on injuries and casualties. I feel that it would be best for the society if we ensure that this learning (and the required data transfer) is not prohibited, as long as we can solve/restrict the potentially harmful uses of the data.
Or maybe we could save more lives by using technology to augment human drivers?
We don't actually know that self-driving cars are possible in unrestricted environments. The industry is asking for highly invasive surveillance but has no liability for failure to deliver. What happens if "more data" turns out to be insufficient? Will the next request be "restrict the environmental context" or "pass new laws to change human behavior"? Where does it end? Other industries have to deliver results before changing the market. Uber et al promised the moon to their investors, then ignored local laws, then promise to save lives, then ignore privacy concerns, then ..?
Laws take a while to catch up with new technology. This will be fixed in 8-14 years after enough people are affected by this and congress finally gets lobbied enough. Early adopters will get hit the hardest though.
Or in 8-14 years they have a critical mass of people who grew up being tracked 24/7 that they just shrug and say "1984? You mean like that weird Apple commercial from, what, the Super Bowl or something? And yes, please pass the Soma."
As engineers, we have an obligation to think through privacy concerns.
These (and nearly all) devices should either not retain data or should encrypt it. Also my self-driving car better not be phoning home or getting updates over the air. I've seen what happens in Fast and the Furious 8… https://fsmedia.imgix.net/19/8a/19/0d/3e81/44e8/ab9d/679cbb2...