jm + autopilot   4

NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash
This is the Tesla self-crashing car in action. Remember how it works. It visually recognizes rear ends of cars using a BW camera and Mobileye (at least in early models) vision software. It also recognizes lane lines and tries to center between them. It has a low resolution radar system which ranges moving metallic objects like cars but ignores stationary obstacles. And there are some side-mounted sonars for detecting vehicles a few meters away on the side, which are not relevant here.

The system performed as designed. The white lines of the gore (the painted wedge) leading to this very shallow off ramp become far enough apart that they look like a lane.[1] If the vehicle ever got into the gore area, it would track as if in a lane, right into the crash barrier. It won't stop for the crash barrier, because it doesn't detect stationary obstacles. Here, it sped up, because there was no longer a car ahead. Then it lane-followed right into the crash barrier.

That's the fundamental problem here. These vehicles will run into stationary obstacles at full speed with no warning or emergency braking at all. That is by design. This is not an implementation bug or sensor failure. It follows directly from the decision to ship "Autopilot" with that sensor suite and set of capabilities.
tesla  fail  safety  self-driving  autopilot  cars  driving  sonar  radar  sensors  ai 
15 days ago by jm
Tesla Autopilot mode is learning
This is really impressive, but also a little scary. Drivers driving the Tesla Model S are "phoning home" training data as they drive:
A Model S owner by the username Khatsalano kept a count of how many times he had to “rescue” (meaning taking control after an alert) his Model S while using the Autopilot on his daily commute. He counted 6 “rescues” on his first day, by the fourth day of using the system on his 23.5 miles commute, he only had to take control over once. Musk said that Model S owners could add ~1 million miles of new data every day, which is helping the company create “high precision maps”.


Wonder if the data protection/privacy implications have been considered for EU use.
autopilot  tesla  maps  mapping  training  machine-learning  eu  privacy  data-protection 
november 2015 by jm
Should Airplanes Be Flying Themselves?
Excellent Vanity Fair article on the AF447 disaster, covering pilots' team-leadership skills, Clipper Skippers, Alternate Law, and autopilot design: 'There is an old truth in aviation that the reasons you get into trouble become the reasons you don’t get out of it.'

Also interesting:

'The best pilots discard the [autopilot] automation naturally when it becomes unhelpful, and again there appear to be some cultural traits involved. Simulator studies have shown that Irish pilots, for instance, will gleefully throw away their crutches, while Asian pilots will hang on tightly. It’s obvious that the Irish are right, but in the real world Sarter’s advice is hard to sell. The automation is simply too compelling. The operational benefits outweigh the costs. The trend is toward more of it, not less. And after throwing away their crutches, many pilots today would lack the wherewithal to walk.'

(via Gavin Sheridan)
airlines  automation  flight  flying  accidents  post-mortems  af447  air-france  autopilot  alerts  pilots  team-leaders  clipper-skippers  alternate-law 
november 2014 by jm
Air France 447 Flight-Data Recorder Transcript - What Really Happened Aboard Air France 447 - Popular Mechanics
The (comp.)risks of overautomation strike again. "When trouble suddenly springs up and the computer decides that it can no longer cope—on a dark night, perhaps, in turbulence, far from land -- the humans might find themselves with a very incomplete notion of what's going on. They'll wonder: What instruments are reliable, and which can't be trusted?"
aviation  crash  flight  flying  autopilot  stalls  warnings  alarms  ui  af447  risks  automation 
december 2011 by jm

Copy this bookmark:



description:


tags: