A fascinating new video reveals what a Tesla in Autopilot mode ‘sees’ in real-time.
The footage shows exactly what the electric car firm’s vehicles can and can’t spot on the road while driving under the guise of Tesla’s advanced AI.
YouTube user greentheonly created the clips by overlaying key data used by the Autopilot feature onto dashcam footage.
Tesla’s Autopilot feature works using an array of cameras, radar, sonar and other sensors spread across the front, back and sides of the vehicle.
Readings from these sensors are fed into an AI that constantly calculates whether the car should stop or steer around objects, adjusting the vehicle’s path accordingly.
Circles in the illustrative video, which shows just a fraction of the computations made by Tesla’s AI at any one moment, indicate tracked objects.
These include other vehicles on the road, as well as street lamps, bollards, road signs, curbs, letter boxes and other potential hazards.
The colour of each circle shows the movement of the object: Green items are moving, while yellow are mobile hazards that have stopped.
Orange circles highlight a stationary object while red illustrates an unknown entity picked up by the Autopilot.
The size of each circle indicates the distance from that object to the car, helping the AI determine which hazards pose an immediate danger.
The larger the circle, the closer the hazard.
Circles in the illustrative video, which shows just a fraction of the computations made by Tesla’s autopilot at any one moment, indicate tracked objects. Yellow circles show mobile objects that have stopped, while orange circles show stationary objects
The colour of each circles shows the movement of the object: Green objects are those picked out by the AI as moving
Writing in the description of their YouTube clip, greentheonly explained that random circles appear in the clips when Tesla’s AI takes a false elevation reading.
‘Sometimes the circle is in place, where there’s nothing present,’ they wrote.
‘Radar have problems with determining the elevation at which the object is located, so such readout can be caused by an object that is higher or lower on the image.
‘Sometimes radar does not report elevation at all and in such cases the circle is drawn at zero elevation.’
Autopilot has been at the forefront of a number of Tesla’s major controversies, with the AI potentially at fault for a string of recent high-profile crashes.
Earlier this month, federal investigators determined that a Tesla SUV using the firm’s autonomous driving system accelerated before it collided with a California freeway divider in March.
The crash on the US 101 killed the driver, Walter Huang, 38, an Apple software engineer.
The Tesla Model X had been operating using the firm’s Autopilot software when it collided with a freeway barrier in Silicon Valley.
YouTube user greentheonly created the clips by overlaying key data used by the Autopilot feature onto dashcam footage
Ojects picked out by the Autopilot include other vehicles on the road, as well as street lamps, bollards, road signs, curbs, letter boxes and other potential hazards
The size of each circle indicates the distance from that object to the car, helping the AI determine which hazards pose an immediate danger
Writing in the description of their YouTube clip, greentheonly explained that random circles appear in the clips when Tesla’s AI takes a false elevation reading
Data showed the Model X SUV did not brake or try to steer around the barrier in the three seconds before the crash. Huang did not have his hands on the wheel.
The National Transportation Safety Board, in a preliminary report on the March 23 crash, says it now will examine the cause of the accident.
Tesla wouldn’t say if the system performed as designed.
A spokesperson referred to a company blog saying that a Tesla with Autopilot is far safer than vehicles without it.
The blog says Autopilot does not prevent all crashes but makes them less likely.
Autopilot has been at the forefront of a number of Tesla’s major controversies, with the AI potentially at fault for a string of recent high-profile crashes
Tesla has repeatedly stated that Autopilot in its current form is not for fully autonomous self driving, but rather to increase safety for the car and its passengers. Pictured is the interior of a Tesla with its Autopilot feature engaged