I am a massive fan of Mark Roeper. Unfortunately he completely f**d up this one.
He tested using Autopilot, not the latest FSD on HW4, which is worlds apart in capabilities.
It is possible that the latest FSD would also crash, but that would be a valid test of FSD capabilities.
Testing using Autopilot and calling it "FSD crashes" is a HUGE misrepresentation of facts.
I am hoping Mark will post an update to the video.
If anything is misrepresenting it's both the name autopilot and the name "full self driving" for two things that neither are autopiloting or full self driving.
Here's what the official Tesla website has to say about FSD vs. Autopilot:
> In addition to the functionality and features of Autopilot and Enhanced Autopilot, Full Self-Driving capability also includes:
> > Traffic and Stop Sign Control (Beta): Identifies stop signs and traffic lights and automatically slows your vehicle to a stop on approach, with your active supervision.
> > Upcoming: Autostreer on city streets
Since I don't see a stop sign, or a traffic light, I cannot imagine how that makes any difference or can in any way be considered a complete f*k up, or how that's a "HUGE misrepresentation of facts". These things, listed here copied verbatim from the website of the manufacturer, are completely irrelevant to what was being tested here. It's like arguing that a crash test is invalid because the crash test dummy had a red shirt instead of a yellow one.
The good old argument about the "latest" version on the "latest" hardware... As Tesla influencers have been saying for the past 5 years 'it will blow your mind'.
In the very first phrase he says "I'm in my Tesla on Autopilot"...
He's testing emergency breaking.
It's not some pay-to-receive service surely? It should be always on, regardless of Autopilot or FSD.
It has come as a default feature with many new vehicles in the modern age.
Are you trying to say that Tesla's FSD emergency breaking and AutoPilot emergency breaking are different?
It's emergency breaking. None of that should matter.
And if it does matter, we are dealing with the possibility that Tesla is selling a deliberately worse product at a lower price point, in exchange for risking the lives of drivers and passengers.
There really is no meaningful difference here, because the result SHOULD be the same, regardless of what feature was enabled or disabled.
Why would FSD be any more capable? Like why would anyone expect that? I get how this could happen, but this isn't advanced navigation it's basic collision avoidance of things in front of the car, something I'd expect autopilot to do at a bare minimum.
He didn’t completely f up, it was a valid test, and a misrepresentation of branding, not facts. The connotation of the name autopilot overlaps significantly with FSD. Nobody forced Tesla to name it that rather than enhanced cruise control or something.
Testing a wider release rather than a more exclusive one meh.
But that wall may come to you. There's a few large truck trailers I've seen completely covered on a side with a picture which could be interpreted as a road / sidewalk / smaller car / whatever.
Aside of all the other obvious reasons to not get a Tesla these days this is #1 imo. Camera feeds and a neural network are not enough for self driving, no matter how much they're training. Never ever.
At the very least they seem to have downsides that the can be easily overcame with a lidar system/combination of them. That alone is enough to help me decide when its my family that would be the passengers.
I am not claiming that Tesla FSD is at this point, but it is obviously possible to use cameras and neural networks to drive a car, because that is literally how humans drive.
Adding to the weirdness of this video, it appears Mark Rober faked his footage to make it look like he was using a Google Pixel to record screen video, but he was actually using an iPhone as can be seen in the screen reflection. And he put the "G" logo in the wrong orientation in the faked shot.
Also it's weird that he's acting like he's so special for having seen the inside of Space Mountain as if it's some kind of secret. Millions have seen it all lit up. Back when the PeopleMover/Rocket Rods attractions were running it was a common sight, as the track ran through Space Mountain and sometimes it would be under maintenance with the lights on. And of course in emergency situations they turn the lights on as well.
Another one: he claims they use thin curtains to project ghosts on in the Haunted Mansion which is true, but while he's talking about it he shows footage of a part of the ride that uses Pepper's ghost which is a completely different (and more interesting) technique. Some of the ghosts he shows while talking about it could not be achieved with the curtain method.
Come to think of it, Pepper's ghost could fool lidar. Maybe that's why he didn't talk about it even though it would have been more interesting. It would have been inconvenient for his sponsor. Someone setting up a reflective surface across a road is probably about as likely in the real world as a Wile E. Coyote-style fake road wall.
I watched the video, the Wile E. Coyote fake wall stuff is a gimmick meant to draw kids in. That, however, par for the course of his videos; they are designed to hook kids into engineering with silly things and secretly teach real engineering before getting to the punch line.
In this case the real engineering is that Tesla's choice of relying on only visual camera has fundamental issues that can not be solved with cameras only. Namely, visually blocking elements, such as heavy rain, fog, or even blinding lights just pretty much can not be solved by camera-only sensors.
(though I guess one "solution" would be for the system to say I can't see enough to drive, so I'm not going to, but then no one would buy the system)
It is also a promotion for Lidar tech, Telling future engineers to be cautious of camera vision-only driving systems. I agree with Mark. But not because of the tests he intentionally created to make Lidar look better.
I really do hope camera only tech will do better than this. But I also hope that lidar technology will eventually make it better. Right now , LIDAR needs much heavier computing power to be reasonably powerful.
Mark posted a video on X showing him getting up to speed, engaging autopilot 4 seconds before the wall, and autopilot disengaging 1 second before hitting the wall.
>Here is the raw footage of my Tesla going through the wall. Not sure why it disengages 17 frames before hitting the wall but my feet weren’t touching the brake or gas.
The playing field level here was significantly slanted comparing a production Tesla, driven by Mark, engaging 10 year old autopilot technology, against a non-production test vehicle, not driven by Mark, using I would assume the latest in LiDAR technology from Luminar.
Volvo sells the EX90 with a Luminar LiDAR sensor (not active it looks like). Why wasn't it used with Mark driving?
GM is working on it, but no indication yet when the lidwr version will be released. Currently their Super Cruise uses radar and cameras, plus pre-scanned lidar maps of roads that it compares everything against in real time.
That looked like a Lexus with its insignia blacked out, so I'm guessing it was a custom build. But if you want a car that comes with LIDAR, look at the Volvo EX90.
The test would have been more interesting if it had included a couple more or so production cars. From what I've read only a couple of cars are available with that expensive Luminar LiDAR.
I'd like to see how more ordinary automating braking systems would fare, which I think usually use radar and camera.
>The test would have been more interesting if it had included a couple more or so production cars. From what I've read only a couple of cars are available with that expensive Luminar LiDAR
There are none you can buy in the US that have LiDAR active, but certainly Luminar could have gotten one from its partner Volvo and had Mark driven that instead of the professionally driven test rig that was used.
Shouldn’t they have tested a human driver too? I have the feeling a majority of drivers would also go right through it if unaware of the setup, as it’s such an inconceivable scenario.
The whole point of driver assistance systems is to be better at this stuff than humans drivers, and most of them, with much less grandiose marketing, would have "seen" the wall in time to emergency brake.
A human would (or at least should) slow down going through the water and super dense fog that fooled the Tesla. From the shot behind I could spot the kid through the water blasts, I don't know how it looked in the car.
edit: on rewatch you can pretty clearly see the kid through the rain.
1) Your video "Can You Fool a Self-Driving Car?" uses Luminar’s latest tech but not Tesla’s latest FSD software. Why?
2) Autopilot was turned on at 42 MPH in your YouTube video but you turned it on at 39/40 MPH in your clip above. Why? Multiple takes?
3) In the clip above, Autopilot was activated MUCH closer to the wall than in the YouTube video clip. Why?
4) In your video above, you turned on Autopilot 3.8 seconds before hitting the wall, but it appears you gave Luminar a much longer head start with their tech "activated." Why? Am I wrong in my assumption?
5) Why was putting a child dummy/doll behind the wall a useful thing to do? What car would possibly see or react to a kid through a wall after crashing into that wall?
I wonder how long until techniques like Depth Anything (https://depth-anything-v2.github.io/) provide parity with human depth perception. In Mark Rober's tests, I'm not sure even a human would have passed the fog scenario, however.
I dislike the fact that Mark's videos appear to increasingly borrow from the Mr Beast style, which is very distracting. There's also the fact that half the video has nothing to do with cars in the first place.
The main result here is not surprising - Tesla's vehicles are plagued by a litany of poor engineering decisions, many at a fundamental level. Not using Lidar for depth detection is beyond stupid.
I found the disney side video extremely interesting and entertaining. It would have been nice to just pad each of them out individually with enough content to have them stand on their own while meeting the criteria he needs to appease the algorithm.
Mark posted a video on X showing him getting up to speed, engaging autopilot 4 seconds before the wall, and autopilot disengaging 1 second before hitting the wall.
> The last scenario of a Wile E. Coyote-style wall with a fake road painted on it was obviously not realistic
...shows a photorealistic road on said wall. Last I checked, human drivers didn't have organs capable of LIDAR. Most would have crashed into this ridiculous obstacle.
I am a massive fan of Mark Roeper. Unfortunately he completely f**d up this one. He tested using Autopilot, not the latest FSD on HW4, which is worlds apart in capabilities. It is possible that the latest FSD would also crash, but that would be a valid test of FSD capabilities. Testing using Autopilot and calling it "FSD crashes" is a HUGE misrepresentation of facts. I am hoping Mark will post an update to the video.
If anything is misrepresenting it's both the name autopilot and the name "full self driving" for two things that neither are autopiloting or full self driving.
Here's what the official Tesla website has to say about FSD vs. Autopilot:
> In addition to the functionality and features of Autopilot and Enhanced Autopilot, Full Self-Driving capability also includes:
> > Traffic and Stop Sign Control (Beta): Identifies stop signs and traffic lights and automatically slows your vehicle to a stop on approach, with your active supervision.
> > Upcoming: Autostreer on city streets
Since I don't see a stop sign, or a traffic light, I cannot imagine how that makes any difference or can in any way be considered a complete f*k up, or how that's a "HUGE misrepresentation of facts". These things, listed here copied verbatim from the website of the manufacturer, are completely irrelevant to what was being tested here. It's like arguing that a crash test is invalid because the crash test dummy had a red shirt instead of a yellow one.
The good old argument about the "latest" version on the "latest" hardware... As Tesla influencers have been saying for the past 5 years 'it will blow your mind'.
In the very first phrase he says "I'm in my Tesla on Autopilot"...
If he doesn't know the difference how is the average car buyer that sees Elon sell such features supposed to know?
So are you suggesting the 8k upgrade can detect some dangerous obstacles that the standard version can't? I doubt that.
It wasnt even on autopilot when he crashed it. At best, he was testing automatic collision avoidance while pressing the accelerator, not autonomy.
He's testing emergency breaking. It's not some pay-to-receive service surely? It should be always on, regardless of Autopilot or FSD.
It has come as a default feature with many new vehicles in the modern age.
Are you trying to say that Tesla's FSD emergency breaking and AutoPilot emergency breaking are different? It's emergency breaking. None of that should matter.
And if it does matter, we are dealing with the possibility that Tesla is selling a deliberately worse product at a lower price point, in exchange for risking the lives of drivers and passengers.
There really is no meaningful difference here, because the result SHOULD be the same, regardless of what feature was enabled or disabled.
Why would FSD be any more capable? Like why would anyone expect that? I get how this could happen, but this isn't advanced navigation it's basic collision avoidance of things in front of the car, something I'd expect autopilot to do at a bare minimum.
> He tested using Autopilot, not the latest FSD on HW4, which is worlds apart in capabilities
Unless HW4 adds lidar then it doesn't matter. The video shows lidar outperforming cameras.
I'm afraid HW4 is still not lidar-like.
He tested AEB, you can see autopilot is disabled right after the cut: https://youtu.be/IQJL3htsDyQ?t=942
This is faked as hell.
He didn’t completely f up, it was a valid test, and a misrepresentation of branding, not facts. The connotation of the name autopilot overlaps significantly with FSD. Nobody forced Tesla to name it that rather than enhanced cruise control or something.
Testing a wider release rather than a more exclusive one meh.
I don’t live near a lot of Wild E. Coyote fake road walls. I get the sense it’s more of a Midwest thing.
But that wall may come to you. There's a few large truck trailers I've seen completely covered on a side with a picture which could be interpreted as a road / sidewalk / smaller car / whatever.
>Midwest thing.
Southwest I think.
It failed also on kid mannequin. Do you have many kids around?
Aside of all the other obvious reasons to not get a Tesla these days this is #1 imo. Camera feeds and a neural network are not enough for self driving, no matter how much they're training. Never ever.
At the very least they seem to have downsides that the can be easily overcame with a lidar system/combination of them. That alone is enough to help me decide when its my family that would be the passengers.
Agreed, but I believe that camera feeds and infrastructure changes could be enough.
I do wonder if this is provable via information theory.
I am not claiming that Tesla FSD is at this point, but it is obviously possible to use cameras and neural networks to drive a car, because that is literally how humans drive.
You started with a bias and ended with another one.
> Camera feeds and a neural network are not enough for self driving
I guess we should ban humans driving then.
Have you used latest FSD on HW4 recently? If not, please try it out for a few days and then come back to correct your comment :-)
Adding to the weirdness of this video, it appears Mark Rober faked his footage to make it look like he was using a Google Pixel to record screen video, but he was actually using an iPhone as can be seen in the screen reflection. And he put the "G" logo in the wrong orientation in the faked shot.
Also it's weird that he's acting like he's so special for having seen the inside of Space Mountain as if it's some kind of secret. Millions have seen it all lit up. Back when the PeopleMover/Rocket Rods attractions were running it was a common sight, as the track ran through Space Mountain and sometimes it would be under maintenance with the lights on. And of course in emergency situations they turn the lights on as well.
Another one: he claims they use thin curtains to project ghosts on in the Haunted Mansion which is true, but while he's talking about it he shows footage of a part of the ride that uses Pepper's ghost which is a completely different (and more interesting) technique. Some of the ghosts he shows while talking about it could not be achieved with the curtain method.
Come to think of it, Pepper's ghost could fool lidar. Maybe that's why he didn't talk about it even though it would have been more interesting. It would have been inconvenient for his sponsor. Someone setting up a reflective surface across a road is probably about as likely in the real world as a Wile E. Coyote-style fake road wall.
I watched the video, the Wile E. Coyote fake wall stuff is a gimmick meant to draw kids in. That, however, par for the course of his videos; they are designed to hook kids into engineering with silly things and secretly teach real engineering before getting to the punch line.
In this case the real engineering is that Tesla's choice of relying on only visual camera has fundamental issues that can not be solved with cameras only. Namely, visually blocking elements, such as heavy rain, fog, or even blinding lights just pretty much can not be solved by camera-only sensors.
(though I guess one "solution" would be for the system to say I can't see enough to drive, so I'm not going to, but then no one would buy the system)
It is also a promotion for Lidar tech, Telling future engineers to be cautious of camera vision-only driving systems. I agree with Mark. But not because of the tests he intentionally created to make Lidar look better.
I really do hope camera only tech will do better than this. But I also hope that lidar technology will eventually make it better. Right now , LIDAR needs much heavier computing power to be reasonably powerful.
Mark posted a video on X showing him getting up to speed, engaging autopilot 4 seconds before the wall, and autopilot disengaging 1 second before hitting the wall.
https://x.com/MarkRober/status/1901449395327094898
>Here is the raw footage of my Tesla going through the wall. Not sure why it disengages 17 frames before hitting the wall but my feet weren’t touching the brake or gas.
The playing field level here was significantly slanted comparing a production Tesla, driven by Mark, engaging 10 year old autopilot technology, against a non-production test vehicle, not driven by Mark, using I would assume the latest in LiDAR technology from Luminar.
Volvo sells the EX90 with a Luminar LiDAR sensor (not active it looks like). Why wasn't it used with Mark driving?
We already knew this to be true by the clusters of Tesla fatalities around certain bay area off ramps.
this is factually just not true, at all. like outrageously not true. why would you just completely make something up like this?
This statement goes against any report and analysis on basic autopilot, not even FSD.
Just like the video, that if you’d take some time to watch l, you’d see that’s just basic autopilot.
Data is saying other things, but if we want to deny gravity, I’m ok with it.
Oh but no worries, FSD is a “solved problem” and should be done in 18 months or so…
Oh dear, your timeline casts doubts on the ability for a Tesla to self-drive from LA to New York before the end of 2017
In fact, he didn’t use FSD.
Remember, you gotta break some eggs to make an omelette; every time something crashes, explodes, or kills – that's a good thing! /s
Where can I buy the alternative lidar based car?
GM is working on it, but no indication yet when the lidwr version will be released. Currently their Super Cruise uses radar and cameras, plus pre-scanned lidar maps of roads that it compares everything against in real time.
That looked like a Lexus with its insignia blacked out, so I'm guessing it was a custom build. But if you want a car that comes with LIDAR, look at the Volvo EX90.
Volvo EX90 has the sensor, but it is not active.
Polestar 3 is the only car with lidar in the US that I know of.
If you get a 2020(19?) model 3 you get the proximity radar as well.
In China.
The test would have been more interesting if it had included a couple more or so production cars. From what I've read only a couple of cars are available with that expensive Luminar LiDAR.
I'd like to see how more ordinary automating braking systems would fare, which I think usually use radar and camera.
>The test would have been more interesting if it had included a couple more or so production cars. From what I've read only a couple of cars are available with that expensive Luminar LiDAR
There are none you can buy in the US that have LiDAR active, but certainly Luminar could have gotten one from its partner Volvo and had Mark driven that instead of the professionally driven test rig that was used.
Shouldn’t they have tested a human driver too? I have the feeling a majority of drivers would also go right through it if unaware of the setup, as it’s such an inconceivable scenario.
The whole point of driver assistance systems is to be better at this stuff than humans drivers, and most of them, with much less grandiose marketing, would have "seen" the wall in time to emergency brake.
A human would (or at least should) slow down going through the water and super dense fog that fooled the Tesla. From the shot behind I could spot the kid through the water blasts, I don't know how it looked in the car.
edit: on rewatch you can pretty clearly see the kid through the rain.
Fair questions from a Tesla Fan Boy...
https://x.com/SawyerMerritt/status/1901481711789109259
>I have some questions:
I wonder how long until techniques like Depth Anything (https://depth-anything-v2.github.io/) provide parity with human depth perception. In Mark Rober's tests, I'm not sure even a human would have passed the fog scenario, however.
Tesla also drives into tractor trailers because they think they're clouds
Here is the original Mark Rober video - https://www.youtube.com/watch?v=IQJL3htsDyQ
I dislike the fact that Mark's videos appear to increasingly borrow from the Mr Beast style, which is very distracting. There's also the fact that half the video has nothing to do with cars in the first place.
The main result here is not surprising - Tesla's vehicles are plagued by a litany of poor engineering decisions, many at a fundamental level. Not using Lidar for depth detection is beyond stupid.
I found the disney side video extremely interesting and entertaining. It would have been nice to just pad each of them out individually with enough content to have them stand on their own while meeting the criteria he needs to appease the algorithm.
I know what you mean. But he's trying his best to be the new Mr. Wizard and youtube has a very long list of demands.
> I dislike the fact that Mark's videos appear to increasingly borrow from the Mr Beast style, which is very distracting.
Yeah. My daughter likes him, but this latest thumbnail made me roll my eyes.
The first half explains lidar in laymen's terms
He actually disabled the autopilot, you can clearly see the system is off: https://youtu.be/IQJL3htsDyQ?t=942
Mark posted a video on X showing him getting up to speed, engaging autopilot 4 seconds before the wall, and autopilot disengaging 1 second before hitting the wall.
https://x.com/MarkRober/status/1901449395327094898
Did he do the same with LiDAR only turning it on 4 seconds before an impact?
Nothing new hear in this well produced video championing LIDAR.
Tesla now suffers from a toxic brand way worse for its future than missing lidar.
This is probably the right answer to all the other arguments I read above and below.
Just an easy target for haters now and then.
But seriously, how much more would Tesla have to spend on each unit, if they were to include a LIDAR sensor? why not optical and LIDAR?
Or reintroduce proximity radar at least.
[dead]
> The last scenario of a Wile E. Coyote-style wall with a fake road painted on it was obviously not realistic
...shows a photorealistic road on said wall. Last I checked, human drivers didn't have organs capable of LIDAR. Most would have crashed into this ridiculous obstacle.