I am a massive fan of Mark Roeper. Unfortunately he completely f**d up this one.
He tested using Autopilot, not the latest FSD on HW4, which is worlds apart in capabilities.
It is possible that the latest FSD would also crash, but that would be a valid test of FSD capabilities.
Testing using Autopilot and calling it "FSD crashes" is a HUGE misrepresentation of facts.
I am hoping Mark will post an update to the video.
Adding to the weirdness of this video, it appears Mark Rober faked his footage to make it look like he was using a Google Pixel to record screen video, but he was actually using an iPhone as can be seen in the screen reflection. And he put the "G" logo in the wrong orientation in the faked shot.
Also it's weird that he's acting like he's so special for having seen the inside of Space Mountain as if it's some kind of secret. Millions have seen it all lit up. Back when the PeopleMover/Rocket Rods attractions were running it was a common sight, as the track ran through Space Mountain and sometimes it would be under maintenance with the lights on. And of course in emergency situations they turn the lights on as well.
Another one: he claims they use thin curtains to project ghosts on in the Haunted Mansion which is true, but while he's talking about it he shows footage of a part of the ride that uses Pepper's ghost which is a completely different (and more interesting) technique. Some of the ghosts he shows while talking about it could not be achieved with the curtain method.
Come to think of it, Pepper's ghost could fool lidar. Maybe that's why he didn't talk about it even though it would have been more interesting. It would have been inconvenient for his sponsor. Someone setting up a reflective surface across a road is probably about as likely in the real world as a Wile E. Coyote-style fake road wall.
I watched the video, the Wile E. Coyote fake wall stuff is a gimmick meant to draw kids in. That, however, par for the course of his videos; they are designed to hook kids into engineering with silly things and secretly teach real engineering before getting to the punch line.
In this case the real engineering is that Tesla's choice of relying on only visual camera has fundamental issues that can not be solved with cameras only. Namely, visually blocking elements, such as heavy rain, fog, or even blinding lights just pretty much can not be solved by camera-only sensors.
(though I guess one "solution" would be for the system to say I can't see enough to drive, so I'm not going to, but then no one would buy the system)
Aside of all the other obvious reasons to not get a Tesla these days this is #1 imo. Camera feeds and a neural network are not enough for self driving, no matter how much they're training. Never ever.
Mark posted a video on X showing him getting up to speed, engaging autopilot 4 seconds before the wall, and autopilot disengaging 1 second before hitting the wall.
>Here is the raw footage of my Tesla going through the wall. Not sure why it disengages 17 frames before hitting the wall but my feet weren’t touching the brake or gas.
The playing field level here was significantly slanted comparing a production Tesla, driven by Mark, engaging 10 year old autopilot technology, against a non-production test vehicle, not driven by Mark, using I would assume the latest in LiDAR technology from Luminar.
Volvo sells the EX90 with a Luminar LiDAR sensor (not active it looks like). Why wasn't it used with Mark driving?
Shouldn’t they have tested a human driver too? I have the feeling a majority of drivers would also go right through it if unaware of the setup, as it’s such an inconceivable scenario.
The test would have been more interesting if it had included a couple more or so production cars. From what I've read only a couple of cars are available with that expensive Luminar LiDAR.
I'd like to see how more ordinary automating braking systems would fare, which I think usually use radar and camera.
I wonder how long until techniques like Depth Anything (https://depth-anything-v2.github.io/) provide parity with human depth perception. In Mark Rober's tests, I'm not sure even a human would have passed the fog scenario, however.
1) Your video "Can You Fool a Self-Driving Car?" uses Luminar’s latest tech but not Tesla’s latest FSD software. Why?
2) Autopilot was turned on at 42 MPH in your YouTube video but you turned it on at 39/40 MPH in your clip above. Why? Multiple takes?
3) In the clip above, Autopilot was activated MUCH closer to the wall than in the YouTube video clip. Why?
4) In your video above, you turned on Autopilot 3.8 seconds before hitting the wall, but it appears you gave Luminar a much longer head start with their tech "activated." Why? Am I wrong in my assumption?
5) Why was putting a child dummy/doll behind the wall a useful thing to do? What car would possibly see or react to a kid through a wall after crashing into that wall?
I dislike the fact that Mark's videos appear to increasingly borrow from the Mr Beast style, which is very distracting. There's also the fact that half the video has nothing to do with cars in the first place.
The main result here is not surprising - Tesla's vehicles are plagued by a litany of poor engineering decisions, many at a fundamental level. Not using Lidar for depth detection is beyond stupid.
> The last scenario of a Wile E. Coyote-style wall with a fake road painted on it was obviously not realistic
...shows a photorealistic road on said wall. Last I checked, human drivers didn't have organs capable of LIDAR. Most would have crashed into this ridiculous obstacle.
I am a massive fan of Mark Roeper. Unfortunately he completely f**d up this one. He tested using Autopilot, not the latest FSD on HW4, which is worlds apart in capabilities. It is possible that the latest FSD would also crash, but that would be a valid test of FSD capabilities. Testing using Autopilot and calling it "FSD crashes" is a HUGE misrepresentation of facts. I am hoping Mark will post an update to the video.
I don’t live near a lot of Wild E. Coyote fake road walls. I get the sense it’s more of a Midwest thing.
Adding to the weirdness of this video, it appears Mark Rober faked his footage to make it look like he was using a Google Pixel to record screen video, but he was actually using an iPhone as can be seen in the screen reflection. And he put the "G" logo in the wrong orientation in the faked shot.
Also it's weird that he's acting like he's so special for having seen the inside of Space Mountain as if it's some kind of secret. Millions have seen it all lit up. Back when the PeopleMover/Rocket Rods attractions were running it was a common sight, as the track ran through Space Mountain and sometimes it would be under maintenance with the lights on. And of course in emergency situations they turn the lights on as well.
Another one: he claims they use thin curtains to project ghosts on in the Haunted Mansion which is true, but while he's talking about it he shows footage of a part of the ride that uses Pepper's ghost which is a completely different (and more interesting) technique. Some of the ghosts he shows while talking about it could not be achieved with the curtain method.
Come to think of it, Pepper's ghost could fool lidar. Maybe that's why he didn't talk about it even though it would have been more interesting. It would have been inconvenient for his sponsor. Someone setting up a reflective surface across a road is probably about as likely in the real world as a Wile E. Coyote-style fake road wall.
I watched the video, the Wile E. Coyote fake wall stuff is a gimmick meant to draw kids in. That, however, par for the course of his videos; they are designed to hook kids into engineering with silly things and secretly teach real engineering before getting to the punch line.
In this case the real engineering is that Tesla's choice of relying on only visual camera has fundamental issues that can not be solved with cameras only. Namely, visually blocking elements, such as heavy rain, fog, or even blinding lights just pretty much can not be solved by camera-only sensors.
(though I guess one "solution" would be for the system to say I can't see enough to drive, so I'm not going to, but then no one would buy the system)
Aside of all the other obvious reasons to not get a Tesla these days this is #1 imo. Camera feeds and a neural network are not enough for self driving, no matter how much they're training. Never ever.
Mark posted a video on X showing him getting up to speed, engaging autopilot 4 seconds before the wall, and autopilot disengaging 1 second before hitting the wall.
https://x.com/MarkRober/status/1901449395327094898
>Here is the raw footage of my Tesla going through the wall. Not sure why it disengages 17 frames before hitting the wall but my feet weren’t touching the brake or gas.
The playing field level here was significantly slanted comparing a production Tesla, driven by Mark, engaging 10 year old autopilot technology, against a non-production test vehicle, not driven by Mark, using I would assume the latest in LiDAR technology from Luminar.
Volvo sells the EX90 with a Luminar LiDAR sensor (not active it looks like). Why wasn't it used with Mark driving?
We already knew this to be true by the clusters of Tesla fatalities around certain bay area off ramps.
Oh but no worries, FSD is a “solved problem” and should be done in 18 months or so…
Where can I buy the alternative lidar based car?
Tesla also drives into tractor trailers because they think they're clouds
Shouldn’t they have tested a human driver too? I have the feeling a majority of drivers would also go right through it if unaware of the setup, as it’s such an inconceivable scenario.
The test would have been more interesting if it had included a couple more or so production cars. From what I've read only a couple of cars are available with that expensive Luminar LiDAR.
I'd like to see how more ordinary automating braking systems would fare, which I think usually use radar and camera.
I wonder how long until techniques like Depth Anything (https://depth-anything-v2.github.io/) provide parity with human depth perception. In Mark Rober's tests, I'm not sure even a human would have passed the fog scenario, however.
Fair questions from a Tesla Fan Boy...
https://x.com/SawyerMerritt/status/1901481711789109259
>I have some questions:
Here is the original Mark Rober video - https://www.youtube.com/watch?v=IQJL3htsDyQ
I dislike the fact that Mark's videos appear to increasingly borrow from the Mr Beast style, which is very distracting. There's also the fact that half the video has nothing to do with cars in the first place.
The main result here is not surprising - Tesla's vehicles are plagued by a litany of poor engineering decisions, many at a fundamental level. Not using Lidar for depth detection is beyond stupid.
He actually disabled the autopilot, you can clearly see the system is off: https://youtu.be/IQJL3htsDyQ?t=942
But seriously, how much more would Tesla have to spend on each unit, if they were to include a LIDAR sensor? why not optical and LIDAR?
Nothing new hear in this well produced video championing LIDAR.
Tesla now suffers from a toxic brand way worse for its future than missing lidar.
> The last scenario of a Wile E. Coyote-style wall with a fake road painted on it was obviously not realistic
...shows a photorealistic road on said wall. Last I checked, human drivers didn't have organs capable of LIDAR. Most would have crashed into this ridiculous obstacle.