Welcome to the debate. Today we're looking at a
controversy that sits right at the bleeding edge of
transportation technology. It's a dispute that really
divides the engineering world right down the middle, and the
outcome is going to determine how and frankly if our vehicles
drive themselves in the next decade.
We are talking about the war between vision only systems and
sensor fusion, specifically regarding the use of Lidar.
And this isn't just a theoretical argument anymore, is
it? We're looking at recent and,
well, quite alarming reports about the deployment of Tesla's
robo taxi fleet. The headline data suggests a
crash rate that's significantly higher than human drivers.
Some reports are saying up to four times higher.
It it just raises a very uncomfortable question.
Can a system that relies only on cameras ever truly match the
reliability of a system that uses active laser sensors?
And that's really the core of it.
Can a computer see the world well enough with just video
feeds to navigate safely? Or does excluding depth sensing
hardware like Lidar resent an insurmountable barrier?
I'm the advocate. My position is that visual input
is, well, theoretically sufficient because it mimics the
biological model. It mimics us.
I also suspect the current crash data is being heavily
misinterpreted because of some serious reporting biases.
And I'm the dissenter. My position is that abandoning
Lidar is a dangerous cost cutting measure that ignores the
kind of redundancy you absolutely need for safety
critical systems. We're seeing higher crash rates
not because of a reporting bias, but because of a fundamental
hardware deficit. When you take away the sensor
that tells you exactly how far away an object is, you are
introducing a level of risk that just shouldn't be on public
roads. OK, so let's get into the
machinery of this. We really need to start with the
first principles argument, because this is the hill the
vision only proponents are willing to die on.
There's a perspective shared by a contributor to our source
material, Retroviridae 6, that's basically an existence proof.
Ah, the humans do it argument. Exactly.
It's the biological argument. You and I drove here today.
We navigated traffic. We merged.
We avoided pedestrians. We did all of that using two
passive optical sensors, our eyes, and a biological neural
network, our brain. We don't have lidar in our
foreheads, we don't emit laser pulses to measure time of
flight, we don't have radar. We rely entirely on optical flow
and pattern recognition. Sure.
So Retrovira day six's point is that if a biological neural net
can drive a car using only passive optical sensors, then it
is physically possible for a synthetic neural net to do the
same thing. The physics allows it.
Therefore, the argument that Lidar is required is just false.
Lidar might be a shortcut, but it isn't a necessity.
The photons entering the camera contain all the information you
need to drive. I'm sorry but I just don't buy
that. Let me tell you why that is a
huge category error. Your conflating otential with
execution. Just because humans can drive
with their eyes doesn't mean robots should drive without
laser precision. But why not if the goal is to
replicate human capability? Theology is full of flaws we're
trying to engineer out of the system.
The promise of autonomy isn't to drive as well as a distracted
ape, it's to drive perfectly. And to drive perfectly, you need
data that the human eye simply cannot provide.
But we're not talking about fatigue.
We're talking about the sensory input required to build a model
of the world. But you cannot compete with
Lidar are using only visual cameras when it comes to what
I'd call ground truth. As another observer, Wiggly Worm
pointed out in the materials, Lidar provides absolute depth
data. So maybe we should break that
down a bit for anyone who isn't a robotics engineer.
Right. A camera is a passive sensor.
It takes in light and it creates a flat 2D image.
To figure out how far away a car is, the software has to analyze
the size of that car in the image, compare it to what it
thinks a car looks like, and then infer the distance it's
guessing. It's a very educated guess, but
it is a guess. It's inference based on
perspective and parallax, yeah. Lidar is active, It shoots out a
laser pulse, it hits a car, and it measures exactly how long it
takes for the light to bounce back.
It's simple physics. Distance equals time multiplied
by the speed of light. It doesn't guess, it knows.
It says there is an object 12.4 meters away.
Wiggly Worm's point is that when you remove that sensor, you're
forcing the computer to hallucinate death.
And when you look at the stats, Tesla's robo taxi is reportedly
crashing at a rate 4 times higher than humans.
That isn't just the learning curve, that is a failure of
perception. I think that statistic, the four
times higher crash rate, is doing a lot of heavy lifting in
your argument, and I I want to contextualize it.
We need to be very careful about comparing apples to oranges
here. A crash is a crash, isn't it?
Not necessarily. Another analyst, Eskrove 2 He
noted that this specific headline is based on a really
small sample size. 5 incidents in Austin in a single month.
But if you look at the granularity of those incidents,
the data includes really minor events like a tire touching a
parking sign or bumping A curb while parking. 5 incidents in a
month for a small fleet is still high.
But think about human behavior. If I scrape my rim on a curb
while I'm parallel parking, or if I tap a plastic Bullard at
one mile per hour, do I call the police?
Do I call my insurance company? No.
It never enters the statistical record.
It just vanishes. Right, it's unreported.
Exactly, but for a robo taxi, every single sensor reading is
logged. Every thump is a reported
incident. The system self-reports
everything. So you're comparing reported
autonomous incidents where every scratch is scrutinized against
reported human accidents, which are usually only the the one
severe enough to require tow truck.
Ask contributors UX Test and Jerkletos pointed out you're
comparing A microscope to a telescope and then claiming the
microscope sees more dirt I. Understand the reporting bias
argument. It's valid to a point, but I
also think it's a convenient way to wave away failure.
You call it rubbing a curb. I call it a failure of object
permanence. That feels like a bit of a
stretch for a scratched rim. Is it?
Contrast this with Waymo. We have user experiences from
San Francisco contributors like Turbo Encapsulator and Luda lol
who describe Waymo as flawless in the same complex urban
environments where these vision based systems are struggling.
Waymo uses Lidar. They have that spinning bucket
on the roof. They aren't scraping rims.
They aren't bumping signs. They're also driving in a
fishbowl. You're driving in San Francisco.
That's hardly officiable. It's a Geo fenced pre mapped
environment that Waymo knows exactly where every curb is
because it has a high definition map stored in its hard drive.
It's not seeing the curb, it's remembering it.
Tesla's vision approach is trying to do something much much
harder. Drive anywhere on any road
without a map, just like a human.
Of course it's going to be clumsier in the beginning.
It's learning general intelligence, not just
memorizing a map. But that clumsiness has real
world consequences. The reports of these vision only
cars hitting stationary objects like parking signs?
That indicates A fundamental flaw.
If a vision system cannot calculate the distance to a
concrete Bullard well enough to avoid hitting it, how can we
possibly trust it to calculate the velocity of a child running
into the street? Because the neural networks are
weighted differently for those tasks, the system is likely
hyper cautious around pedestrians, but has a higher
tolerance for static objects to facilitate, say, parking.
That is an assumption. Lidar solves the static object
problem instantly. It doesn't need to infer or wait
anything. It hits the Ballard with a laser
and it knows it's there. The fact that these vision based
cars are hitting stationary objects suggests that the
software is hallucinating free space where there is solid
matter. That is terrifying.
It implies the car literally does not know the physical
boundaries of its own environment.
I will concede that static object detection is a hurdle
right now, but identifying these edge cases is exactly how you
train the network. Every time it hits a parking
sign at 2 mph, it uploads that failure and the entire fleet
learns not to do it again. And this brings us directly to
the concept of systemic risk. We have to talk about the
multiplier effect. Explain how you view that.
This was articulated very well by the source contributor Becker
Hollow. The argument is all about error
scaling. When a human makes a mistake, it
causes 1 accident. Human error is stochastic.
It's random. You might get distracted by a
text. I might drop my coffee.
It's isolated. Sure, individual variants.
But when a vision based software has a flaw in its programming,
say a specific inability to distinguish a white truck
against a bright sky, that error is replicated across every
single device on the road. The centralized bug.
Exactly. If the software misinterprets a
specific shadow or glare, thousands of cars become
dangerous simultaneously in the exact same way.
You aren't dealing with one bad driver, you're dealing with a
fleet of clones all sharing the same blind spot.
That is a systemic risk profile that we have never, ever dealt
with in automotive history. That's an interesting point,
though I would frame it differently.
That logic flips both ways. It's actually the strongest
argument for autonomous systems. Yes, an error is distributed,
but so is the solution. If they catch it in time.
Think about it, when a human driver is bad at merging,
they're usually bad at merging forever.
You can't patch their brain, but if you solve the edge case in
software, if you fix that white truck against the sky bug, you
instantly fix every car on the road.
You can upgrade the safety of the entire fleet overnight with
an over the air update. The multiplier effect applies to
safety even more than it applies to error.
You're leveraging the collective learning of millions of miles.
But. Until that fix arrives, the risk
is distributed to the public without their consent.
The public roads are becoming a beta testing environment.
We're seeing a move fast and break things mentality applied
to two ton metal projectiles. Beckerhollow's logic holds the
error rate is a normal human error multiplied by the number
of devices using the program. If the program is flawed, the
carnage is scalable. I see why you think that, but
let me give you a different perspective on the technical
reliability piece. You keep going back to Lidar as
this source of truth, but Lidar has its own failure modes.
It does, but they are different from cameras.
Lidar struggles with heavy rain. The laser pulses scatter off the
water droplets. It struggles with fog.
It can get confused by interference from other lidar
units. It's not magic.
And this is where the whole sensor fusion argument gets
tricky. Go on.
When you have a camera an A lidar, they will often disagree.
The camera sees a plastic bag blowing across the road and
thinks it's nothing. Lidar sees an object and says
obstacle emergency brake. Now the computer has to decide
which sensor to trust. This is the sensor fusion
conflict. By removing Lidar, Tesla is
arguing that you remove the noise.
You force the neural net to resolve the visual data just
like a human does, without getting confused by conflicting
signals. That sounds like a very
convenient engineering rationalization for saving
money. Cameras are cheap, Lidar is
expensive. It's definitely cheaper, but
retroverted Z6 mentioned. We went from the horse and buggy
to the moon in just a few decades.
Assuming that computer vision can't bridge the gap just
because it hasn't yet is premature.
The issue isn't that the camera is blind, it's that the
processing isn't yet sophisticated enough.
But processing power is scaling exponentially.
Software cannot conjure photons where there are none.
That's the physics problem. But it can interpret context.
Let's talk about those photons. Cameras are passive.
They need light. What happens when you drive
directly into the sunset? We've all done it.
The visor goes down. You squint.
You can barely see. Cameras get blinded by sun
glare. They get obscured by mud.
We have reports that Tesla is having to employ trailing chase
cars with human safety monitors for their autonomous taxis.
If the system is so theoretically sound, why does it
need a human babysitter in a separate vehicle?
Every developmental technology has safety protocols during
testing, but. This is being sold as a future
that is just around the corner. The reliance on cameras
introduces A fragility that Lidar solves.
Lidar cuts through sun glare. It works in total darkness.
It is a second layer of truth. If the camera sees a shadow and
thinks it's a hole in the road, the Lidar says no, the ground is
flat. Removing that sensor removes a
layer of survival. It's engineering hubris to
believe you can derive 100% certainty from a sensor that is
susceptible to optical illusions.
I'm not convinced by that line of reasoning because it assumes
we can't solve optical illusions with better AI.
But let's pivot to the consequences of this, because
the legal aspect is fascinating. It's a nightmare.
This leads us to the inevitable question of accountability.
When these systems do fail, whether it's a clumsy bump or a
serious collision, who is responsible?
This is the question posed by Shifty Mennonite in our source
threads. Who is going to be held
accountable when these things mow people down?
It is a legal quagmire. Is it the driver, which in this
case is the software? Is it the manufacturer or is it
the limitations of the sensor suite itself?
I think we need to distinguish between a software bug and a
design choice. This is crucial.
If a car crashes because of a line of bad code, that's one
thing. But if a manufacturer knowingly
removes a safety sensor like Lidar, a sensor that is industry
standard for competitors like Waymo, and that removal leads to
a crash because the camera couldn't estimate depth, that
feels very distinct from a mere coding error.
You're suggesting negligence. I'm saying it borders on it.
There's this sentiment expressed by User Beneficial Soup 3699
regarding blatant fraud. While that is, you know, strong
language, the core sentiment is valid.
If you claim a camera is sufficient and the physics
suggest it isn't, and the data shows it crashing, at what point
does adherence to a vision only philosophy become liability?
But that assumes Lidar was prevented that specific crash.
We don't know that. Like I said, LIDAR isn't a magic
bullet. It's not magic, it's redundancy.
In aviation, we don't fly with one altimeter, we have three.
Why on earth should we drive with one type of eye?
If the camera fails due to glare or a bug or mud, there's nothing
to catch the car. It is a single point of failure
system. But there's an economic argument
here too. If you require Lidar, you make
autonomous cars cost $100,000. They become toys for the rich.
If you can solve it with vision, the hardware costs 500 $100.
You can put it in every car. A vision based system that's 99%
safe and available to everyone might save more total lives than
a lighter system that's 99.9% safe but only 1000 people can
afford it. That is a utilitarian calculus
that works on a spreadsheet, but it doesn't work when you're the
one crossing the street. The public Rd. should not be a
testing ground for cost cutting measures, disguises innovation.
The experiences of users in San Francisco and Austin show a
clear divide. Waymo with Lidar is providing A
flawless service while vision only systems are struggling with
basic static objects. But again, Waymo is on rails.
It's a local maximum. It's great for San Francisco,
but it doesn't scale to the rest of the world.
I'd rather have a safe local maximum than a dangerous global
beta test. Until vision systems can match
the redundancy and depth accuracy of Lidar, the safety
consequences are real and they are statistically proven.
We cannot verify the safety of a black box neural net without
ground truth sensors. It ultimately comes down to that
multiplier effect we talked about.
It does. We are at a crossroads.
We can take the safe, expensive route with Lidar, which might
limit the scalability of the technology but provides that
warm blanket of redundancy. Or we can push for the vision
solution, which, if it works, multiply safety exponentially
across the globe and solves general intelligence.
But if it fails, it multiplies error.
It multiplies the risk of a single software blind spot into
a nationwide. Hazard, and that is the gamble.
Is the current risk worth the future reward?
I tend to believe that without taking that risk, we stagnate.
We'd still be driving horses if we waited for the perfect car.
And I would argue that safety is not a place for gambling when
you're moving 2 tons of steel at 60 mph.
Pretty good isn't good enough. You need absolute truth and
cameras just don't provide that. A fundamental disagreement on
the philosophy of engineering. Thank you for listening to the
debate. We hope this exchange has
illuminated the complexities behind the sensors.
Drive safe everyone, and watch out for the robots.
Goodbye.