I'd like to see an architectural overview of a self-driving car. What are the most challenging systems? I can imagine that the most challenging part is the computer-vision system for detecting the road, and the radar system for detecting other vehicles and pedestrians. Given this information, finding a path from A to B without collisions seems not so difficult in comparison.
Interesting as something to explore, but I'm curious if a network of self-driving cars can be optimized to recognize and avoid these situations altogether.
Following a well-marked winding road is not a difficult challenge for self-driving cars. Driving in a city, with bus lanes, light rail, bikers, and pedestrians is much harder.
The history of self driving cars shows that it is not an easy task. We've been working on this for over 15 years. I was on a self driving car team during 2008-2010. We won multiple intercollegiate competitions. At that time I was certain it would be 20+ years before we had something that could drive on the roads. The progress has been spectacular.
I would argue these are not self driving cars. I would expect a self driving car to have multiple compute nodes in the trunk that use something more advanced than a gaming engine, a voting system and numerous sensors. LIDAR, infrared, thermal, sonar, other.
An autonomous car that expected to operate in locations and conditions that humans often fail in should be advanced enough to know everything around it and then some.
This is why self driving cars are so much harder than one might think. Just getting the basics of self driving down is challenging. Now throw in all the possible edge cases, some of which shouldn't ever happen, and you've got a wildly interesting and challenging problem. It would be a blast to work on that team.
But it seems like the main problem for self drive is accurately understanding the world around the car. The actual driving is pretty easy. Being able to see a partial object and understand what the rest of it is is very useful to human drivers
I'm not sure if the biggest challenge is sensors or software. I don't know what Tesla is running, but I have a strong feeling that software in the large is not up to the task of autonomous driving. Most software (including automotive) has only very limited realtime behavior due to memory allocations, OS preemption, interrupts, ..., error-prone programming languages like are used and resource (memory) usage is often unbounded. I can't imagine that Tesla or anybody else that produces self-driving features at the moment is using something like Ada Ravenscar or advanced static validation techniques through all components that are involved in the self-driving features - and which are often quite complicated (image recognition, etc.) and therefore hard to run in such an environment.
The hard part of self-driving cars is not staying within the painted lines of the road, the hard part is avoiding other valuable things like cars, trucks, bikes, people, wildlife, etc. There is no reliable visual grid for those things.
Subways handle this problem by being given dedicated underground tunnels which exclude all those things. And even subways hit people sometimes when they fall onto the tracks.
Self driving on highways. I know many companies are focusing on tier V, everywhere. but it's starting to look like an AI-hard problem to me, and might not yield a usable solution for a while.
Meanwhile, I believe cars could do motorway driving from entry to exit today, if we focused on that. With LiDAR, some extra safety hardware on roads, coordination with local highway services, a company could make a car that is fully self driving on specific highway stretches.
A self-driving car is just an intrinsically easier problem to solve. In fact, the first car that automatically centred a car within a lane using cameras to track lane markings hit the road in 1977! They had cars doing cross-country trips on highways with traffic in the 1990s!
Obviously there are complexities to it, which is why don't have any level 4 vehicles is just so much lower.
Humanoid robots are just several magnitudes harder because you first have to master the basic mechanics of human-like movement before you can move onto higher order things like avoiding running into a person. With a bipedal robot, a simple operation like walking up a hill is a major achievement. In the car domain, you essentially get that stuff for free.
I'm wondering how this is going to be different than self-driving, where we can get 90% of the way there, but the last 10% is notoriously difficult with not nearly enough edge cases represented in the data
hmm, I think the real time aspects of self driving are hard. There is lots of promise with masked language and image modeling, but plenty of work to be done, even for non real time applications.
Part of self driving is dealing with those human drivers who are terrible drivers though.
This is what makes the whole thing so hard. If we could have dedicated roads for self driving cars it would be easy peasy. No need to interpret road markings, signs, other vehicles and their intentions. Because it would all be communicated digitally over radio. Steering a car is total peanuts for a computer.
It's the unexpected stuff that makes it hard. Like that Uber car that killed that lady that crossed the road with her bike in the dark at a totally unexpected spot. A human is better at those things because they're caused by other humans who aren't logical.
But such matters are really hard to test and evaluate for precisely because they are unexpected.
I know this is a heated debate, but it's interesting to see the divergent approaches that are being taken to achieve self driving. In one corner, it's lidar-based, equipping vehicles produced by other companies, organically and incrementally growing the navigation technology.
In the other, it's video-based with now 100s of thousands of vehicles feeding data to a massive neural net.
Each has very well thought-out arguments for the approach they are taking, and I expect that both would ultimately 'work' by reducing the inherent risks of taking a car ride by a substantial amount (90%?). I think the winner will be the one who gets to that milestone soonest.
reply