----

Eliezer Yudkowsky is fond of shitting on the AI developers of the 1960s for thinking they could write `APPLE` in the source code of a symbolic language and that that made them weeks away from a human intelligence which could reason about apples, and how simplistic that looks now.

Like YouTube auto-transcribed subtitles are useful but they are obviously transcribing sounds without understanding, they lack understanding of where the context indicates that a spoken thing should be a name, or they will transcribe the same word two different ways in two different sentences with no understanding that it was the same object as before being referred to again, or where a sound is unclear I can fill in what was intended but the auto transcriber can't, or I can see from lip movement that the transcription was wrong, the audio processor can't integrate multiple inputs in that way, and they will transcribe sentences which are grammatically correct but human background knowledge of the world tells you it makes no sense.

Similar with self driving cars, it's pretty clear from the outside that you can't have a car which can reason about the state of a city, its roads, the things in the roads, the environmental conditions, without having a large amount of interconnected human level background understanding of the world and the things in it. e.g. not just seeing a shape and identifying it as a cyclist, but knowing that you passed a cyclist a few seconds ago and now you are slowing down for traffic lights the cyclist will be coming back alongside you momentarily. Not just identifying a parked car, but seeing a car stop moving and turn its lights off as it parks implies the doors are about to open. Not just seeing lane markers in the road, but seeing no lane markers and being able to complete the pattern of where the lane markers should be because you understand how humans design roads. Not just seeing rain and slowing down, but the hinkiness feeling of "these conditions are dangerous" from the way other cars are driving, the road conditions, and slowing down in advance of anything objectively happening because you predict what could happen. Not just seeing a sign saying 'Diversion' but being able to look around expecting to see the next diversion route sign either down this turning or up ahead by another turning, and using that extra information to decide what to do. Not just identifying an erratically moving vehicle when you see it, but hearing a siren and seeing a flash of blue in the mirror and thinking ahead that an ambulance is coming and then looking for places to pull over to let it past and expecting the cars around you might move like that as well. Not just seeing the car in front slowing down, but seeing the driver inside it move and understanding that they ware waving you past because they are double-parking to drop someone off or pick someone up instead of slowing down because of traffic. And countless other situations.

Humans have good reaction time when it comes to touching something hot and pulling our hands away before we understand and are aware of what happened. Sensor equipped cars have good reaction time when it comes to ultrasound sensing a thing up ahead and applying the brakes without understanding what's happening. Humans have bad reaction times when driving because we can't feel the thing in the road, it has to go through our slower higher level thinking to understand what's happening before we can choose to respond.

Self-driving cars, then, are either the pretense that you can put a human level AI on top of the car's unconscious reactions, without compromising the reaction time, to get a superhuman level driver. And that's not something you can do because human level AI doesn't exist. Or they are the unfounded claim that you can drive through humanspace without human understanding, which is about as convincing as saying you can send a machine to the butcher, baker and candlestick maker to do your shopping without it having any AI. As soon as anything goes off-plan the robot is stuck. And you get into "well, we'll hard code a workaround for this situation and simply enumerate everything which could go wrong in a decision tree". Shop door closed with a sign saying "please use other door"? Hard code that, OK now are we good? Shop door closed with a sign saying "please ring bell for attention"? OK, hard-code that, now are we good? Shop door propped open with a mop and bucket and a sign saying "caution, wet floor"? OK, hard-code that, now are we good? Butcher says "sorry we have no liver but we're expecting a delivery in 5 minutes are you OK to wait?"? OK, hard-code that, now do we have AI? And then you get to Amazon which controls the warehouse layout, temperature, environment, shelving, can put tracks in the floor, put all items into regular sized boxes tagged with machine readable labels, which is more analogous to trains and trams on rails, and still Amazon use humans to pick and pack things.