The use case does not include operating remotely at 70mph on freeways, esp because internet connectivity is hard over most the USA. That bit is mostly lvl 3/4, and is something that is addressed well by other companies. It's the long tail between lvl3/4 and all the scenarios that remain in which a remote driver solves the problem. Esp in logistics yards, which are chaotic, fast paced, and driven by voice communication.
Definitely can't remotely operate a 40 ton truck. Wireless network latency is a bit high even under ideal conditions, and latency spikes could easily cause disastrous accidents.
I saw a thing on CBS Sunday Morning about a company planning to run the local parts of truck routes on remote control.
More: the point being that it is a huge opportunity and there doesn't necessarily need to be a perfect, fully capable system to start having a significant impact on the number of drivers.
Great and detailed article. I agree with the comment from balaselvam though. On the way towards full autonomy (whenever that might be achieved) you will get there incrementally (like Tesla is trying to do, their constant delays/promises notwithstanding). At some point highway driving will be ok, but even for highway driving it is absolutely imperative to have a very good understanding of how to drive, the level of autonomy needs to be quite advanced. Otherwise you can quickly end up in a situation where unforeseen traffic guidance, debris, loss of communication will result in a situation where the truck does not react well enough.
I doubt it is really possible to design a system for 'just' highway driving of such high quality that no intervention is required within the reaction time of a remote operator without actually trying to fully solve it. This could be limited to highways of course, but a loss of connection in city regions would then automatically mean the truck would have to stop where it is (in the middle of the road with hazards on)
I do like the remote operator scenario for start and end quite a lot though as it would ease the incremental inclusion of autonomy into the truck economy and offer higher safety with machines this big.
The video stream bandwidth and requirement for near-zero latency will make remote control very challenging, if not impossible. I'd say the last-mile driver will be in the truck.
Any kind of remote assistance will most likely not include steering the truck in real time over the internet.
Rather, things like "unknown obstacle on the road, can't continue", with the operator instructing it to just run over that empty cardboard box. All realtime safety decision-making should have safe autonomous fallbacks, a human cannot be expected to drop in and take charge in seconds. The entire level 3 / 4 "semiautomation" market is a bunch of make believe.
The trucks could have AI that worked most of the time. Not well enough to rely on for normal operations, but good enough for a fallback in case of losing connectivity.
Some made-up numbers to explain my point: Consider a situation where the remote operator will make the right response 99.9% of the time, failing 0.1%, and the AI is 20 times worse, making an error 2% of the time. The actual error rate will be (0.1% * connection reliability) + (2% * (1 - connection reliability)). For 3-nines reliability, which is pretty poor, the error rate will be about 0.102%. For 4-nines it will be 0.1002%.
Those sorts of figures would effectively pay for themselves in terms of safety, once you factor in the benefits for monitoring and work conditions for a desk-operator vs a physically present driver, and they don't require either a particularly reliable connection or a particularly good backup AI.
Basically operate it remotely for the first and last mile (instead of the driver physically being in the truck), and let the computer take over for the long stretches on the freeway.
I think there are a lot of other interesting applications here too:
* "drive-by-wire" optimization of operator inputs to save on fuel or reduce machine wear
* failsafe gps boundaries to prevent the chance of equipment leaving the operating zone
* smart personnel and equipment tracking to avoid collisions or accidents in the workplace
* future reduction in cost of remote-only equipment - cabins along with everything else needed for "creature comfort" requirements is probably a notable percentage in bill of materials and related shipping/running costs.
* cheaper labor operating costs - jobsite risks, repetitive-stress-injuries and operator insurance would go down for sure - it's also safe to assume savings by employing drivers from anywhere vs localized and often remote
* uptime: machines can run 24/7 with the exception of fueling - no downtime to change drivers or have a driver ferry back and forth for a shift change
* easy path to automation, even if extremely limited in scope during first iterations it is reasonable to see how simple tasks such as repositioning and repetitive tasks could be automated, even with macros.
* Having deeper telemetry about equipment operation (how many lb-ft has the forklift lifted in the last 3 months, how much time did the loader spend idling, how quickly did the machine respond to X input, what are the outliers for this machine vs all the other models in the fleet) can lead to much improved maintenance and preventive repairs. Maintenance programs in most cases today are simply based on hours and only the manufacturer or dealer would see anything beyond that with regards to machine usage.
Larger manufacturers seem to be exploring this and offering remote operating consoles as an add-on but it seems pretty far behind.
The future might look something like this https://www.google.com/search?q=spmt+remote&tbm=isch where highway miles are done by computer and the last few hundred feet are done by remote. For low speed work, sitting in the cab is a significant disadvantage.
This could be great for freight trucking where they spend majority of time on freeways, which should be easy win for self-driving, and then do the last leg with a virtual, or physical pick-up driver. Likewise for shipping.
I wonder how latency is handled. I wonder how close nearby you'd still want the remote driver since someone across the world would have at least hundreds of milliseconds in delay communicating back and forth
Also, what happens in periods of disconnectivity? It seems like you'd still need some autonomous capability then, although it might be lessened to only needing to pull over?
I agree but some problems disappear too. For example, occupants of cars get frustrated when they are stuck for quite some time on a complicated turn that a human would accomplish quickly. Packages don't care either if the driving is not smooth and the vehicle responds in favour and with extra care of other vehicles around it. Not to mention routes that might pick up and deliver multiple simultaneous loads. It just seems that would be an easier first step for the tech that should be profitable. But these are just questions I have. I'm very unsure about what the reality is.
You can't do it with autonomy, and you can't do it with remote piloting, but you can do it with a mix of both. That's my assertion. Spending 20 hours looking at Nebraska / Illinois farm fields is not the use case for remote operation - and as you say, you lack the infra for that anyway. And navigating traffic or interacting with distribution center logistics is not the case for autonomous operation - it's an infra nightmare to get a autonomous vehicle to integrate with all that radio-voice-comms madness. Even OTR drivers just want to get that part of the journey over with and get back to the hotel.
Having a pod with local drivers near major hubs, for example, means the drivers take over when the trucks get close enough for it to matter. It wont' work for delivery to, say grocery stores (which can be local driving anyway), but it will work for center-to-center transport ala between Walmart hubs. It's the long haul OTR trucking that has high attrition / people shortages, because you're away from family for so long. That little bit can be somewhat autonomous, with handoff of remote operators only near hubs. Think air-traffic-control for trucks and hubs, except probably more hands on than just telling it what to do.
It's not necessary for the automation to solve the most difficult environments. As long as it can handle the majority of the distance it would be quite useful. For example a small number of operators could get the trucks to a depot near a highway, trucks could drive autonomously non-stop across the country, and then another operator could manually move them from a receiving depot to a local destination.
So really this is just a demonstration of the truck being remote controlled and being monitored by a full-time driver in a remote location? Okay, I can see why that's an important tool to make this a viable product, but it doesn't scale and doesn't strike me as a differentiator (I know some of your competitors already do this).
Anyway, rather than argue, I wish you luck. I've got no horse in this race.
The problem is that long haul trucking isn't just "most big interstates", the only advantage of long-haul trucking is the logistical flexibility, if you're constrained to "most big interstates" you might as well use a train.
The remote on-call strategy doesn't scale, yes it'll save some money relative to drivers, but there's a delicate balancing act involved in mitigating risk of multiple simultaneous failures, yes this might be rare, but it'll also be costly.
See my response to wpietri for more about security, but with respect to warehouses and the like, they are frequently staffed by armed guards or dogs, it's also orders of magnitude easier to secure a stationary facility compared to a mobile autonomous vehicle traveling across the country. Insurance will only cover it if they can tolerate the risk.
Increased availability is definitely a win, but the other advantages you listed are just alternative ways of saying "lower cost". Yes, the costs are lower, but truck driver salaries aren't that high to begin with and the cost of building out and maintaining automated trucking infrastructure will be passed on to any trucking companies that decide to adopt the technology.
Yes, and that has to be safely possible. From my reading, the article has no problems with remote operators giving command. The truck just also needs the ability to both safely execute them and come to some safe state by itself if necessary commands don't arrive [in time].
The interesting question is how complicated those safe states are to realize. Just hitting the breaks on a freeway seems hardly acceptable. Generally, this being harder than for cars in cities is a core conclusion of the article.
It's simply too risky and will be for the foreseeable future. The liability cost for a semi-accident is huge, not just in lives but in cargo and in the truck itself.
Highways have decent network coverage but nothing close to the levels of reliability necessary to satisfy the actuaries. From a risk perspective, failure mitigation strategies like slowing or stopping are largely hamstrung by the same reliability concerns that prevent the vehicle from simply driving on its own, the vehicle may simply not recognize that the situation is dangerous until it's too late, that's without factoring in the reality that stopping or slowing can actually be extremely dangerous and can't be relied upon as a catch-all fallback. Additionally, I think you're greatly underestimating the practical problems involved in operating a remote emergency-on-call driver system, there's also some questionable economics around the idea of paying to keep someone on call when you could just pay them to drive the truck.
On top of all that there are various other hard problems that need solving like fueling and security. You can be sure that automated trucks hauling valuable cargo will find themselves regularly waylaid by sophisticated criminals who will be greatly encouraged by the lack of a human element to resist their efforts. Truck drivers don't really get paid that much money, the risks just aren't worth it.
reply