Part of the problem is accounting. Since the value of a battery pack is highly variable depending on its remaining life, the only way it would work is if you subscribed/leased the battery so it didn't matter what the delta in value of the packs are.
Without that, you have to return to the exact same swapping station and pickup your original battery.
You missed the key point of swapping batteries. You don't own the battery nor did you pay ($16K) for the battery. So there is no "original battery". I assume that in ten years we will have "standard units of power" - the equivalent of a large "D" cell battery.
That is an option I kind of outlined as leasing/subscribing to the battery back are the easiest mechanisms to do this.
If the car owner only charged via swapping you could do a system where the cost of the pack was part of the electricity cost (eg: like a propane/co2 tank swap). The downside to this specific one is that you would have to either prevent charging or charge a fee if they did not swap in a specific time period as the battery is a significant capital expense.
Swapping is getting more unlikely as car makers are switching over to structural battery packs which makes swapping the pack non-trivial.
reply