Does anyone have a design that works for this kind of adhoc meshing network with good privacy guarantees? It seems like a really hard problem to solve, especially the social graph problem because inherently messages will take time to propagate through the network based on proximity. Maybe adding random wait and hop count increments? Efficient routing kind of depends on being able to discover the network graph.
Ah, yeah, I was thinking of the initial connection.
I guess the mesh needs to be pretty small so that you can have a fully connected mesh or you'd need some way to deal with netsplits and a gossip-style protocol for discovery of new nodes, right?
The problem with hub and spoke is that the hubs hold the social graph and then become the target for censorship. I don't think there are any techniques employed by Matrix to mitigate that threat at this time (please correct me if I'm wrong).
In short, if you have a small network (under 10k nodes), I think P2P can work, but for networks of large scale (>10k nodes) I think you need hub and spoke, at which point the routing nodes in the center are the lynchpin. You can use some kind of mixnet tech to try to get around this, but that increases latency and computational overhead, thus lowering throughput. You can go the Signal route and throw the graph in an enclave, but there's still side-channel analysis (which is the thing that mixnets are trying to deal with, albeit I can't comment on their efficacy). Harry over at Nym has some cool ideas here.
I am not sure that spanning trees or gossip protocols solve the problem I'm describing, but, if they do, I'd appreciate if you could elucidate further.
Edit: Yes, I agree that N to N routing networks don't scale well. I think a K of N broadcast network can scale, but it's a tricky UX tradeoff.
One idea that I am fond of is Nym, an implementation of the high-latency mix network Loopix[0], but with incentives for nodes to act as mixnodes and message providers, using a model in which users pay nodes for their bandwidth. The goal being to provide strong anonymity in the face of a global passive adversary.
This was previously linked (e.g. [0]), but it didn’t get any attention, so I reposted it with a ‘more interesting’ title.
Hope that’s acceptable, because I’m interested in learning more about this network architecture. Is a vpn mesh — if embedded in the client — a possible backbone of a truly server-less and decentraized/distributed social network?
I wouldn't suggest that we need a social network, but your first suggestion isn't too far from the truth.
The exact thing needed is very boring, very low-level, and very unsexy, and I believe can be summarized thusly:
We need a method for locating hosts on a network graph which does not have a central point of failure and which cannot be easily disabled.
We then need a method for authentically routing messages back and forth between these hosts without fear of man-in-the-middle attacks that can change the contents of the messages.
This is pure transport-layer engineering.
As long as we can locate anyone connected to our network, and communicate with them without interference, we can build whatever else we need to on top of that.
We shouldn't confuse our efforts by trying to make a social network, or new hashcoin lottery, or advanced supergovernment, or whatever else. We shouldn't worry about interception of message contents--that way lies madness; as long as I know my message reached somebody in one piece, and as long as they know that a message signed by me is from me, we can fix the rest later.
This is a pure, straightforward, fucking hard engineering problem.
Yeah I love the idea of this kind of robust mesh. I think it'll happen eventually. Alternatively it could propagate until it reaches whoever it's going to on the limited network (your friend in a different neighborhood) and then send out a kill order for the other propagating packets.
I know a major issue with most mesh protocols is that when the network is both big and dynamic configuration traffic will start to swamp data traffic.
Another issue in most mesh protocols is susceptibility to Sybil, flooding, cache pollution, and other denial of service attacks by well-resourced adversaries. It's really hard to create a distributed permissionless system that can't easily be attacked in those sorts of ways. Proof of work and similar schemes are really the only ways we've found so far and those are expensive and cumbersome.
Any mesh network used by dissidents is going to be targeted for denial of service attacks.
Is that inevitable though? A graph of independent but connected nodes could theoretically scale indefinitely in the case where I hold my data and you hold yours. If I want to add someone to my network, I can either get their direct address or navigate through friends-of-friends (graph traversal). The only necessary delegation would be a mailbox provider, but the contents of any messages could be encrypted.
Not that any of this is easy to engineer, but in my mind it seems viable.
Hey, I worked in a related research field a few years ago and published some of the results. Here are some ideas I find important:
- Generally I found that overlay DHT based routing is not very scalable for large graphs. It performs well for random looking graphs, but becomes very inefficient for large planar-like graphs. If pinecone is designed for small local networks, it might work well, but it will probably not scale well to large networks with the current design.
The internet, which largely runs on IP and extended by NAT, appears to have been constructed poorly in terms of privacy. There are protocols like ICE/STUN/TURN (which is what WebRTC is built on) that supposedly try to bridge the gap to enable true P2P connectivity, but I believe even ICE appears limited in that it doesn't actually solve the hairpin problem, where two peers are behind multiple levels of NAT routers. I'm still investigating this issue, so correct me if I'm wrong.
What we need are better routing systems. Take a look at CJDNS which has a novel routing protocol. I don't know what the scaling limitations there are, there doesn't appear to be documentation on empirical or theoretical performance guarantees; but it's a start, and there's a growing community behind it.
With ever-cheaper devices, it should become possible to create a new family of router hardware that connects devices in a mesh fashion, scalable to the world at large. It's not going to be easy to design, and it sure as hell won't be easy to get physical adoption for (considering that the current internet is "good enough" for most), but it's possible.
P.S. I've created a subreddit at /r/fourthtech if anybody wants to get into deep technical discussions about various topics on better internet/communication protocols.
Yeah, a few people have been looking at how to do things like this too (I've been talking to them); the idea being mix-net vs. onion routing, which is the big win of being async, and sensible defaults. It's a lot easier to do confidentiality/integrity/etc. on messages vs. traffic analysis resistance vs. strong adversaries, though.
Lots of interesting stuff there - thanks :) We're using https://github.com/mwarning/meshnet-lab rather than imunes.net for network simulation currently, but will take a look.
Power usage is looking pretty positive so far; as long as we route the Matrix traffic over the routing topology rather than going full-mesh it should minimise radio usage (the main battery suck, other than screen).
For store-and-forward, honestly using P2P Nodes as intermediaries is an okay approach other than exposing metadata to them. Our plan in the longer term is to switch to loopix-style mixnets to obfuscate the store and forwarding, a la nym.
In terms of joining the network by deriving a private key from a passphrase... yup, that could be cute, although slightly terrifying in terms of the risk of weak passphrases :)
We're hoping to get the P2P network stable in the coming year (although we were also aiming for this year originally :P)
You'd need some kind of mesh network, rather than a tree network to get anywhere near enough reliability. I'd also guess that passing through a node might cause some slowdowns that kinda stack up.
There still is a major privacy issue with peeps that are nodes. And maintenance is going to be a bitch.
That said, those are all hypotheticals. Don't let my imagination stop anyone trying this.
He has been trying to make it extremely secure and anonymous, and all communications are end-to-end encrypted. To me it sounds like the ideal protocol for protection against censorship and spying on the users.
However, I don't know how well it can be combined with a mesh network, especially one that wants to piggy back on the normal Internet, since this protocol is meant for a stand alone network. But perhaps it can be modified for that. It would probably be best to talk to the creator of the protocol about it directly and ask for his help.
Aside from widely broadcast one-time pad encrypted/authenticated messages (which provides anonymity for the receiver, not the sender -- that can be bootstrapped), deterministic rotating asymmetric key pairs (say, a non-compromised curve ECC scheme using Key Families) coupled with a relatively-high non-deterministic latency mixnet delivery system using onion/garlic routing for intermix peers goes a long way towards achieving data security, authentication and metadata anonymity for all involved parties. Discovery and routing in such a system is a very complicated problem -- and it's one yet to be solved; though, generally if one is going to such lengths, one has a rendezvous system already in place.
The primary problem is that it's expensive for users -- expensive in terms of time, cost, and complexity; which means usage is low. Consequently, it also makes people targets simply for using the system as the number of those using the system is relatively small -- something one can't really get away from easily unless makes it look like one isn't using the technology. To do that, one then has to go through the effort of creating cover traffic -- and creating consistently good cover traffic (good enough to fool a human analyst, because one is one of thousands, not one of millions/thousands of millions) is immensely difficult and techniques change over time w/ local conditions so it's hard to automate. Life gets really really difficult when survival depends not only on keeping people out but also on keeping them from knowing anything of interest is there in the first place!
Don't forget about stylometry and inadvertent signatures; never communicate in real time, avoid absolutely everything but plain text data if at all possible, write very plainly (say, using only first few thousand most common words in your language) and use stylometry defeating tools (for example, Anonymouth -- though I haven't audited it; others have, but it's still just a thin layer to apply to other work) to prevent others from creating signatures based on the words you use and how you use them. NEVER forget that binary data you send might contain metadata fields to give you away (version numbers, encoding settings -- all seemingly innocuous but possibly unique to you!). Sending images? Make absolutely certain your camera doesn't have sensor glitches that can create a signature. ( see https://www.schneier.com/blog/archives/2006/04/digital_camer... ) Don't forget your surroundings either ( NSFW Language/Topic (4Chan helps track down targets) but incredibly illustrative - https://i.imgur.com/nLCklgZ.jpg ). Sending scans of documents? Most new printers print identifying patterns using steganographic techniques. ( see https://www.eff.org/issues/printers ). Stop and meet someone and both of you brought your mobiles? The fact that your phones traveled together, and where you went, is recorded (CO-TRAVELLER; see https://www.washingtonpost.com/world/national-security/nsa-t... ). The world is INCREDIBLY hostile to anonymity seekers.
Relying on third parties to strip data isn't a workable anonymity solution because you can't trust them to do so, correctly or at all. Not to mention that; but with pervasive internet monitoring (which, thanks to Snowden, we know is real) the mere fact that you've communicated with someone or a site storing your data is stored in a place where it cannot be wiped by any party authorised to participate in your conversation. Generally, if you're not anonymous to the person you're communicating with, until you choose to identify yourself during communications who's contents may then be repudiated (say, an olm/axolotl ratchet) at a later time by publishing that private key, now expired and no longer good for future authentication -- then you won't be anonymous to any party, period.
It's a trust no-one, verify everything type of situation; people don't deal well with that. Pervasive encryption is only the first and easiest step. If you want true (or even just reasonable) anonymity, things get very expensive, very quickly.
- Peer-to-peer proxy mesh network, no internet required.
- Automatically discover nearby nodes and connect directly via wifi, bluetooth, internet.
- Nodes run in memory, no writing to disk. Constantly discovering and caching indexes of common nodes.
- A way to store data fully distributed across the network. You only get a 'piece' of the data, you need to hit more than a certain threshold of nodes to get full piecture.
- A self policing of nodes from a ledger of some sort that is distributed and shared by everyone. Bad ones are not punished, they are grouped in their own 'neighborhoods'. Spammers end up spamming each other.
- Fully anonymous and risk free. You never hold more than a encrypted piece of the puzzle which alone have no meaning. Only with a certain threshold of nodes can you access that data. The data is constantly shifting across nodes.
- While a node is online, it's broadcasting it's piece of the puzzle to other nodes randomly so that if it goes down, the backup is held by another node.
I don't know I'm just writing some random ideas so obviously these will need additional work.
reply