One of the benefits of our new storage engine[0] is that it'll be much easier for us to host it in any datacenter, rather than just the biggest, best-connected ones. We still have a lot of work to do to make this available to all durable objects and actually start utilizing smaller datacenters this way, but we're working on it.
It's going to be so good to finally have an open source option for large storage building blocks. It's of particular interest to those of us in HPC as if you are building a large parallel file system and you don't want re-build times to suck your options are basically one of the following.
A few months ago, Frank Frankovsky left Facebook to start a startup based on this tech! He was previously "VP of Hardware Design & Supply Chain Optimization" and has also been super involved in the Open Compute Project.
I'm pretty excited to see what they produce, and hope they can ship before someone like Amazon tries to acquire them. This level of cold storage is going to be critical as the world generates exponentially more data every year. He also mentioned this is an open source venture, which is doubly awesome. :)
I've been using this since the very early alpha status. It's a really nice storage solution with a very sophisticated technology too.
For anyone curious about the technology behind here is an early tech talk from one of the founders. It may be a little bit outdated now but still interesting: http://www.youtube.com/watch?v=3xKZ4KGkQY8
There will be another blog from us at some point about running the data warehouse at scale. We're already working on integrating with s3 storage, and distributed compute is in the roadmap. Both possible today with open source extensions, and our friends in industry are already doing it.
(Blogpost Author) Yup. We've been thinking about colocation for a while, so we've just been building these up. Basically all that's left is to make our volume storage bulletproof. We'll do that as we're moving stateless workloads to bare metal early next year, and ideally be off GCP EOY latest
Finally it's out! @WavefrontHQ managaes petabyte scale clusters with #foundationdb today!
reply