True, but there are a lot more applications and use cases for an open-sourced redis than an open-sourced twitter. Let me try again: Twitter was built on an open source foundation, changed the way people communicate forever, and created a huge amount of liquid value. Without open source things like Twitter probably would not have come into existence. Twitter has in turn contributed a lot of important open source software back into the community. Elasticsearch or redis, by contrast, enabled many of these cool new things to exist, but by themselves those excellent software components changed very few lives. If you made redis and tossed it out in the world with a ta da! very few people would care. I always felt that open source was an explicit recognition that we care about and attach value to the big ideas that change the world, and that by making the building blocks freely available we caused more of those big ideas to come to fruition. Now that the big ideas are perhaps a little harder to find we seem to want to attach a price tag to the parts again.
I'm always amazed to see the degree at which open-source is used at Twitter. While it might be possible that other big players have a similar number of open source projects, they certainly don't seem to advertise this aspect. Something I expect will help their engineering recruiting quite substantially.
I agree with this. I think that even products that aren't core to a business could be open-sourced. Twitter is a great example of open-sourcing code that is not critical to their core business. I think it's a great way to give back as well as helping your company get notoriety.
Open Source has generated $Trillions in value creation over the last few decades.
Twitter is fighting to stay relevant, they dropped to Top 35 website from Top 14 in last several months.
This is an opportunity for them to become a global standard, and that is worth it.
The most important piece is that they can scale when they pull it off. I've been working on decentralized tech for a while (I run https://github.com/amark/gun) and can say it takes a lot of attention that most projects (dare I say "blockchains") have no scaling abilities at all.
FOSS is fundamentally about freedom--the freedom to control the software you use.
Even if LinkedIn or Twitter were open source, I would still not be able to control them--I don't have a spare server lying around and I certainly don't have a way to get everybody else on my own system! So if I took LinkedIn's source and improved it, it would be useless: I would first have to get a server to run it and then I would have to get everybody I know to switch to it. This means that, even if LinkedIn was open source, I would still have to use their implementation on their servers; it would not have much of a practical effect.
Put another way, Reddit and HN are open source. But this does not mean I can meaningfully modify or control the code; this control is naturally left to the people with the servers. There is very little practical difference for me as a user between an open source Reddit and a closed source Reddit.
Now, this is not to say that web apps should not be open sourced--quite the contrary! Open sourcing is great for reasons besides freedom: you contribute to the common good, give more people insight into the development process and help other people with similar projects. But it is less critical than having open source programs that run on my own computer.
I don't mean to put down Twitter's contribution to open source – which is substantial – but I think you're underestimating Google and Facebook's contributions when you suggest that they are only "one-upping each other with ... open data center designs".
I'm more familiar with Google's contributions, but I would imagine both high-user-visibility projects like Android and Chromium and more developer-oriented projects like protobuf, the Closure Compiler, the Dart and Go programming languages, googletest, Guava, Guice, WebM/WebP, V8, Breakpad qualify. I could consider all of these "projects that are applicable to a large group of people". This, on top of large contributions to other key projects like the Linux kernel and LLVM.
I know less about Facebook's contributions, but I do know that their open source projects includes HipHop/HHVM, and they contribute significantly to Hadoop, LLVM and Mercurial (among other high-profile projects).
One of the reasons I worked at twitter is that they want you to open-source virtually everything. They weren't likely to put up a fuss, and everyone was really proud of bootstrap.
Twitter is also a leading silicon valley tech company with hundreds of extremely intelligent employees. You could easily argue that they may have one of the best engineering staffs in the world. Given that, of course they would use open source versus buying enterprise products.
In terms of engineering staff, they are the exception, not the norm. A company like home depot or kelly moore paints would have a tough time relying on open source for all their operational software.
There are two kinds of companies for whom OSS makes sense. The first category of OSS company is the data driven companies. Twitter, Google, Yahoo, and Facebook all fall into this category. Data driven companies derive their primary business value from the quality and quantity of their underlying data. They give away their software because it is free PR, endears developers to their platforms and it is a big morale booster for their engineers. Google will never give away its search index. Facebook and Twitter will never give away dumps of their graphs. Those things are the crown jewels, not the software that supports them.
The other kind of company that benefits from open source is companies like Red Hat, Basho and Opscode. Their open source offerings are loss leaders intended to stimulate enterprise sales. Mission critical systems get built on the open source software, and then managers can elect to buy enterprise features and support from the vendors.
As a long time Redis user, source code admirer and spectator
of its evolution, I have to say that I learned quite a lot
about open source project management.
The way this has come about with relation to web and internet software is that the actual applications being built are closed source but a lot of the libraries and commodity pieces of software are being open sourced with permissive licenses.
for eg. Twitter, Google, Facebook etc. produces and consumer a lot of open source software but their core products are still close source. You can segment out portions of your software stack and release them for free while still retaining a competitive advantage with your core application. This isn't an all-or-nothing decision and this hybrid approach is the most common.
I think Twitter is a good example. They both built and scaled up using open source and also contributed their own open source software, which has been improved upon by others and in-turn benefited Twitter. It didn't require a license where contribution was essential and nor did it require giving up any competitive advantage (i'd argue it solidified their position).
Yes, but open-source software adoption and growth is about more than just those who build it, it's about those who use it. Elastic wouldn't exist without the users either.
Actually there are relatively few real (TM) open source projects driven by the community, at least if you look at important projects. Many open source projects are just commercial projects driven mainly by a single company. Look for example at Redis, MongoDB, MySQL, and Elasticsearch. They follow exactly the model described in the article. Technologies like these could have been developed by a community, too, but it is hard to form such a community and keep it alive.
For a community-driven project from the size of a database some serious sponsors would be needed. Good examples are Rust, Linux, and PostgreSQL. I wonder why so many companies are happily paying Oracle (and the likes) tons of money instead of sponsoring an open source project like PostgreSQL.
In rebuttal I'll paraphrase a little from Paul Ramsey (maintainer of 20-year open source project PostGIS)[0]
His basic view is that Open Source is the dominant model today, but tension comes as very little of the value produced comes back to the community that creates this value. He argues this will always be 'the bare minimum' by virtue of economics, but that if something important slows down too much someone will put some money in it. But this is a model that operates and works. It is borne out by his history in postgis, which is maintained by a small number of people mostly in moderately-profitable service companies, in the red-hat mould. He's concerned about value being captured by cloud companies though who frequently don't employ open-source maintainers however. Some of this is further expounded in another talk by him here [1] (slides at [2]) on the future of open-source where he is very bullish.
We stand on the backs of giants. Yet, for some reason, corporations feel entitled to not give back. Modern computing lives by open source. Companies should be obliged to contribute similarly.
Companies like Twitter, Facebook, etc, regardless of what technology they invent that is of use for the public, often healthily contribute to the open source ecosystem. Twitter has contributed a lot to the Scala ecosystem, and Facebook to the PHP pseucosystem.
They also fund/maintain OS project they themselves use. Facebook, through their FriendFeed acquisition, obtained Tornado [1] and have maintained it very well since. I use Tornado every day.
I get your point but your anger is misplaced if you think the large "SV tech companies" don't "invent". You want tech companies that don't contribute anything to mankind? Look at purely closed source hardware companies.
reply