I applaud anyone willing to look back and reason about what we've lost.
In this case, I prefer leaving the past in the past. Startups used to need to hire dedicated DBAs, and these days they largely don't. Scaling business logic that lives on horizontally-scaleable hosts is markedly easier than scaling business logic that lives inside a database. Debugging, zero-downtime deploys, and logging are all complicated by putting logic in the DB.
Unless latency is far-and-away your highest concern, I don't recommend using these features.
I'm obviously being hyperbolic, not sure about you. Point remains that scaling RDBMSs is not exactly trivial and it does look like most companies eventually give up.
I wonder how many people in the NoSQL and SQL doesnt scale crowd either have never met a truly competent, much less good DBA (trust me, they are very very rare) or decided it could not scale because they applied their programmatic and procedural logic to a tool that operates in a very different (SET based) paradigm.
Amazing! But why does this feel like such a de ja vue all over again.. (surely I'm missing something).. They've spent 5 years telling us that we just CAN'T scale SQL.. Now they'll tell us that actually.. they've figured it out! :)
It's not so much the DBMS that doesn't scale. It's particular data access patterns in combination with particular requirements that don't scale. RDBMS alternatives do little more than educate me about that fact
For every 1k companies who think they'll be the next Amazon, probably 1 is. Program to reality, not ego. Get walking down before you run.
Normalize your database in a practical way and you should be able to scale fairly well into typical growth. Bad database designs are the most common culprit I see.
You're refusing to learn from someone who lived it. I also know of zero services which failed because they couldn't scale their technology. But I know of 100s that failed because they couldn't get enough users or usage.
Your example about the DB tables is exactly the trap to avoid while you are iterating rapidly to try and find product/market fit.
I'm not sure it even makes sense for smaller platforms any more. The metric of most business success is really growth. Tying yourself to something which is troublesome to grow because it favours doing things that are difficult to scale is an evolutionary dead end. I've spent the last 25 years constantly battling this problem in relational databases. Moore's Law has been pretty kind so far assuming you have the money to buy ridiculous computers and licenses.
I am out of touch of what is available in the NoSQL side of things but about 5 years ago there were too many things which had a terrible failure mode somewhere in the stack thus I'm never sure which way to go instead.
No application, especially enterprise applications can have all business logic in the DB. It is spread out over huge, _multiple_ application code bases & even shell scripts and ad-hoc jobs.
So it doesn't matter how "good" the DB is. If the App & presentation layers are so bad, that they introduce logical or other anomalies every so often, does it matter if the DB does not produce any error in a billion years?
The Porsche analogy is cute. But if the business case could be solved by 10 scooters, isn't it be better & cheaper? When your "Porsche" is down, all customers are affected. With 5 scooters down, only 50% can be (if you design it that way).
In many of these so called "enterprise" applications, what they _needed_ was 10-20 scooters, but they bought 2-3 Porsches. And it's usually because "no one gets fired for buying IBM". In many internal apps, companies literally are putting Wordpress like shitty CRUD CMS on Oracle because of this mentality.
Why is that? It's because we don't learn from the past and keep thinking we're the smartest and have to reinvent everything with every new generation of developers.
I've worked at a company where elasticseatch, postgres, message queues, custom microservice platforms, custom cicd platforms were deployed for a load that could fit in ram of one machine. With maybe 10 customers per minute.
And you're wrong about accounting, new laws and regulations are being passed constantly and one has to keep up with them all the time
reply