Personally I go back and forth on whether the hostile, aggressive gatekeeping is part of why stack overflow is failing, or is part of what kept it functioning as long as it did. Probably both. Both is good.
But this one terribly accurate line included in the alternatives to SO is worth the whole price of admission:
> ...you can even go to ChatGPT, where it’ll give you a confidently wrong answer that looks so correct that you’ll spend another 7 hours debugging why your code doesn’t work.
> No rigid format or gatekeeping like stackoverflow.
What bothers about gatekeeping? I could guess, but I'm asking so you say it out loud. Then you can compare it against other problems, such as moats (competitive barriers).
OpenAI spent something like $3M on training GPT-3. This is a pretty big moat. But almost certainly more valuable in dollar terms is the first-mover advantage which provides millions of human eye-hours used for RLHF.
I wouldn't be so eager to trade the gatekeepers you so fear for even an openly available chat service that is happy to automate away as much information work as possible.
The Stack Overflow model is (was) pretty darn good -- people help each other out, the company made money, some people got noticed for their skills, products got build faster and better (on the whole, I hope). Contrast the human-generated content era to what we have now which appears to be the machine-ingesting content era. There are legions of lawsuits against companies scraping data without permission and/or attribution.
Stack Overflow is a casualty of its hostile atmosphere, of the elitism displayed by its most powerful members, of its past community moderation and advertising scandals, and of its stubborn refusal to implement features long requested by users.
ChatGPT is just the executioner. If people are abandoning ship in droves the moment an alternative becomes available, you know just how bad your platform is.
I hear this strawman all the time. The first thing you read when you go to stackoverflow.com is:
> Find the best answer to your technical question, help others answer theirs
And here's one of the first snippets on their about page:
> Stack Overflow helps people find the answers they need, when they need them. We're best known for our public Q&A platform that over 100 million people visit every month to ask questions, learn, and share technical knowledge.
Stack Overflow's website says nothing about this when you're looking at it briefly. They highlight the fact that it's primarily a Q&A site. The people that insist that SO is _not_ a Q&A site are needlessly defending the gatekeepers that have ruined the site.
Finally, it's funny that everyone always says that SO is supposed to produce high quality answers, because it stopped producing any high quality answers around 7-8 years ago. The new answers on the site suck. My usual routine is:
1. Google question
2. Click 5-6 different SO links
3. Get pissed off because the answers are all crap
4. Go to the documentation
5. Spend an hour to find the one small doc reference that actually gives me the correct answer.
I wish people would stop propagating this lie that gatekeepers are necessary for "high quality". It's a lie propagated by gatekeepers to defend themselves.
SO is at its best when it’s actual error debugging IMO. When you google some specific error whoever else has the similar error it’s right there. I feel like GitHub is replacing this more and more though - I often get the GitHub ranked specific error higher than Stackoverflow these days. Usually you get better discussions on the GitHub issues too, for a multitude of reasons. Two off the top of my head:
1. all of the people working on the stuff related to the issue are very close by
2. the moderation is not nearly as heavy handed as SO.
ChatGPT is also much better than SO as well if you can give it enough context and the thing you are working on wasn’t built on stuff released after 2021
I also really like Stackoverflow for current event type stuff, like black swan type events. One recent example is when google’s Paris data center was on fire and infra guys were helping each other out trying to get systems online.
All of this combined means that StackOverflow the forum is probably on its way out though. They made the mistake of taking VC money and the model hasn’t really proven profitable so they have really made some poor decisions to please the vc overlords.
I won’t miss Stackoverflow much other than nostalgia unfortunately - better alternatives have arrived. Seeing the decline of all of the other Stackexchange sites kind of sucks though. There aren’t better alternatives for many of those
>I largely treat ChatGPT as my personal Stackoverflow. Except I get to break all the stackoverflow rules.
This is the sweetest thing about ChatGPT.
With Stack Overflow, if I ever have the misfortune of needing to ask a question, I feel like I'm a contractor with some seriously constraining contractual obligations. I feel like I'm serving them rather than them serving me. I have to perform an exotic dance through a field of eggshells just to get what I'm after.
With ChatGPT, I feel the opposite. I feel like it's really serving me. It's so refreshing to just be able to ask any kind of question, any follow-up questions, no strings attached.
Even beyond Stack Overflow, with forums and message boards in general, I no longer have to deal with moderators locking my post, users reprimanding me for not using their atrocious search feature, no whining or rhetorical judgements "why would you even want to do this, you stupid idiot?"
That said, the effectiveness of ChatGPT wouldn't be what it is without Stack Overflow question martyrs and their answerers.
It seems like you are pretty invested in Stack Overflow. At best I got marginal use out of it so I don’t care for the site one way or another. For most questions where I’d use Stack Overflow (dev stuff) for I simply get a just as good of an answer or hint much faster using Chat-GPT so much so that I just use that to start with, or if it’s a unique problem to an organization I work for I can’t use Stack Overflow really anyway because the question is too specific.
A flow in the past might have been Google “question content or error message or something” -> Stack Overflow links to browse through until something plausible shines through -> problem solved
Now it’s more like Chat-GPT (problem probably solved, or additional details to Chat-GPT) -> Google -> Stack Overflow
Even if tools like Chat-GPT were half as good as Stack Overflow they are vastly more efficient and since I know what I’m doing or looking for I can sniff out how plausible something is (no difference with Stack Overflow today).
I get that you have had a great experience on the site and truly it delivers value for a lot of people!
But I don’t see how such things keep the lights on at Stack Overflow.
The reason the OP is relevant is because even if you think you can identify the answers today it won’t be long before you can’t. It follows the principal that it’s easier to destroy stuff than build it.
If Stack Overflow wants to survive in any form it probably needs to Verify Human and eliminate any method in which someone can interact with the site (content-wise) except hands on keyboard.
Personally I’ve found the so-called human responses to be +- useful compared to Chat-GPT on any given topic I use it for and the only way I come to believe they’re human responses is because of the time stamp.
For example you say:
> …LLMs will pretend like they know what they're talking about and give you the run-around endlessly, providing random, plausible-looking nonsense for as long as you have patience to put up with it
I honestly think you can just plug in “contributors to most Stack Overflow questions” in to this sentence and it’s equally true.
You are focusing on the 1% of cases like in your example. I’m looking at most cases.
A quick look at pretty much any question on SO will explain that - between the "you're doing it wrong", "your question is worded 95% too imprecisely for me to help", "use something else", "[answer to entirely different question with overtones of patronisation and insult]" etc. answers, life's generally too damned short to bother with SO.
StackOverflow started with the vision of not being what made Experts Exchange so terrible - that was a brilliant plan at the time and SO did many things very right.
Now, many years have passed and not only have we seen SO grow into the whole plethora of QA sites that are Stack Exchange but also a couple of other QA sites. None of them are great, none of them are nearly as useful as the original SO was. I think it is save to say, that this approach simply does not scale.
I might be wrong and maybe, some day, someone comes around and nails QA at scale. My bet is that with ChatGPT the humanly curated QA site is as dead in the water as the humanly curated web directories were when Google came around.
StackOverflow gets a lot of people Copy+Pasting, and a lot of really bad answers that just don't apply at all.
Or sometimes I'll find the exact question I want on StackOverflow, but no answer!
ChatGPT always has SOMETHING for you. Rarely is it so unhelpful that I can't move forward, whereas sometimes StackOverflow just doesn't have an answer for me to even get started.
I've been a member for 10 years now. I racked up a few thousand points after I joined, before the novelty of gamification wore off for me. After a decade of steady usage, here's my take on Stack Overflow.
Stack Overflow is an excellent resource when you access it from a search engine. If you're looking for a particular bit of programming knowledge and you type your query into your search engine of choice, chances are excellent that Stack Overflow results will be plentiful on the first page and that more than one will be relevant and helpful.
Stack Overflow is good enough when you have a very concrete problem that nobody before has asked about before. For example, if you don't know how to do X in the framework Y, you can expect one of the following: 1) someone who did it before will tell you how, 2) someone who had the same problem and found that it actually has no solution will tell you that, or 3) nobody will answer.
Stack Overflow is absolutely terrible when you're trying to learn something new just because you want to learn it or you're asking for advice of any kind. If you're trying to become a better programmer, Stack Overflow is categorically not the tool for the job. The less practical your question , the more it's likely to receive votes to be closed.
My favorite example is a question that outlined a thing you can normally do in .NET and Java because they are executing bytecode and can do certain checks before deciding whether to allow the code to execute or not. The question then proceeded to ask whether there are any pre-existing solutions to do so when working with native code or whether you would have to do it yourself. It received a downvote and a vote to close within 2 minutes of being posted. The voter posted a comment recommending Software Engineering Stack Exchange as a better site. The same question posted on Software Engineering Stack Exchange got a bunch of comments, only one of which actually addressed a question to a certain degree, the rest being ideas on how to change the requirements, snarky sniping of other comments, one idea that was already outlined in the question, and one snarky recommendation to "brush up on your understanding of [concept]" where the [concept] is not the primary concern in question.
I wish I could say that last anecdote is an exception, but it's simply the most egregious example of what's prevalent on Stack Overflow and programming-related Stack Exchange sites. Interestingly enough, I haven't seen it on any non-programming Stack Exchange site, so it's definitely something in our culture as an industry.
I really don't blame the author, here. It's uncanny how many negative experiences I've had with Stack Overflow. There are occasionally posts where I notice they have this heavy-handed brand of authoritarian moderation and adversarial approach to problem-solving that I find really frustrating. I've since stopped using it. I generally find more value in first-party documentation, the source, and GitHub issues, anyway.
> Stack overflow is caustic so you don't want to ask there.
I have only positive experiences with asking on Stack Overflow. Nine times out of ten I have gotten a useful answer. In one case, an answer provided a 20-line code snippet, which would have taken me days to figure out on my own.
The key is to ask clear and focused questions, and explain what you have already tried and why it didn't work.
Sure, there are always some answers or comments which are unhelpful or which completely miss the point of the question. But the value of a QA forum is given by the value of the best answers, not the worst.
This explains the success of StackOverflow: don't have to wait on your colleagues or even your own investigative work, the answer is usually a Google query away.
This also explains the success of ChatGPT in dev work: StackOverflow is still too much bullshit to deal with. LLMs strip that all away. It's such a big difference it's worth risking being led astray by hallucinations.
I encounter situations where I used to rely on Stack Overflow, which carries a similar (if not higher) likelihood of being incorrect, often due to outdated information.
For instance, I was recently inquiring about a specific task with CMake and consulted ChatGPT. Initially, the response was inaccurate, but it was obviously so when it didn’t compile. Upon reprompting, I received the correct answer.
I wouldn't say StackOverflow is especially beloved by developers. Coders on X/Twitter used to complain about how much they dislike SO all the time; I see less of those now, probably because they've switched to using ChatGPT. When I've seen blog posts or headlines about them in the past 1-2 years, they're usually about how "StackOverflow is dying."
I can definitely see why StackOverflow is suffering. Why would you ask questions there, when 98% of the time your question will be duped to an unrelated question or some overbearing fuckwit with half a million karma will undermine the premise of your question as though you don't know what you are doing? With chatbots you can query a free, compressed representation of StackOverflow, without the terrible attitude.
That's a fair take. I can't overstate Stack Overflow has been in the past, I think I often had that experience that it usually caters to common but specific scenarios, or I'm just impatient and lurk instead of asking a question and waiting for a response. Interesting I could paste Stack overflow discussions into ChatGPT to ask if this insight is related to my issue and it was usually open to relating the new info and helping me
No, StackOverflow is quite a reliable source of correct and relevant answers to programming questions, with effective mechanisms for the community to identify and fix wrong or unhelpful answers.
It won't remain that way if those mechanisms are DoSed with a firehose of free plausible garbage. The flippant attitude that what we currently have is no better than that firehose, and the implication that it isn't worth worrying about or attempting to protect, is starting to grate.
But this one terribly accurate line included in the alternatives to SO is worth the whole price of admission:
> ...you can even go to ChatGPT, where it’ll give you a confidently wrong answer that looks so correct that you’ll spend another 7 hours debugging why your code doesn’t work.
reply