I was also running a news server for my school and an ISP around the same time (1994-1996). The alt.binaries hierarchy was a constant headache. The total data flow and storage requirements were already about 10X the rest of Usenet. We had to manually tweak refuse lists when the disk drives filled up.
I wasn't still around, but I expect the second wave of automated binary posting/retrieval tools in the late 90's probably created the surge that changed Usenet access into a separate paid service, no longer something that came with an ISP account like email.
However, it wasn't just warez and porn that killed Usenet. Spam took off there before email, and it wasn't until email spam became untenable that modern automated filtering tools were created.
I ran a Freenix-competitive Usenet server for a popular ISP in the mid-1990s (by way of bona fides: we were competitive because I hacked a history lookup cache into INN, a concept we apparently co-invented alongside Netcom). Usenet was by far our most expensive and most time-consuming infrastructure.
The reason for that was binaries. The amount of storage we were required to keep online for binaries was staggering. We ended up buying those ridiculous chrome-plated NetApp fileservers to handle the load. The hardware was expensive, but more expensive was the admin overhead: things went wrong with the INN filesystems regularly, and there was nothing you could do to recover from them quickly; simple filesystem errors that were really just an fsck away from repair could mean 4-6 hours of downtime. Which, by the way, tended to happen at night.
File-sharers sprayed multiple copies of huge files across several newsgroups, in little chunks. If any of those chunks went missing, our users screamed bloody murder. ISPs that tried to host no-binaries Usenet became the target of PR campaigns. Hosting discussion groups on Usenet was cheap and easy. Running a competitive full-feed server, on the other hand, required nearly full-time attention from an admin that could do light filesystem hacking.
The result was a death-spiral: as Usenet got more expensive to host, fewer ISPs hosted it; many outsourced to other providers. They could easily have hosted just the discussion groups! Usenet could have been kept alive and decentralized, and maybe even evolved alongside the web. Instead, software and pornography pirates coerced the network into a few centralized providers, who eventually decided not to waste huge amounts of money hosting infrastructure for those kinds of users.
Usenet died because of 2 reasons:
The burden of carrying alt.binaries
The burden of pirate media on distributed Usenet servers from alt.binaries
They made a great point of attack for the MPAA and RIAA member associations. And traditionally, ISPs provided these services. I think it was 2006 when Comcast finally pulled the plug. Most other ISPs doesn't even offer a connection at all, which is a shame.
Usenet never stood a chance. It was invented in the days when net access was something of a privilege that could be revoked if a user did not behave in a socially acceptable way. As such, it was acutely vulnerable to spam and abuse. What proportion of Usenet traffic consisted of porn and pirated binaries in the late nineties? Is it any surprise that ISPs ,fearing they might be held accountable for distributing such material, started cutting off their Usenet service?
The good news are: Usenet still exists. There are free NNTP servers that host most non-binaries groups. Many groups are abandoned, but some are quite lively. Every time I visit, I am struck by the fact that no website forum approaches the ease of use of a good threaded newsreader, and I am amused when I see each new generation of coders, ignorant of the past, trying to re-implement an incomplete, flawed replica of what once existed.
You mention the filter effect: I think what really killed Usenet, at least for me, was spam, much of which was automated, with no great mechanism in place to combat it.
As usual I am obliged to point out that what killed Usenet was software piracy. The amount of work it took to run a competitive news server with reliable binaries was unbelievable, easily the most expensive and fussy hardware we had at the ISP, and if your service fell behind or dropped any binaries, users would absolutely lose their shit: Usenet was an all-or-none proposition, so if you weren't going to buy a rack full of NetApp filers to run binaries you might as well not run Usenet at all. The protocol centralized before web interfaces made centralization palatable to users, and then died.
I don't think this was true. It was entirely common practice for an ISP to ignore alt.binaries but carry all the rest of Usenet fairly reliably. To the point where users would rebuke each other for posting binary content in a non alt.binaries hierarchy, lest the ISPs detect that as a stealth binary group and drop that too.
What killed Usenet was as the parent said - users are better served by the centralized management and reliability of a web forum, they don't need or care about decentralization and peering. The same thing happened for Discord and Slack replacing IRC.
What USENET? The one my ISP took away from me before I ever got the chance to experience it? The USENET that's today basically just a venue for piracy?
Maybe it's time to remake Usenet, minus binaries. Binaries, piracy, and their data load per server are what killed Usenet.
(Yes, I know it's still living on in paid-service world. But gone are the days your ISP runs a machine.)
reply