There's a case to be made that detecting and penalizing SEO is making results worse, if, as you suppose, SEO for a sufficiently good search engine is equivalent to producing good content.
Obviously Google doesn't think their search is good enough, and I would agree -- piling more "inputs" and arbitrary branches into a ranking algorithm, however, is no solution. This will only devolve into an endless game of cat and mouse until a new search engine comes along and does to Google what Google did to Yahoo.
I agree that SEO has made it harder and harder to find good content on the internet.
I don't agree with Google being blamed for this. They're trying desperately to fix the problem. Maybe they're not as effective as you'd hope, but why would a "multipolar" search world be better? Wouldn't all search engines be plagued with people trying to game the system?
Imo the problem mainly lies in Google's square. All SEO tools exist for the reason of it being possible to game search results in the first place. It's inevitable that companies will offer tools and services to do so when there's such a huge market for it.
Google has gotten better at discerning between crap and quality content, but they still have a long way to go, and I'm unsure if it can be fully fixed the way search engines currently work.
This seems a bit backwards to me. If you have the best website for a subject, and Google doesn't rank it, the problem is not that your SEO is bad, it's that Google is failing. You have the best website for the query and Google's job is to find the best websites for a query. If Google fails to do that, that shouldn't mean you need to do more work. It should mean Google needs to make some kind of change.
Somehow humans are very good at telling apart SEO crap from legitimate content even without understanding the content or the language itself - SEO crap has some common elements such as ads, affiliate links, a certain page layout, etc.
I remember using an open-source ML model (trained on Buzzfeed article titles) to detect YouTube clickbait based on titles and it worked brilliantly, and that was just downloading some code on GitHub and running it as-is. I'm sure the same could be applied to search results and you could achieve much better quality if you actually put some effort into it.
I very much doubt this is some kind of hard problem as opposed to Google just giving up because their business model doesn't actually incentivize good search results.
As I understand it, the problem with Google's search results is that they made SEO the primary determinative factor and didn't publish how their page ranking system worked.
However, it seems that businesses have figured out how the system works, and since there is big money in being the #1 result, people are constantly cramming every technique they can get into their sites to get to that #1 result location, even if it means that humans can no longer find what they are looking for.
Google needs to rework their search algo or they need to die.
There was a very good thread on HN about this not long ago(1). Google search is getting worse because it is letting companies like Pinterest game it.
Instead of fixing the spam they are instead encouraging companies to spend more and more time on SEO and coming up with their own shenanigans like better ranking for using AMP (defunct now).
People who generally make great content (think a researcher or a great software maker) can't compete with billion dollar companies like Canva, Shutterstock and Pinterest who spend millions of dollars on SEO and have dedicated SEO employees who spend all day sending outreach emails and doing experiments. Henceforth the good content never even sees the light of the day; drowned by all this "SEO" optimized content.
FWIW i still believe it's the job of the search engine to find great relevant content and show it to the user instead of the other way round. Though I know it's much easier said than done.
For decades now, SEOptimizers have constantly found new tricks to get their websites ranked higher than they should, while search engines have constantly updated their algorithms to make them harder to game like that.
These two things have co-evolved in response to each other, and what this article laments is the result of exactly that process.
But the interaction between SEO and Google search is way more complex than the article implies. In particular, the influence goes both ways, and it's not a clear one-way cause and effect relationship. Also playing into this are a lot of other factors, like the increasing commercialization of the web, how the online advertising business works, Googles dominant position in both web search and web advertising, etc...
I really don't think the solution to this can be as simple as "change how Google rewards keywords". It's not that simple.
Lots of people in this thread claiming that google is slipping, focusing on the wrong metrics, etc. But I agree, google is losing the battle against SEO and its a really hard battle to win. Eventually there will need to be a shift away from search as we know it to win this game.
Disclaimer: I work for Google. I do not work in search quality (or on search at all). Below are strictly my personal views.
I can't (and won't) comment on the specifics of this change. In fact, my knowledge of them is pretty much what everyone else's is.
What I will say is that in the last few months there have been several stories about the quality of Google search results in the last few months (eg scraper sites rating higher than the original).
The problem I see with such criticism as leveled with those episodes (and this one too) is that there is the implicit premise that Google's search results and algorithm are static. This most recent change should be evidence of that.
So while Google's search is algorithmic the people who are in charge of it are not. To put it another way: if you try and game Google's system, it will possibly work for a time but at some point, when the problem is viewed as being of sufficient severity to warrant attention, that algorithm will change.
Search, as I see it, is an arms race. SEO, particularly black hat SEO, is on the other side of that. But this isn't as simple as SEO. The world changes over time too. New business models form. New memes come into existence (eg the idea of social search).
So let's assume for a second the OP's argument is sound and that Google has merely killed off Demand Media's competition. If true, there are now a lot less content farms on highly ranked pages than there used to be. Sounds like a win to me. Is it a perfect solution? No. But is it better? Absolutely.
Google's mission is to deliver quality content to it's users. The more people use our properties, the more money we make. We are very focused on the user experience. Gaming our system is, at best, a short term proposition as there are an awful lot of bright and talented people here constantly striving to defeat such attempts.
To me it seems the problem with Google goes far deeper than struggling with bad SEO.
- For years it has been next to impossible to get a result that is faithful to the search you actually typed in. This is not dependent on SEO spammers at all, only on Googles unwillingness to accept that not every user is equal and some of us mean exactly what we write, especially when we take the time to enclose our queries in double quotes and set the "verbatim" option.
- Ad targeting has been so bad it has been ridiculous. Yes, on average it works but around the edges it is somewhere between tragic and hilarious. For ten years after I met my wife the most relevant ads Google could think of was dating sites. Not toys, not family holidays, not tech conferences, not magazine subscriptions, not offers from local shops, but scammy dating sites that was so ridiculous that I cannot imagine how most people would fall for them. (For a while I wondered if this was all a fluke but now I have confirmed it happens to others in my situation as well.)
- Also in other areas it is becoming ridiculous. For example: what is the idea behind aggressively showing me captcas while I'm logged in with two different google controlled accounts, one gmail and one gsuite, both paid?
To me this isn’t a problem with Google, but with SEO. Gaming the system to get a better ranking is an entire industry now, and that won’t stop no matter which engine is the most popular.
The problem is that Google actually aren't good enough at search yet.
They still rely on this dumb word-based approach to document retrieval. Example, if my page is about how to manage your time, and it's a really fantastic resource about that, but I don't actually mention the phrase "how to manage your time" anywhere, I won't rank for people searching for that phrase. I should, but I won't.
So I have to write my content for two audiences - humans and Google.
I don't want to do this. I'd be much happier just building great content for my human readers, and if I happen not to mention the exact keyword phrase a searcher might use, it doesn't matter, Google still knows it should put my site at the top for that phrase, because it understands that my site is about that phrase, even though I don't mention it exactly.
But Google just isn't good enough yet. Someone can set up a page which has a bunch of headers and URLs and variations of the keywords and beat me to #1, even if their content is utter rubbish for human consumption.
That's why SEO still exists. It's symptomatic of a bug in Google.
This penalty will help, hopefully, but we're still going to be in a state where we have to compromise our content to serve two masters.
Unfortunately, the sad truth is likely that Google has hard telemetry evidence that users in aggregate prefer this SEO junk, engage with it, and move on feeling that Google successfully satisfied their query.
If Google could drive higher levels of engagement and satisfaction by screening out the dross I think they would, it’s clearly within their abilities.
Hopefully AI gets good enough and scales efficiently enough so they can disaggregate this kind of ranking decision.
Google's major problem is that it's in an adversarial relationship with the SEO industry. For any given keyword there is a lot of money to be made by being in the top 3 results, which means the top results tend to be dominated by businesses with the resources to mount a focused and persistent SEO campaign. This leads the results to become saturated with content marketing rather than "organic" content.
Disclaimer: I work for Google, but not on search. These opinions are my own and based only on the linked article.
I took a beating on the SEO blogs for calling SEO a bug a few months ago, but I'm glad the rest of the world is finally realizing that I'm right :)
Ultimately, Google is trying to rank you highly for providing the best content; you shouldn't be spending your time trying to figure out how to game Google by making superficial changes to the presentation of your content. Want to rank better? Write better!
The whole problem is rooted in the fact that Google is a leaky abstraction. It tries to be an omniscient Sherpa, guiding the wary Web traveler with its infinite understanding of the Web and the individual user's needs. The reality, though, is that Google is actually just a computer program. So there is a gap between the user's mental model of Google and what Google actually does, and it's this gap that SEO exploits for its own profit.
An infinite amount of exploitation would mean that Google would just return results randomly, and so it makes a lot of sense to detect signs of SEO and penalize the behavior before it further broadens the perfection/reality gap. Gaming the system is currently profitable, since the worst thing that can happen to you is nothing, but the best thing is that you get more traffic. A penalty aligns the risk/reward spectrum to favor "write better content" rather than with "spam a bunch of wikis".
Obviously Google doesn't think their search is good enough, and I would agree -- piling more "inputs" and arbitrary branches into a ranking algorithm, however, is no solution. This will only devolve into an endless game of cat and mouse until a new search engine comes along and does to Google what Google did to Yahoo.
reply