Imo the problem mainly lies in Google's square. All SEO tools exist for the reason of it being possible to game search results in the first place. It's inevitable that companies will offer tools and services to do so when there's such a huge market for it.
Google has gotten better at discerning between crap and quality content, but they still have a long way to go, and I'm unsure if it can be fully fixed the way search engines currently work.
To me this isn’t a problem with Google, but with SEO. Gaming the system to get a better ranking is an entire industry now, and that won’t stop no matter which engine is the most popular.
IMO that particular aspect is what made Google stand out to the general audience. Most people don’t search by keywords, or optimize their queries to hint the engine. They just type a question and assume Google will make sense of what they meant. Google usually does that very well, and that high level understanding is not compatible with our keyword query expectations. To us that feels broken, but to the majority of people it’s working as designed.
Instead I think search results have been getting worse for everyone because of SEO. Companies want to optimize for number of ads viewed, not quality of content, thus quality goes down in favor of clickbait and keyword stuffing.
I don’t think there’s much Google can do here to resolve this issue. It’ll always be a game of cat and mouse between Google and companies using SEO to push more ads for less money.
It's not entirely Google's fault. SEO-oriented content is ruining the web. It's a battle I'm not sure search engines can win.
Keyword indexing and PageRank worked for a while because the underlying data wasn't written trying to game them. Then came spam linking and keyword stuffing and more. I'm surprised search engines are still useful in spite of that.
Disclaimer: I work for Google. I do not work in search quality (or on search at all). Below are strictly my personal views.
I can't (and won't) comment on the specifics of this change. In fact, my knowledge of them is pretty much what everyone else's is.
What I will say is that in the last few months there have been several stories about the quality of Google search results in the last few months (eg scraper sites rating higher than the original).
The problem I see with such criticism as leveled with those episodes (and this one too) is that there is the implicit premise that Google's search results and algorithm are static. This most recent change should be evidence of that.
So while Google's search is algorithmic the people who are in charge of it are not. To put it another way: if you try and game Google's system, it will possibly work for a time but at some point, when the problem is viewed as being of sufficient severity to warrant attention, that algorithm will change.
Search, as I see it, is an arms race. SEO, particularly black hat SEO, is on the other side of that. But this isn't as simple as SEO. The world changes over time too. New business models form. New memes come into existence (eg the idea of social search).
So let's assume for a second the OP's argument is sound and that Google has merely killed off Demand Media's competition. If true, there are now a lot less content farms on highly ranked pages than there used to be. Sounds like a win to me. Is it a perfect solution? No. But is it better? Absolutely.
Google's mission is to deliver quality content to it's users. The more people use our properties, the more money we make. We are very focused on the user experience. Gaming our system is, at best, a short term proposition as there are an awful lot of bright and talented people here constantly striving to defeat such attempts.
> This, in turn, leads Google’s suppliers to work to make Google better — what is SEO but a collective effort by basically the entire Internet to ensure that Google’s search engine is as good as possible?
Feels pretty confused to me. IMO, crappy sites with good SEO make Google worse for lots of queries.
Google is in such a hard place. I unwillingly came across a SEO conference while I was vacationing and every single SEO practitioner is using AI tools to fill the web with articles and low quality rehashes while using Google's inability to punish them while not punishing Forbes and the like (while also publishes low quality articles at times). I honestly don't know how Google is going to solve this one and in another decade how will things look like.
Originally, Google was very effective at looking at the interconnections of content people had put online and using that to infer which pages were most relevant. SEO tactics immediately started gaming this system to create false signals of relevance, but for many years Google did an impressive job of staying ahead of that game.
I think what finally killed their search quality is the fact that there's no longer a public human-curated network of websites to draw meaning from. Most content on the web is bulk-generated crap, personal blogs and websites are rare, and many passionate hobbyist communities are hidden from crawlers in places like closed Facebook groups.
I think the problem is just that the solution isn't in Google's wheelhouse: There is no algorithmic ranking system that can't be gamed. Human moderation and curation is the only way to provide true quality, and Google is allergic to solutions that don't automate and scale.
I think a really good search engine would still algorithmically search it's index, but the content library should be human-curated with a goal of ingesting content via author, not via platform. Once a given author was human-approved as a quality source of information, content they produce could be automatically ingested going forwards, and conditionally re-reviewed by a human if there were reports the quality had decreased.
To me the main way of determining search engine quality is whether or not it can find pages that I know exist because I've read them recently (they're still in my history) based on the terms that to me make sense for that page.
By that metric there are no good search engines at the moment and the older the pages the worse this effect gets. It's really nice to see Google do lots of 'moonshots' and interesting tech demos but I'd be far happier if they fixed search and kept their focus on that.
If a page doesn't show up in either Google or Bing for sensible queries then that page effectively ceases to exist. The perverse incentive that these companies have to avoid you going to a page with relevant results as long as you spend more time on pages with their advertising on it ensures that more and more content will end up missing in action.
Many people have been upvoting my comments here so the evidence I am seeing suggests that I haven't lost too many friends here with this line of thinking.
Continuing out the logic of your suggestion here, you would seem to imply that every other currently existing search engine besides Google also has bad quality as well in this regard. I say this because if other search engines had better quality that Google, they would be doing better than Google as per your suggestion. But, obviously Google is on top in a big way, so I have to ask, do you think the entire search engine industry, not just Google, has poor quality search results? What do you know that they don't?
The linked tweets imply that this decline in search results was Google's choice, led by desire for further monetization or exec incompetence, but I think Google is simply facing an impossible task.
Receiving an arbitrary question and finding the most helpful site for that question out of the entire web is already nearly impossible.
Now consider the above problem, except the sites you have to search are highly adversarial. More precisely, the internet is roughly divided into people who post useful content and have little interest in SEO and those who only care about SEO and clicks and not about creating useful content. The latter are more motivated and have more resources. For every useful site, they can take that site and create 100 of their own copies with the same content, more aggressive SEO and their own ads.
How is Google, or anyone else, supposed to navigate this landscape?
The whole reason google spends billions on their search engine is that humanity does not yet have a program that can differentiate fake relevancy from relevancy, with perfect accuracy.
The venn diagram between SEO and user satisfaction is gradually being compressed into a circle by Google as they improve their algorithm.
SEO is already basically human oriented now- anyone selling mumbo jumbo SEO magic now is a crank. It used to actually work quite well.
Agreed. However, Google has a similar problem with SEO and sponsored results. I cant be sure anymore that I actually get what I was looking for. In a sense, thats the same problem.
Garbage websites hyperoptimizing for SEO have existed since the late 90s. I agree with the GP, the issue I have seen over the deterioration of search in the past 5-10 years is specifically a result of their business model:
1. Any remotely commercial search has an entire first page of ads, organic results are pushed way down.
2. Google has made it difference between ads and search results as minimal as possible. I long for the days of the early 00s of big yellow boxes.
3. On many pages the amount of content Google stuffs in at the top before you get to actual search results gets more annoying every year.
Honestly, I wish I had a button that made Google result pages look like they did 15 years ago.
Google's major problem is that it's in an adversarial relationship with the SEO industry. For any given keyword there is a lot of money to be made by being in the top 3 results, which means the top results tend to be dominated by businesses with the resources to mount a focused and persistent SEO campaign. This leads the results to become saturated with content marketing rather than "organic" content.
Have to be honest, I'm surprised the article wasn't about SEO. That gets a lot of blame for ruining the internet, especially on tech sites.
But Google's propensity to reward sites/pages that are popular or new rather than those which are actually more accurate/better in terms of quality is definitely an issue.
This pretty much sums it up. The amount of crap and plagiarized content in most searches is what is killing most search results.
Google makes money regardless of the quality or 'originality' of the content your search comes up with so they currently have no motive to change things.
As I understand it, the problem with Google's search results is that they made SEO the primary determinative factor and didn't publish how their page ranking system worked.
However, it seems that businesses have figured out how the system works, and since there is big money in being the #1 result, people are constantly cramming every technique they can get into their sites to get to that #1 result location, even if it means that humans can no longer find what they are looking for.
Google needs to rework their search algo or they need to die.
Google has gotten better at discerning between crap and quality content, but they still have a long way to go, and I'm unsure if it can be fully fixed the way search engines currently work.
reply