{"id":26810,"date":"2022-12-31T12:24:33","date_gmt":"2022-12-31T12:24:33","guid":{"rendered":"https:\/\/mailinvest.blog\/index.php\/2022\/12\/31\/is-this-googles-helpful-content-algorithm\/"},"modified":"2022-12-31T12:24:33","modified_gmt":"2022-12-31T12:24:33","slug":"is-this-googles-helpful-content-algorithm","status":"publish","type":"post","link":"https:\/\/mailinvest.blog\/index.php\/2022\/12\/31\/is-this-googles-helpful-content-algorithm\/","title":{"rendered":"Is This Google’s Helpful Content Algorithm?"},"content":{"rendered":"

<\/a>\n
<\/p>\n

\n

Google published a groundbreaking research paper about identifying page quality with AI. The details of the algorithm seem remarkably similar to what the helpful content algorithm is known to do.<\/p>\n

Google Doesn\u2019t Identify Algorithm Technologies<\/h2>\n

Nobody outside of Google can say with certainty that this research paper is the basis of the helpful content signal.<\/p>\n

Google generally does not identify the underlying technology of its various algorithms such as the Penguin, Panda or SpamBrain algorithms.<\/p>\n

So one can\u2019t say with certainty that this algorithm is the helpful content algorithm, one can only speculate and offer an opinion about it.<\/p>\n

But it\u2019s worth a look because the similarities are eye opening.<\/p>\n

The Helpful Content Signal<\/h2>\n

1. It Improves a Classifier<\/h3>\n

Google has provided a number of clues about the helpful content signal but there is still a lot of speculation about what it really is.<\/p>\n

The first clues were in a December 6, 2022 tweet announcing the first helpful content update.<\/p>\n

The tweet said<\/a>:<\/p>\n

\n

\u201cIt improves our classifier & works across content globally in all languages.\u201d<\/p>\n<\/blockquote>\n

A classifier, in machine learning, is something that categorizes data (is it this or is it that?).<\/p>\n

2. It\u2019s Not a Manual or Spam Action<\/h3>\n

The Helpful Content algorithm, according to Google\u2019s explainer (What creators should know about Google\u2019s August 2022 helpful content update<\/a>), is not a spam action or a manual action.<\/p>\n

\n

\u201cThis classifier process is entirely automated, using a machine-learning model.<\/p>\n

It is not a manual action nor a spam action.\u201d<\/p>\n<\/blockquote>\n

3. It\u2019s a Ranking Related Signal<\/h3>\n

The helpful content update explainer says that the helpful content algorithm is a signal used to rank content.<\/p>\n

\n

\u201c\u2026it\u2019s just a new signal and one of many signals Google evaluates to rank content.\u201d<\/p>\n<\/blockquote>\n

4. It Checks if Content is By People<\/h3>\n

The interesting thing is that the helpful content signal (apparently) checks if the content was created by people.<\/p>\n

Google\u2019s blog post on the Helpful Content Update (More content by people, for people in Search<\/a>) stated that it\u2019s a signal to identify content created by people and for people.<\/p>\n

Danny Sullivan of Google wrote:<\/strong><\/p>\n

\n

\u201c\u2026we\u2019re rolling out a series of improvements to Search to make it easier for people to find helpful content made by, and for, people.<\/p>\n

\u2026We look forward to building on this work to make it even easier to find original content by and for real people in the months ahead.\u201d<\/p>\n<\/blockquote>\n

The concept of content being \u201cby people\u201d is repeated three times in the announcement, apparently indicating that it\u2019s a quality of the helpful content signal.<\/p>\n

And if it\u2019s not written \u201cby people\u201d then it\u2019s machine-generated, which is an important consideration because the algorithm discussed here is related to the detection of machine-generated content.<\/p>\n

5. Is the Helpful Content Signal Multiple Things?<\/h2>\n

Lastly, Google\u2019s blog announcement seems to indicate that the Helpful Content Update isn\u2019t just one thing, like a single algorithm.<\/p>\n

Danny Sullivan writes that it\u2019s a \u201cseries of improvements which, if I\u2019m not reading too much into it, means that it\u2019s not just one algorithm or system but several that together accomplish the task of weeding out unhelpful content.<\/p>\n

This is what he wrote:<\/strong><\/p>\n

\n

\u201c\u2026we\u2019re rolling out a series of improvements to Search to make it easier for people to find helpful content made by, and for, people.\u201d<\/p>\n<\/blockquote>\n

Text Generation Models Can Predict Page Quality<\/h2>\n

What this research paper discovers is that large language models (LLM) like GPT-2 can accurately identify low quality content.<\/p>\n

They used classifiers that were trained to identify machine-generated text and discovered that those same classifiers were able to identify low quality text, even though they were not trained to do that.<\/p>\n

Large language models can learn how to do new things that they were not trained to do.<\/p>\n

A Stanford University article about GPT-3 discusses how it independently learned the ability to translate text from English to French, simply because it was given more data to learn from, something that didn\u2019t occur with GPT-2, which was trained on less data.<\/p>\n

The article notes how adding more data causes new behaviors to emerge, a result of what\u2019s called unsupervised training.<\/em><\/p>\n

Unsupervised training is when a machine learns how to do something that it was not trained to do.<\/p>\n

That word \u201cemerge<\/em>\u201d is important because it refers to when the machine learns to do something that it wasn\u2019t trained to do.<\/p>\n

The Stanford University article on GPT-3<\/a> explains:<\/strong><\/p>\n

\n

\u201cWorkshop participants said they were surprised that such behavior emerges from simple scaling of data and computational resources and expressed curiosity about what further capabilities would emerge from further scale.\u201d<\/p>\n<\/blockquote>\n

A new ability emerging is exactly what the research paper describes.\u00a0 They discovered that a machine-generated text detector could also predict low quality content.<\/p>\n

The researchers write:<\/strong><\/p>\n

\n

\u201cOur work is twofold: firstly we demonstrate via human evaluation that classifiers trained to discriminate between human and machine-generated text emerge as unsupervised predictors of \u2018page quality\u2019, able to detect low quality content without any training.<\/p>\n

This enables fast bootstrapping of quality indicators in a low-resource setting.<\/p>\n

Secondly, curious to understand the prevalence and nature of low quality pages in the wild, we conduct extensive qualitative and quantitative analysis over 500 million web articles, making this the largest-scale study ever conducted on the topic.\u201d<\/p>\n<\/blockquote>\n

The takeaway here is that they used a text generation model trained to spot machine-generated content and discovered that a new behavior emerged, the ability to identify low quality pages.<\/p>\n

OpenAI GPT-2 Detector<\/h2>\n

The researchers tested two systems to see how well they worked for detecting low quality content.<\/p>\n

One of the systems used RoBERTa<\/a>, which is a pretraining method that is an improved version of BERT.<\/p>\n

These are the two systems tested:<\/strong><\/p>\n

They discovered that OpenAI\u2019s GPT-2 detector was superior at detecting low quality content.<\/p>\n

The description of the test results closely mirror what we know about the helpful content signal.<\/p>\n

AI Detects All Forms of Language Spam<\/h2>\n

The research paper states that there are many signals of quality but that this approach only focuses on linguistic or language quality.<\/p>\n

For the purposes of this algorithm research paper, the phrases \u201cpage quality\u201d and \u201clanguage quality\u201d mean the same thing.<\/p>\n

The breakthrough in this research is that they successfully used the OpenAI GPT-2 detector\u2019s prediction of whether something is machine-generated or not as a score for language quality.<\/p>\n

They write:<\/strong><\/p>\n

\n

\u201c\u2026documents with high P(machine-written) score tend to have low language quality.<\/p>\n

\u2026Machine authorship detection can thus be a powerful proxy for quality assessment.<\/p>\n

It requires no labeled examples \u2013 only a corpus of text to train on in a self-discriminating fashion.<\/p>\n

This is particularly valuable in applications where labeled data is scarce or where the distribution is too complex to sample well.<\/p>\n

For example, it is challenging to curate a labeled dataset representative of all forms of low quality web content.\u201d<\/p>\n<\/blockquote>\n

What that means is that this system does not have to be trained to detect specific kinds of low quality content.<\/p>\n

It learns to find all of the variations of low quality by itself.<\/p>\n

This is a powerful approach to identifying pages that are not high quality.<\/p>\n

Results Mirror Helpful Content Update<\/h2>\n

They tested this system on half a billion webpages, analyzing the pages using different attributes such as document length, age of the content and the topic.<\/p>\n

The age of the content isn\u2019t about marking new content as low quality.<\/p>\n

They simply analyzed web content by time and discovered that there was a huge jump in low quality pages beginning in 2019, coinciding with the growing popularity of the use of machine-generated content.<\/p>\n

Analysis by topic revealed that certain topic areas tended to have higher quality pages, like the legal and government topics.<\/p>\n

Interestingly is that they discovered a huge amount of low quality pages in the education space, which they said corresponded with sites that offered essays to students.<\/p>\n

What makes that interesting is that the education is a topic specifically mentioned by Google\u2019s to be affected by the Helpful Content update.
Google\u2019s blog post written by Danny Sullivan shares:<\/p>\n

\n

\u201c\u2026our testing has found it will especially improve results related to online education\u2026\u201d<\/p>\n<\/blockquote>\n

Three Language Quality Scores<\/h2>\n

Google\u2019s Quality Raters Guidelines (PDF<\/a>) uses four quality scores, low, medium, high and very high.<\/p>\n

The researchers used three quality scores for testing of the new system, plus one more named undefined.<\/p>\n

Documents rated as undefined were those that couldn\u2019t be assessed, for whatever reason, and were removed.<\/p>\n

The scores are rated 0, 1, and 2, with two being the highest score.<\/p>\n

These are the descriptions of the Language Quality (LQ) Scores:<\/strong><\/p>\n

\n

\u201c0: Low LQ.
Text is incomprehensible or logically inconsistent.<\/p>\n

1: Medium LQ.
Text is comprehensible but poorly written (frequent grammatical \/ syntactical errors).<\/p>\n

2: High LQ.
Text is comprehensible and reasonably well-written (infrequent grammatical \/ syntactical errors).<\/p>\n<\/blockquote>\n

Here is the Quality Raters Guidelines definitions of low quality:<\/strong><\/p>\n

Lowest Quality:<\/h3>\n
\n

\u201cMC is created without adequate effort, originality, talent, or skill necessary to achieve the purpose of the page in a satisfying way.<\/p>\n

\u2026little attention to important aspects such as clarity or organization.<\/p>\n

\u2026Some Low quality content is created with little effort in order to have content to support
monetization rather than creating original or effortful content to help users.<\/p>\n

Filler\u201d content may also be added, especially at the top of the page, forcing users to scroll down to reach the MC.<\/p>\n

\u2026The writing of this article is unprofessional, including many grammar and punctuation errors.\u201d<\/p>\n<\/blockquote>\n

The quality raters guidelines have a more detailed description of low quality than the algorithm.<\/p>\n

What\u2019s interesting is how the algorithm relies on grammatical and syntactical errors.<\/p>\n

Syntax is a reference to the order of words.<\/p>\n

Words in the wrong order sound incorrect, similar to how the Yoda character in Star Wars speaks (\u201cImpossible to see the future is\u201d).<\/p>\n

Does the Helpful Content algorithm rely on grammar and syntax signals? If this is the algorithm then maybe that may play a role (but not the only role).<\/p>\n

But I would like to think that the algorithm was improved with some of what\u2019s in the quality raters guidelines between the publication of the research in 2021 and the rollout of the helpful content signal in 2022.<\/p>\n

The Algorithm is \u201cPowerful\u201d<\/h2>\n

It\u2019s a good practice to read what the conclusions are to get an idea if the algorithm is good enough to use in the search results.<\/p>\n

Many research papers end by saying that more research has to be done or conclude that the improvements are marginal.<\/p>\n

The most interesting papers are those that claim new state of the art results.<\/p>\n

The researchers remark that this algorithm is powerful and outperforms the baselines.<\/p>\n

They write this about the new algorithm:<\/strong><\/p>\n

\n

\u201cMachine authorship detection can thus be a powerful proxy for quality assessment.<\/p>\n

It requires no labeled examples \u2013 only a corpus of text to train on in a self-discriminating fashion.<\/p>\n

This is particularly valuable in applications where labeled data is scarce or where the distribution is too complex to sample well.<\/p>\n

For example, it is challenging to curate a labeled dataset representative of all forms of low quality web content. \u201c<\/p>\n<\/blockquote>\n

And in the conclusion they reaffirm the positive results:<\/strong><\/p>\n

\n

\u201cThis paper posits that detectors trained to discriminate human vs. machine-written text are effective predictors of webpages\u2019 language quality, outperforming a baseline supervised spam classifier.\u201d<\/p>\n<\/blockquote>\n

The conclusion of the research paper was positive about the breakthrough and expressed hope that the research will be used by others.<\/p>\n

There is no mention of further research being necessary.<\/p>\n

This research paper describes a breakthrough in the detection of low quality webpages.<\/p>\n

The conclusion indicates that, in my opinion, there is a likelihood<\/em> that it could make it into Google\u2019s algorithm.<\/p>\n

Because it\u2019s described as a \u201cweb-scale\u201d algorithm that can be deployed in a \u201clow-resource setting\u201d means that this is the kind of algorithm that could go live and run on a continual basis, just like the helpful content signal is said to do.<\/p>\n

We don\u2019t know if this is related to the helpful content update but it\u2019s a certainly a breakthrough in the science of detecting low quality content.<\/p>\n

Citations<\/h2>\n

Google Research Page:<\/h3>\n

Generative Models are Unsupervised Predictors of Page Quality: A Colossal-Scale Study<\/a><\/p>\n

Download the Google Research Paper<\/h3>\n

Generative Models are Unsupervised Predictors of Page Quality: A Colossal-Scale Study<\/a> (PDF)<\/p>\n

Featured image by Shutterstock\/Asier Romero<\/em><\/p>\n<\/div>\n