At Google, we like to say that Search is not a solved problem: We’re constantly making improvements (more than 4,800 last year alone). These changes can be big launches or small tune-ups, but they’re all designed to make Search work better for you, and to make sure you can find relevant, high quality information when you need it.
One of the key ways we determine if an improvement to Search works well is through the help of search quality raters. This group of more than 10,000 people all over the world work from a common set of search quality rater guidelines used to evaluate the quality of search results — which are publicly available. Today, we wanted to give you an idea of how these guidelines work, and how — just like Search itself — they improve over time.
What are the search quality rater guidelines?
The quality rater guidelines are more than 170 pages long, but if we have to boil it down to one phrase, we’d say they help make sure Search is returning relevant results from the most reliable sources available.
Information quality is at the heart of Search, and our systems fundamentally work to surface high-quality information. The rater guidelines help raters determine if a planned improvement is meeting that goal by providing a clear, uniform definition that all raters use to assess the results they see.
More specifically, high-quality information is content which demonstrates expertise, authoritativeness and trustworthiness on a topic, or E-A-T for short. For example, a health site with content from doctors and produced by a medical institution would have a high level of what many would consider to be expertise, authoritativeness and trustworthiness. The rater guidelines also define low-quality content on the web, such as content that spreads hate or seeks to deceive users.
Who uses the guidelines?
As we noted, the changes we make to Search are rigorously tested and evaluated by real people. Our search quality raters provide us with insights and evaluate pages against our guidelines to help make sure our systems — and proposed improvements — are working as intended.
What that looks like in practice is often a “side-by-side” test where a rater will look at two sets of Search results, one from the current version of Google and the other from an improvement we’re testing. Raters will review the pages in each set of results, and evaluate if the pages are a helpful match for the query based on our rater guidelines.
The ratings they provide don’t directly impact how a page or site appears in Search. Instead, they help us measure how well our systems are working to deliver great content.
