Voice search - New guidelines for quality raters

Google recently published on its blog, Google Research blog, an article concerning the evaluation criteria for voice searches results. According to the document from the Californian firm, the determining criteria are: the satisfaction, the elocution, the length and the formulation. Explanation.
 

Google assistant and voice search, the rules change

 
New instructions inherent to the quality standards of search have been published on Google Research blog. Those mainly refer to Google Assistant and search voice results. Similar to the guidelines for web search, the directives, which we mention today, change considerably the situation, as far as no screen can be examined during the evaluation of the results. Indeed, only the voice responses from the Google Assistant are assessed!
 
Google attempts to explain this situation: " The Google Assistant needs its own guidelines in place, as many of its interactions utilize what is called ‘eyes-free technology,’ when there is no screen as part of the experience.” Google has designed machine learning and algorithms to try to make the voice responses and “answers grammatical, fluent and concise.”
 
 

What are the quality raters satisfaction criteria?

 
There are 4 guidelines for voice search quality raters. It is the respect for these guidelines that allows Google to determine if the answers to a voice search are correct and relevant:
 
· Satisfaction: it is obvious that the content of the answer has to answer the question formulated by the user;
 
· Length: given that it is impossible to scan and locate the information when the answers are spoken, it is important that the proposed answers are neither too long nor too short. More important, the information must be useful;
 
· Formulation: it is difficult to understand a spoken answer if it is badly formulated and grammatically incorrect, that is why it is particularly important to pay attention to grammatical and formulation correctness.
 
· Elocution: the pronunciation and the prosody of the answers have to be the object of a certain effort and attention.
 
 Back to news
Fran├žais English