Katherine Young, 23, is a Google rater — a contract worker and a college student in Macon, Ga. She is shown an ambiguous search query like “what does king hold,” presented with two sets of Google search results and asked to rate their relevance, accuracy and quality. The current search result for that imprecise phrase starts with links to Web pages saying that kings typically hold ceremonial scepters, a reasonable inference.
Her judgments, Ms. Young said, are “not completely black and white; some of it is subjective.” She added, “You try to put yourself in the shoes of the person who typed in the query.”
How smart do you need to be to do this? How well-educated? How is the quality of your work to be judged? The full article is here, interesting throughout.