Here is Michael Huemer’s very interesting critique of student evaluations of professors, full of cites and links. Yes, student evaluations correlate positively with other measures of teaching effectiveness. Take multiple sections of the same course and give a common final exam, the correlation is in the neighborhood of 0.4 to 0.5.
On the other hand, a professor gets a much better evaluations if students think they will get good grades. The statistical correlations are strong and hard to deny. And in one study 70 percent of students admitted that their evaluation was influenced by the grade they expected to get. See this game theory article on how one-shot reciprocity can work.
In one survey, 38 percent of professors admitted to dumbing down their courses to get better evaluations.
Cosmetic factors such as appearance have a big influence on evaluations.
Huemer offers no policy conclusion. He does note that ratings by colleagues and other observers do not agree with each other very much and thus cannot stand as a serious alternative.
If you are curious, I could not find Huemer’s student evaluations through a web search.