Assorted links


The solution I propose is comprehensive exams at the end of each course, much like Advanced Placement exams, that thoroughly and objectively distinguish students on merit alone.

Yes. This. Absolutely. Imagine how different (and better) things would be if the teacher was responsible onlyfor the teaching/coaching/intellectual development of students but had NO control over evaluation. The lousy teachers who phone it in by giving everybody high grades would quickly be exposed. And the same would go for excellent teachers, who motivate and challenge their students and produce true learning.

The problem here is that if the instructor knows what is on the test, that is all that will get taught, in my field, geology, I think that would be catastrophic. I would be much more enthusiastic about this sort of system if it was based on a yearly exam. For example to progress each year in a field, all students would have to pass a progress exam. Unfortunately that would require a rigidity that would only work in certain STEM majors. Another and possibly better idea is if we had professional exams such as the Bar, or CPA exams that students took at the end of their study. Again I think this would only work in the non humanities fields in this country.

If the instructor defines his own test he knows what's on it! The situation is improved when someone else defines it, not worsened.

Even a not-very-good exam would be better than the status quo. But there's no reason the exam has to be lousy. And, yes, it would be easier in K-12 than at the university level, and somewhat easier in technical fields -- but there's no reason we should trust teachers to provide tough, honest assessments of student writing any more than any other assessment. We should probably trust such assessments even less because they're inherently subjective. So, step #1, break the link between instruction and assessment (and remove the perverse incentives to game the system). Step #2, work on ever better ways of doing assessment.

Isn't this why you have SATs?

In the UK, we have national exams at ages 16 and 18. The results of these are used by universities as entry criteria. The teachers have no knowledge of the precise questions, though there is a tightly-defined curriculum. The final grade is calculated as 20% of coursework and 80% of exams, so there is relatively little scope for teachers to inflate grades.

The main flaw in the system is that schools are free to choose which exam board they want to administer the tests. The market for exam boards is competitive. In recent years exam boards have been accused of luring schools with promises of easier exams and therefore higher grades.

I don't see an obvious solution to the exam board problem. As competition intensifies, the gulf in grades between different exam boards could widen significantly. Universities and employers will have to learn the quality and reputations of different exam boards. This informational asymmetry (particularly for small businesses trying to hire school-leavers) is inefficient for the economy as a whole.

The SAT is basically a verbal and math IQ test. These tests would require knowledge of a specfic subject, like history or physics.

The USA and Canada are the only first world countries where objective exams are not the major factor in admission to elite universities. The SAT is not taken seriously by elite schools except to eliminate a few obviously unsuitable applicants.

Admissions are done by sports performance, extracurricular activities, personal evaluations by important local figures if you're in a position to introduce them to your children, and school grades assigned by individual teachers with no oversight. (roughly in that order of importance)

"The SAT is not taken seriously by elite schools except to eliminate a few obviously unsuitable applicants."

This is a ridiculous statement.

"The SAT is not taken seriously by elite schools except to eliminate a few obviously unsuitable applicants."

This certainly wasn't true in the early 1990's. I don't think it's changed since then.

In pretty every industry, including software, Quality Assurance is done by someone else than implementation, research and development. If people are told to evaluate their own work, it usually sucks.

For some reason, education seems to be The Exception.

"If people are told to evaluate their own work, it usually sucks.

For some reason, education seems to be The Exception."

No, it holds true in education also. ;)

I agree, the cross purposes put on teachers in schools puts them in a difficult position. The teacher is expected to help the students to learn, and then to grade students on a rigorous basis. The teacher is to be for the student an against them, their coach and their judge.


I'm all for making evaluations of academic mastery exam-based.

However, people already sputter and complain that AP tests only measure test-taking ability.

People already complain that Asians score higher on objective tests because they grind and prep too hard. Or cry "disparate impact" or some sort of unfairness when blacks score lower.

In general, a lot of people tend to dislike objective evaluations on themselves... to either excuse their ineptitude or because they think objective tests can't measure their special snowflakiness. Pride and solipsism at work.

Daniel Tosh: "Don't you love it when people in school are like, 'I'm a bad test taker.' You mean you're stupid. Oh, you struggle with that part where we find out what you know? I can totally relate see, because I'm a brilliant painter minus my god awful brushstrokes. Oh, how the masterpiece is crystal up here but once paint hits canvas I develop Parkinson's."

Ability at school and tests are probably not great that greatly correlated with ability to work.

In many European countries (including the UK & Ireland) there is a system of 'public' examinations in place at both High School and College level, (for all subjects, not just STEM fields) whereby assessment is by anonymous final exams marked by an independent examiner rather than by the course instructor. I don't see why such a system would not work in the U.S (although it would necessitate a move away from a 'continuous assessment' style system).

In response to link #1, the problem discussed and the solution proposed fail for the same reason. See Campbell's Law for more.

As a high school AP teacher, I was with Knight all the way until he said "AP exams test a broad array of knowledge and understanding." It's possible that's true for AP physics, but in my experience it's not true for AP US Government, where the focus is on memorizing minutiae. It's also not true for AP Macroeconomics or Microeconomics, which focus on rote manipulation of diagrams. It is typically possible to get a 5 out of 5 on the economics exams without writing a single complete sentence. 2/3rds of the score is based on the multiple choice section alone.

The general concept of externally moderated exams is a good one, but Knight is wrong when he suggests that such exams will not encourage teaching to the test. Give someone a metric by which they will be judged, and they will do their best to score high on that metric. The problem isn't teaching to the test, it's having a test that so poorly measures what we're actually interested in measuring. Teaching to the test would be great if we had a test worth teaching to.

"2/3rds of the score is based on the multiple choice section alone"

I am not sure by itself this is a problem. The average multiple choice test is, in my experience, significantly better than the average free response test. Ambiguous wording is easier to misinterpret when you have to generate a response yourself, and it's often a guessing game with free response tests what concepts the graders want you to focus on and what constitutes a sufficiently detailed response.

The purpose of free response tests is often to obscure what is being tested over.

In any case, you could still have non-multiple-choice tests and yet have them graded objectively by someone other than the professor who taught that class.

If the student can't figure out the diagrams, he or she didn't learn any economics. What you are arguing for is more multiple choice questions. In a standardized exam, other than math or chemistry type problems, non multiple choice test are lousy because of the need for uniform grading. Over the years I have seen many excellent multiple choice problems. Heck, the two most grueling exams of my life were True/False exams. One in calc and the other in physics.


(a) Greg Smith, look what you've started.

(b) MIT -> Georgetown Law -> public school teacher as a career path could indicate a number of things, not all of which make one an authority on public education.

(c) "There is no such thing as “teaching to the AP test." " False.

(d) All of the above.

(b) MIT -> Georgetown Law -> public school teacher as a career path could indicate a number of things, not all of which make one an authority on public education.


"Andrew Knight has recently been named the official Patent Expert of, Inc., the nation's largest filer of provisional patent applications for independent inventors. Mr. Knight produces and maintains the website's patent content."

This is truly bizarre

Tyler has fallen for linking to a guy who is an inventor hustler - who is just trying to be helpful!

He self-publishes his book (Full Ride to College) about which one Amazon reviewer says this:

"I didn't agree with most of the info. It says you'll stick out if you invent something and get a patent on it. Then, it includes 5 pages on applying for a patent, with a photocopy of a patent application. It also advocates writing a book."

(Hmm, and Andrew Knight is just the guy to help you do both.)

Next, to drum up business for his "how to get into college book" it appears that he targets a high school with a very wealthy demographic (dollars to donuts he applied to teach at TJ but they spotted his schtick), then after the "high school patent application business" and self-published book sales slow down he leaves Potomac and flogs this article to several papers. Hmm, targets Fairfax Times, published in the county that is home to TJ and lots of insecure parents (with plenty of disposable income) of high school students.

Then Tyler picks up on it, helping perpetuate his college counseling cum patent application and book as marketing device "business".

What's not to like?

But Wait! There's More!!!!!

Andrew Knight and Knight Publishing have been busy:
(titles followed by ISBN or Amazon ASIN)

"Absolute Truth: The Afkerian Book I" 0966102606
(wondering: where is Book II?)

"The poor man's nuclear bomb" 0879470976

"At Least in Hell the Christians Won't Harass Me" 0966102630

"Go To Any College: Supplemental Course Materials" 0966102665

"New Title 1" (the cover says "The Mobius Strip") B001GIOERK

(b) MIT -> Georgetown Law -> public school teacher as a career path could indicate a number of things, not all of which make one an authority on public education.

He could just be a general crank (NB: I use the term affectionately; a lot of progress happens because cranks deliberately broke the old system). If I had enough money to be able to say F-U to any parent who complains, I could see becoming a teacher.

One of my best high school teachers had a JD from Georgetown. Like many lawyers he got sick of his job. Unlike many he decided to pursue something more meaningful. Coincidentally, he also hated grades, although he didn't want them replaced with anything.

Dumb question (maybe): What does MIE stand for ahead of the link re El Salvador?

Markets in Everything.

#1 The strange thing about the American system is that, for college-bound students, especially those bound for "elite" colleges, there is no specialization in high school. All students are expected to take advanced courses in all subjects. Even if you are planning to major in English literature, if you want to go to a fancy college, you have to take advanced science and math. This creates a need for grade inflation. It's not like this in many countries and it didn't used to be like this in the US.

Very true. This comment, I think, is more insightful than the entire Fairfax Times article.

I took nine AP exams in high school, passing them with 5s and 4s, and make the mistake of attending my state's flagship public institution when I should pursued an "elite" private education for signaling purposes.

How can one compare humble Joe who attended State U. to spoiled Sally who attended Private U.? I say we require students to post their scored exams, term papers, recorded presentations, etc. online and let employers evaluate the quality of work for themselves.

Why was it a mistake? Why not go to grad school to get your signal? You probably saved a ton of money, right?

I second Cliff's point. I did the same thing, more or less, and I very much enjoy not having to fret about student loans. Signalling, too, is overrated, unless you're trying to get blog hits.

"I say we require students to post their scored exams, term papers, recorded presentations, etc. online and let employers evaluate the quality of work for themselves."
This is a joke, right? I've submitted ten-page writing samples and had interviewers say "I didn't read the whole thing but it seemed pretty good." Hell I've had interviewers who spent the first sixty seconds of the interview scanning my resume for what was very clearly the first time. And you want them to go re-grade term papers?

I give #1 an A+.

I think this is the best way to break the signal of high school grades. Take it to its logical conclusion right now: give everyone an A+, even if they never show up for class.


"If I wanted some sort of scheme that laid out the stages of civilization, the period before machine super intelligence and the period after super machine intelligence would be a more relevant dichotomy."

Agreed. However, to put it succinctly, I do not reject the null hypothesis of infeasible Strong AI at α = 0.05.

"When you look at what's valuable or interesting in examining these stages, it's going to be what is done with these future resources and technologies, as opposed to their structure. It's possible that the long-term future of humanity, if things go well, would from the outside look very simple. You might have Earth at the center, and then you might have a growing sphere of technological infrastructure that expands in all directions at some significant fraction of the speed of light, occupying larger and larger volumes of the universe---first in our galaxy, and then beyond as far as is physically possible. And then all that ever happens is just this continued increase in the spherical volume of matter colonized by human descendants, a growing bubble of infrastructure. Everything would then depend on what was happening inside this infrastructure, what kinds of lives people were being led there, what kinds of experiences people were having. You couldn't infer that from the large-scale structure, so you'd have to sort of zoom in and see what kind of information processing occurred within this infrastructure."

It is possible, though speculative, and more subject to frame bias than I would prefer.

"... simulations ..."

Indeterminate, at this time. However, I have not read Bostrom's paper.

I do not reject the null hypothesis of infeasible Strong AI at α = 0.05.


He's saying that he thinks its likely that strong AI is infeasible. He's arbitrarily assigning probabilities to it without any actual data, to sound scientific. This is known as scientism, and is the second worst sin unto science.

He is giving his own personal confidence in his opinion, albeit in a slightly inaccessible manner.

I'm glad someone else thinks you can game the system at MIT and get good grades. People deny it until they meet an idiot with straight As or close to it but that aren't enough idiots revealing their undeserved grades to convince everyone.

I taught at one of those Asian test prep cram schools for awhile. Increasing those would not be good for society.

The SATII tests are course-specific exams in Biology, US History, etc and most colleges require 3 reported scores for admission. This provides some corroboration for teacher grades.

The column in the Fairfax Times makes many good points, but the writer reveals himself as an out of touch curmudgeon by advocating a return to grading on a curve instead of grading against a standard. Nor has he thought through how one should conduct assessments in a history course where the difference between a degree of learning deserving of a meaningfully high grade and learning 100% of the information presented is vast.

Colleges know about grade inflation, and it unfortunately does mean that HS teachers have to finesse a system that can be understood in context without enabling the sorts of behaviors that the writer apparently has in his classroom. It can be done, but he hasn't figured it out yet. If you want to cut down on grade grubbing nonsense, establish a reputation for being a strong teacher and then use your authority/credibility to shift things in your classroom in a better direction, without tilting at windmills and dishing out low grades to prove that you are the defender of Western Civilization.

PS: no offense to the commenter above, but the AP Gov course is not about memorizing random facts. Actually, the curriculum does a nifty job at distinguishing those who teach concepts and analytical thinking and those who turn the material into a series of lists to memorize. One can be successful in the latter mode, and many teachers adopt it because they want a threshold level of 3 or higher scores on the exam. Those teachers find it very hard to generate a lot of 4 and 5 scores because they are approaching the subject from the point of view of the multiple choice section and not what makes most sense for the subject.

Comments for this post are closed