Does the quality of blog comments deteriorate?

by on February 5, 2008 at 6:49 am in Web/Tech | Permalink

Forget about MR and its superb commentators, I am talking about the typical above-average blogs.  I often have the impression that the best comments come in the first fifteen or so, after which quality declines precipitously and often exponentially.  Why might that be?

1. The truly smart people only like to make smart points on "fresh" posts.  For instance more people read the comments on fresh posts (but why?), so the
benefit of a quality comment is lower as the post becomes older.

2. As time passes, the chance that a warring twosome find each other, and take over the thread, increases.

3. There is a tendency to attack or respond to the stupidest or most controversial thing said, and the longer the comments thread runs for, the stupider this will get.

4. As the number of comments multiplies, so does the number of independent discussion threads and the optimal number of threads is exceeded.

5. (Addended) As one (early) commentator notes below, the simple fact of diminishing marginal utility.

Might some of these mechanisms also help explain why a) history of thought is "ghettoized" as a field, and b) there is such a high premium to working in hot, new fields?  The general point is that there are increasing returns to scale for high quality discussions; furthermore those quality discussions are quite fragile and require cultivation and subsidization through norms.  Freshness matters, so stale topics will indeed encounter discrimination.

Comments are open, who wants to go first? 

1 Edgardo February 5, 2008 at 7:59 am

It’s an election year.

2 Peter February 5, 2008 at 8:11 am

Geez, I don’t know about this one. What about something like Making Light?

There, open thread can have 500+ comments, and the thread can elicit interesting conversation beginning at comments in the 300s. And topical posts (many fewer per day than here) can have long and substantive discussion throughout.

So I’m putting in for two more explanatory variables: moderator/community norms and a more gentle post-pacing.

3 odograph February 5, 2008 at 8:27 am

I think trolls often turn up at the ends of long comment threads posts because they don’t bother to read earlier posts.

Just kidding. I think the progression is: point made, conversation, argument, flame war.

Maybe the “points” are raised early.

4 Grant Gould February 5, 2008 at 8:35 am

Fresh posts have fewer comments to read. Most commenters feel that they must read the entire comment thread before commenting (why else, after all, is the “Post a comment” form at the bottom of the page?). This means that late comments will be of two sorts: Those that are of sufficiently high value to the poster to be worth reading all of the others, and those by people who have already read most of the comment thread in order to post previously. Both of these alternatives favor narcissists and flamers over people with other priorities.

Hypothesis: The quality of comments (but not of the discussion as a whole?) could be improved by moving the “Post a comment” form to above the comments. This would discourage give-and-take among commenters, though, so such a system would probably be best accompanied by some sort of administrative promotion of important comments — a “comments of the week” thread like some blogs have taken to doing, perhaps.

Hypothesis: The quality of comments could be improved by capping the number of comments. Perhaps after the first fifty comments only people who have not yet posted on the thread should be allowed, and after a hundred no further comments should be allowed at all.

5 Probiscus February 5, 2008 at 8:39 am

Interesting to think that the ghettoization of thought is caused by the
same mechanisms that apply to the unravelling of a coherent thread. Though
it seems to me that this gehttoization is due more to the fact that people
have to think in discrete categories. Our capacity for consilience is
pretty limited.

Points two and three, driven by ego, probably account for some convoluted
and contradictory theories in economics and the soft sciences. Maybe
it’s not so applicable to the hard sciences. But look at Aristotle.

Also, when everything has already been said and done, you gotta make stuff
up to look smart. Look at anthropologists.

6 Zettel February 5, 2008 at 8:49 am

I agree. This comment is an example.

7 Tom February 5, 2008 at 8:50 am

I’m so pissed off that there are already 9 comments here that I can’t even remember the important point I had to make. What do you people do? Hover over this site all day do you can be the first to comment one each post?

8 Rue Des Quatre Vents February 5, 2008 at 9:03 am

It’s not that a topic grows stale; it’s that it moves elsewhere. For instance, you wrote an entry about the quality of Tide recently. I was away for some time so I didn’t read about it until days after it was posted, enough time for the comments to add up. The best comment–a rough summary report by a marketing analyst–came after the initial flood of comments. But no one responded. Conversation is king. And no one wants to have one in the comments section of a post five days old.

9 puzz February 5, 2008 at 9:13 am

Grant Gould: “The quality of comments could be improved by capping the number of comments. Perhaps after the first fifty comments only people who have not yet posted (…)”

Or maybe make the latest comments (those at the bottom) appear with smaller fonts?

10 Mark February 5, 2008 at 9:15 am

I believe you pointed out how the number of comments was highly correlated to the topic. i.e. any post mentioning Krugman elicited more comments that were generally lower in quality.

The tone of the website contributes greatly to the quantity and quality of comments. Tyler’s posts are mature, insightful and entertaining. Compare this to CalculatedRisk (great blog). The subject matter and generally negative, satirical tone attracts less constructive feedback and creates an atmosphere where it is more acceptable to spam bandwagon or confrontational posts.

11 Peter L February 5, 2008 at 9:27 am

@Jonathon’s point above is that there’s a tendency for the same comment to be repeated over and again, and a growing likelihood that people won’t read the whole thread to see if there are repeats (btw, it would help if comments were numbered so it would be possible to refer back to people, you know, like conversation). The mostly brilliant comic XKCD has an answer – an IRC channel bot that catalogs all the existing logs and chucks out people who make repeat statements. Of course, this is a fancy-but-repeat of my own point that moderation makes a difference, so there is still something left to be desired…

12 Andromeda February 5, 2008 at 9:30 am

If my husband comments at all, it tends to be very early in the thread (as he did above, #7). This is because he is the sort of easily bored information junkie who needs a continual flow of stimulation to his brain.

I hypothesize that such people are more likely to make interesting comments, because their brains are fast-moving and full of exciting, diverse information.

I hypothesize that such people are more likely to use, and frequently check, rss readers, which deliver information to them with maximum speed (because it would be OMG AWFUL if a minute might elapse without new information, if an interesting blog post might be up for a whole minute before you saw it). Therefore they are among the early commenters, because they see your post .3289 seconds after it happens.

Having switched over to rss relatively recently myself, I note that one of the changes in my reading style is I’m much less likely to reread old posts/threads, because they don’t show up in my default view. So I imagine the only posts you get late in threads are (a) the sort of deliberative thinker so rare on the internet; (b) people who have gotten obsessed with a particular topic (who are more likely to be raging partisans, chewing on one another like rabid dogs); or (c) spam. Signal:noise not so good here.

Be interesting to see if the time distribution of good comments was different before rss became widespread.

13 Samir Nurmohamed February 5, 2008 at 9:40 am

You better watch out Tyler – you might create a self-fulfilling prophecy by making these comments.

14 Godwin February 5, 2008 at 9:55 am

Damn, Justin beat me to it. Let me just say the treasonous Dems want us to lose in Iraq and Bush is to blame for everything.

Thank you, good night.

15 odograph February 5, 2008 at 10:10 am

“Hitler” was an old joke (only 1990? I would have thought earlier.

To what degree do people believe in the ‘group mind’ (or ‘hive mind’) thing? That was bandied about in the early days of USENET and computer-mediated communication. Are comments and posts random, or do they stitch together to form some kind of snapshot consensus (or lack thereof)?

16 lw February 5, 2008 at 10:18 am

unfogged is a counterexample in the mean.

17 Matt February 5, 2008 at 10:46 am

I blame it all on Godwin’s Law (http://en.wikipedia.org/wiki/Godwin's_law).

18 londenio February 5, 2008 at 10:58 am

I just would like to see a simple graph showing the relationship of comment LENGTH an comment POSITION. Or something like the volatility of comment length over time. It seems to me that in as the comment thread grows one finds very short and very long comments (the long comments surely push the average length upwards over time).

Then we could look at content, but length would be an interesting place to start. It could be that as the comment number increases, people with short comments do not bother because of the large dilution. Only rants are worth posting.

For those with time to waste: check the non-political non-religious discussion in kottke.org about a show exploring the myth of whether a plane on a conveyor belt can take off.

http://www.kottke.org/08/01/mythbusters-airplane-on-a-conveyor-belt

It does deteriorate over time. With longer and longer comments.

I hope this comment is not too long too early. It is, isn´t it? My apologies.

19 jay barnes February 5, 2008 at 11:32 am

Collin: I like ponies!!!!!

20 Justin February 5, 2008 at 12:10 pm

I was going to write an insightful comment but then I realised at comment number 38 nobody would read it or take it seriously. Damn!

21 Brent February 5, 2008 at 12:21 pm

Seems like the recent comments usually come from the people who read the blog regularly — ie, people with a true interest in the topic, are knowledgable, and can get the true understanding as they relate back to old posts. Later commenters usually are latecomers who don’t share the same passion for the topic, and thus, aren’t as knowledgable — or come in via random google searches or pass along readers. Because they don’t have the same passion for the subject, or familiarity with the blog itself, they tend to have less valuable input.

I agree that the “sparing” between posters tends to cause a comments section to spiral downward faster than anything.

All that said, I’m a late commenter on this, so my opinion is probably equally as bad as I would project out on the curve.

22 anon February 5, 2008 at 12:56 pm

lolzomg latest post!!!!!1

23 Randy February 5, 2008 at 1:10 pm

Some of us don’t come to blogs primarily to read the posts and identify with the musings of the chief blogger. We come here primarily to debate – and the best debates often happen at the tail end of the thread. I don’t mean this as an attempt to shatter your ego. You post good stuff or we’d be somewhere else. Just trying to help you better understand your market.

24 Yancey Ward February 5, 2008 at 2:19 pm

Ok, time to bust up this thread. Anyone want to start a flamewar?

25 yakov February 5, 2008 at 3:49 pm

The other comment discussed above is that posters post for the recognition, not the value of the discussion itself; thus, they don’t bother posting unless someone will read it.

26 jkottke February 5, 2008 at 6:10 pm

In my experience with kottke.org, I find the best comments at the beginning *and* the end of threads (usually comments posted a day or two after the thread starts). Not sure why the end of the thread gets good again…maybe some people reflect on the post for a time before posting something thoughtful. Or perhaps a few hours is too short a time period to capture a casual reader who is perfectly suited to respond to the topic at hand. Or an inbound link brings the perfect commenter in from another site a day or two later.

27 Eric H February 5, 2008 at 7:53 pm

Up yours, Yancey. You wouldn’t know a good flame war if it bit you. 😉

I think that there is a high premium placed on “getting in a point” even though it is sometimes not well thought out. I think people want to be read more than they want to be right. As in this example.

28 Jacqueline February 6, 2008 at 1:11 am

On my blog it was definitely #2 and #3.

29 Alex Tabarrok February 6, 2008 at 8:19 am

The last blog comment is always by the smartest person.

Comments on this entry are closed.

Previous post:

Next post: