The prophet Jonathan S. Hartley

by on February 6, 2017 at 1:39 pm in Current Affairs, Philosophy, Sports | Permalink

In 2015, to make extra point plays after touchdowns more uncertain, the NFL moved the extra point distance from the 2-yard line to the 15-yard line. Since the rule change, the expected points from an extra point attempt has fallen from 0.99 (averaging between the 2002 and 2014 NFL seasons) to 0.94 (averaging the 2015 and 2016 NFL seasons) while the expected points from the two point conversion remains 0.95 (averaging between 2002 and 2016 NFL seasons). While the total number of two point conversion attempts per season has almost doubled, most coaches still rarely attempt 2 point conversions when it would be point maximizing (and win maximizing under risk neutral or risk seeking preferences). Using dynamic programming, this paper argues that this result is evidence of a conservative bias and that teams could improve expected wins by attempting more two point conversions.

Hartley is at the Wharton School, here is the link (pdf).

1 Turkey Vulture February 6, 2017 at 1:48 pm

There is much lower hanging fruit in football than this. That expected value difference is small enough that a worse offense or better defense could make the extra point the point-maximizing play.

I’m all for pushing the extra point try back even more though. Increase the gap in expected value enough and eventually you’ll knock people out of their old frameworks for evaluating the right decision. Maybe.

2 A Black Man February 6, 2017 at 2:42 pm

Bring back the drop kick. Instead of a holder for field goals and extra points, have the punter attempt a drop kick.This would do a number of good things. One is it would increase kicking as a specialty, thus attracting better athletes to punt and kick FG’s. That would bleed into better punting by making directional kicks more common. The big thing is it would change the math on strategy inside/outside the red zone. A drive that stalls at the 30 would result in more fourth down tries and more pooch punts.

You get a different game, but maybe a faster game and certainly a more interesting game inside the red zone.

3 Ray Lopez February 6, 2017 at 3:02 pm

Shorter A Black Man: replace American football with rugby?

4 Cliff February 6, 2017 at 3:09 pm

Rugby after-try kicks are not drop kicks

5 Quantitative Sneezing February 6, 2017 at 3:43 pm

In Rugby 7s (the version played in this past summer’s Olympics) the conversion attempts are drop-goals.

6 OldCurmudgeon February 6, 2017 at 5:45 pm

Nah…move them back and make the uprights narrower instead. Advantages: (i) you can fake an extra point try again; (ii) teams will ‘go for it’ on 4th down more often.

7 jason y February 6, 2017 at 2:04 pm

not as interesting or useful as the benjamin morris article ‘when to go for 2, for real’ published on 538 a few days ago, but it does include the words ‘dynamic programming’ so i guess i’m supposed to be impressed.

8 Turkey Vulture February 6, 2017 at 2:12 pm

I hope the day comes when a team scores a late touchdown to bring themselves within 4, and then goes for 2 to narrow the gap to 2. The commentators’ heads will actually explode.

9 A Black Man February 6, 2017 at 2:45 pm

IIRC, there was a high school coach who never punted and never tried a PAT. He has been proven out over time, but games are episodic. Over a season, going for two may work, but you could also lose three games in a row by not taking the easy points.

10 Cliff February 6, 2017 at 3:10 pm

Why the heck would they do that?

11 Anonymous February 6, 2017 at 3:16 pm

May be to win later with a field-goal , rather than tying. But if they miss the 2 the coach will be fired , so probably academic.

12 Cliff February 6, 2017 at 3:20 pm

Hmm, does the math work out for that? Winning in overtime is about 50/50 and the 2-pt conversion is just less than 50-50 so seems like a wash, but maybe it’s a tiny bit better. You do have the situation where they kick another FG and you’re down by 7 instead of 6.

13 Turkey Vulture February 6, 2017 at 3:24 pm

The win probability is substantially higher.

If they succeed on the 2 point try, a field goal now wins them the game outright rather than sending them into the likely 50/50 win probability of overtime. That is a substantially improved outcome which happens, say, 45% of the time.

If they fail on the 2 point try, they will now be forced to try to score a touchdown on the ensuing drive. There are many times when teams kick field goals on fourth down where the point-maximizing choice is to go for it. By eliminating the ability to sacrifice expected points to tie the game, coaches are forced to go for it on fourth down and thus behave closer to the point-maximizing optimal overall.

The net effect is that you are more likely to win the game if you go for 2 when down 4 than if you go for 1.

14 Turkey Vulture February 6, 2017 at 3:34 pm
15 Turkey Vulture February 6, 2017 at 3:35 pm

(The other really interesting one being that if with 10 minutes left, you are down by 8 after scoring a touchdown, you should also go for 2).

16 Roger Sweeny February 7, 2017 at 1:24 pm

Thank you, Bill Belichick.

17 Slocum February 6, 2017 at 2:06 pm

Coaches want to maximize their chances of winning, obviously, but they also want to protect their reputations by avoiding criticism — and criticism is much more likely when the coach makes an atypical call that’s unsuccessful. The difference between the expected value of a 1-point vs 2-point conversion attempt is so slight that coaches probably maximize their reputations by continuing to kick extra points except when necessary.

18 chrisare February 6, 2017 at 2:10 pm

Is the difference between 0.94 & 0.95 really statistically significant? Is the magnitude worth the criticism blow back that comes with failure, not to mention the possible in game emotional momentum swing for players?

19 Jeff R February 6, 2017 at 2:48 pm


The other point I’d make is that this measure is probably over-aggregated. The majority of missed extra points probably come from a handful of kickers, so taking one overall expected value for extra points league wide obscures a lot of individual variation, which teams are probably in a pretty good position to figure out, just going on previous success rates, since each team only has one kicker and they tend to have fairly long careers. This is like a Bill James wannabe telling you his model says a pitcher should always throw a curveball when he’s ahead 1-2 in the count. Uh…kinda depends on whether the pitcher actually throws a decent curve, does it not?

20 Cliff February 6, 2017 at 3:12 pm

I think you are wrong about the missed extra points coming from a few kickers and the fairly long careers.

21 Ray Lopez February 6, 2017 at 3:05 pm

Talk about momentum swing: did you notice in yesterday’s Super Bowl, how the air went out of the football (pun intended) of the NE Patriots when the ball was stripped and the fumble occurred in the first half, just when they looked like to score? It took them almost two quarters to get their groove back. Made for a good game though. I personally think blatant stripping of the ball should be illegal; though they have so many rules in football one more is not too bad.

22 Cliff February 6, 2017 at 3:13 pm

Is the paper based on that difference?

23 Hadur February 6, 2017 at 2:11 pm

Everyone knows that football is currently played irrationally. Everyone, that is, except the people making coach hiring decisions. Therefore, coaches play 100% rationally to preserve their jobs.

24 Ray Lopez February 6, 2017 at 3:09 pm

Shorter Hadur: no football coach, especially if he wins, has ever been fired for not using Sabermetrics in football. True enough. But of the winning coaches, most do (the Wall Street Journal had an article on this about a month ago). So if you want to win more, you have to get away from gut feel and more towards metrics. Either that or have a winning team that attracts future winner as well for synergy effects (though B. Belichick does use ‘sabremetrics’ in football).

25 The Other Jim February 6, 2017 at 2:13 pm

>teams could improve expected wins by attempting more two point conversions.

Because .95 is so much more than .94?

All of those hundredth of a point losses could turn into wins?

All righty then.

26 OldCurmudgeon February 6, 2017 at 5:55 pm

Why take the chance?

27 rayward February 6, 2017 at 2:27 pm

Does this mean that a team is more likely to convert a two-point conversion attempt (.95) than a one-point conversion attempt (.94) or is the point differential taken into account in determining the percentage? If it’s the former, then only a bone-head would attempt a one-point conversion; if it’s the latter, then it’s understandable that coaches would stick with what they know. Speaking of Wharton School, I learned this weekend in a newspaper article about Donald Trump Jr. that he, like his father, attended Wharton. It seems that Junior is as truthiness challenged as his father. What do they teach those kids at Wharton?

28 Anonymous February 6, 2017 at 8:43 pm

Having been a student at Penn in the mid-70s , I’m always confused when I read that Mr.Trump attended Wharton. The Economics dept from which he graduated was not a part of Wharton, although Economics courses were taken by Wharton (Business Admin) students. . Was it a part of Wharton when he was a student?

29 Sam Haysom February 6, 2017 at 2:52 pm

For me the fact this article appears to highlight almost an infinitesimal difference in outcomes suggests that this could really be the kind of article that’s in need of a Straussian take. Perhaps the writer simply means to “Straussianly” highlight how obtuse sports nerdom has become via advanced stats.

But then again I don’t think Cowen has actually ever read Strauss so I doesn’t surprise me he’d miss this one.

30 blades February 6, 2017 at 3:16 pm

I thought today people might be talking about the OT rule. Pats fan myself but is “one TD and done” really the way to end OT in playoff football?

31 Turkey Vulture February 6, 2017 at 3:29 pm

I will be amazed if the rule isn’t changed now. It sometimes takes seeing the worst-case scenario actually happen (a Super Bowl OT where one team never even gets a possession – so that the game was, in a sense, decided by the coin toss) to motivate change.

I think part of the reason this isn’t already being obsessed over is that at a certain point, it all just felt like destiny unfolding. Which is BS of course, but I think a lot of us are still in that frame of mind.

32 Roy LC February 6, 2017 at 4:54 pm

Overtime wins in the playoffs have happened before and I am hearing less complaint than usual, so I would be shocked if it changed after only 4-5 years.

33 Ted Craig February 6, 2017 at 3:33 pm

Teams could win more if they went for the two-point play more often and so they have: “teams attempted 109 two-pointers this season compared to 59 in 2014, the last year of the old rules.”

34 Flannery Bro'Connor February 6, 2017 at 3:35 pm

A conservative bias among football coaches? Stop the presses!

35 Quantitative Sneezing February 6, 2017 at 3:45 pm

Yes, it’s the FoxNews of vocations.

36 Duke Santos February 6, 2017 at 4:20 pm

That there are conservative coaches that don’t maximize their winning percentage is the worst kept secret in sports, so no new ground here (dynamic programming or not).

On this issue, the overall difference is minor enough that this shouldn’t the needle, but there are some pretty wide variances in how well some teams in perform in short yardage situations so some teams definitely should be doing this more often, while others are probably correct in sticking with the kick.

37 The Original D February 6, 2017 at 4:21 pm

Haven’t read the paper but I don’t think expected points is the best way to think of this because in the case of the two point conversion, you have to score multiples of two touchdowns to get the expected result to match up with extra point attempts.

In other words, for a given touchdown, there’s a 94% chance you’ll get an extra point but only a ~48% chance you’ll get a two point conversion. And if you miss on the first two point conversion, that means you have to try for two on the second touchdown.

And that assumes you get the next TD. If you can’t drive the field then your best hope is a field goal. But late in a game a four point deficit makes a field goal virtually worthless.

38 Antonin February 6, 2017 at 4:41 pm

Tyler may be trolling us a bit. I read the paper, and while it includes a dynamic programming equation, it doesn’t actually engage in any actual dynamic programming analysis of the go-for-two decision. All it really has is a bunch of graphs showing success rates of various teams and the suggestion that teams don’t go for two as much as a pure expected-point maximizing strategy would suggest. Furthermore, the actual statistical analysis, such as it is, does not find a significant difference in expected points between the one and two-point conversions (the author labels one statistic as “significant at the 15% level but this is not a significance level I would acknowledge). The 538 analysis may not be ideal but it is certainly much more informative.

The Romer (2006) article cited by Hartley on 4th down coversion attempts is actually quite good.

39 Jack pq February 6, 2017 at 7:01 pm


15% significance level LOL!!

40 Kyle February 6, 2017 at 8:11 pm

It’s by an MBA candidate. I thought the same thing.

41 NatashaRostova February 6, 2017 at 6:05 pm

I vaguely remember from Game Theory courses, some similar example from Soccer. Only the counter-point was that while it seemed they should attempt to change their distribution of left/center/right kicks for tie-game penalty shots, it didn’t account for the N.E. of the other sides best response.

In this case for all we know they are already playing an optimal mixed strategy, and there are nonlinearities in the game dynamics.

42 Li Zhi February 6, 2017 at 8:02 pm

I didn’t read the paper, but I doubt whether there’s enough data with only two seasons and I seriously wonder whether the REAL risks are even addressed in the paper. I suspect (but these are just prejudices) that if one team played 2 games in the last week and the other team hadn’t played any, then the odds wouldn’t be even (ceteris paribus). Injuries matter. In this study, was the potential for injury (offense) included as a risk? Was the offense’s ground (short) game (momentum) included as a variable? Was the FACT that not achieving a conversion has a psychological COST on the team’s playing? I doubt it.

43 BC February 6, 2017 at 8:33 pm

Maximizing expected points scored is not the same as maximizing the probability of winning by 1 or more points because winning by 2 (or 3 or 4,…) is not more valuable than winning by 1. Maximizing expected points might be the optimal strategy if each team played the game without knowing what the other team has scored and the teams compared scores only after the game was over, but that’s not how the game is played. Instead, each team can adjust its strategy based on the other team’s score. Thus, when a team goes for 2, it gives a free option to its opponent: the opponent gets to know the outcome of the 2-pt try in deciding whether it wants to go for two after subsequent touchdowns.

As a simple example, consider college overtime, where each team gets one possession. Suppose Team A has first possession and scores a touchdown. If it goes for two, then there will be an asymmetry of information. If the conversion fails, then Team B will know that it doesn’t need to go for two if it scores a touchdown. Team B can win with a simple kick with near certainty. If A’s 2-pt conversion is successful, however, A is still not guaranteed a win. Team B will know that it needs to go for two after its touchdown and might make it. So, if 2-pt conversion probability is about 0.5 and 1-pt probability is about 1.0, then conditional on each team scoring a touchdown, A will win with probability of only about 0.5*0.5 = 0.25 (A makes, B fails). Team B will win with probability 0.5 (A fails, B goes for 1). The teams will tie and go to the next overtime with probability 0.5*0.5 = 0.25 (A makes, B makes). Given the conditioning that each team scores a touchdown, the probabilities should be symmetric. Instead, due to A’s suboptimal strategy, B has an advantage because there is no scenario where A fails and B also fails (because if A fails, B will kick for 1 instead of going for 2). Conversely, suppose A goes for 1. If B goes for 1, then the teams will tie and go to the next overtime. If B goes for two, then A will win with probability 0.5 (B fails) and B will win with probability 0.5 (B makes). The results are symmetric for A and B as expected.

I conjecture that this analysis can be extended by some sort of induction to cases where each team scores multiple touchdowns, as would be the case of regulation (non-overtime). Every time a team goes for two and misses, its opponent can gain a 1-pt advantage with its next touchdown. On the other hand, a successful 2-pt try does not guarantee a 1-pt advantage because the opponent can also go for 2 after its next touchdown.

There may be some high scoring cases where the option value conceded to one’s opponent doesn’t matter much early in the game because one will also know the outcome of those early 2-pt tries in adjusting one’s own subsequent strategy. However, I don’t see how one can gain an advantage by going for 2 always because one’s opponent could do the same thing. Conversely, going for 1 conveys no option value to one’s opponent (because the outcome is nearly certain). I would think that the difference in expected points between 2-pt and 1-pt tries would need to be much more than 0.01 to overcome the option value.

Regardless, I don’t know why these papers about going for 2, going for it on 4th down, never kicking field goals, etc., always seem to consider only expected value and never give any consideration to game theoretic aspects, even though these are literally games. It might just be possible that playing strategies that have been developed from experience over many decades might not be completely irrational, even though the players and coaches don’t have PhDs.

44 TheRealInterest February 6, 2017 at 8:43 pm

It seems very, very, very unlikely the conversion rate of two-point conversions would be the same under “routine” conditions as they would under the current “do-or-die” conditional distribution. The effects of pressure may be ambiguous, but the rarity (and lack of tape to study) of two-point conversions surely favors the offense. Do those in favor of the riskier strategy really think NFL plays are just the results of a bunch of random number generations? Coaches and players spend hours and hours studying each others’ playbooks, strategies, and tendencies. *EVERY* play in football carries with it the cost that it will defenses will be more familiar the next time you run it. That cost is especially high when your play choice set is constrained, be it by the clock, down, or yards to go. (85%+ of the time you hear fans who have never played ask “What was that coach thinking when he ran that play?”, the answer is “Preservation of choice.”)

How’s this quote from the paper make for a Lucasian/Straussian read?

“While teams on average have slightly increased the frequency of 2-point attempts, the Pittsburgh Steelers have been arguably the only team to dramatically step up 2-point attempts, trying 20 over the course of the 2015-2016 regular seasons. This likely was a result of identifying from the 2002-2014 time period that the Steelers had the highest 2-point conversion rate in the NFL above 75% when using data from 2002-2014. However, this practice was halted after the Pittsburgh Steelers loss to the Cowboys on November, 13, 2016 where 4 two-point conversions were missed in one game contributing to the Steelers 30-35 loss. ”

The Steelers were 8-for-11 in 2015 on two-point attempts, and 3-for-9 this year.

45 Eddie February 6, 2017 at 9:38 pm

“The effects of pressure may be ambiguous, but the rarity (and lack of tape to study) of two-point conversions surely favors the offense. ”

Agreed. The authors have obviously never coached.

Teams do not just randomly run interchangable plays in 2-point conversion situations. In the process of gameplanning, they select a handful two-point plays they think are likely to work this week against this opponent; since teams rarely go for two more than once or twice a game, in practice, that means that something like 90% them the offensive coordinator’s “first-choice play” for that situation. If a head coach decided to go for two after every TD, by definition you’d end up running the offensive coordinator’s third, fourth, and fifth choices, and the conversion rate would drop.

Comments on this entry are closed.

Previous post:

Next post: