What are humans still good for? The turning point in Freestyle chess may be approaching

by on November 5, 2013 at 5:58 am in Games | Permalink

Some of you will know that Average is Over contains an extensive discussion of “freestyle chess,” where humans can use any and all tools available — most of all computers and computer programs — to play the best chess game possible.  The book also notes that “man plus computer” is a stronger player than “computer alone,” at least provided the human knows what he is doing.  You will find a similar claim from Brynjolfsson and McAfee.

Computer chess expert Kenneth W. Regan has compiled extensive data on this question, and you will see that a striking percentage of the best or most accurate chess games of all time have been played by man-machine pairs.  Ken’s explanations are a bit dense for those who don’t already know chess, computer chess, Freestyle and its lingo, but yes that is what he finds, click on the links in his link for confirmation.  In this list for instance the Freestyle teams do very very well.

Average is Over also raised the possibility that, fairly soon, the computer programs might be good enough that adding the human to the computer doesn’t bring any advantage.  (That’s been the case in checkers for some while, as that game is fully solved.)  I therefore was very interested in this discussion at RybkaForum suggesting that already might be the case, although only recently.

Think about why such a flip might be in the works, even though chess is far from fully solved.  The “human plus computer” can add value to “the computer alone” in a few ways:

1. The human may in selective cases prune variations better than the computer alone, and thus improve where the computer searches for better moves and how the computer uses its time.

2. The human can see where different chess-playing programs disagree, and then ask the programs to look more closely at those variations, to get a leg up against the computer playing alone (of course this is a subset of #1).  This is a biggie, and it is also a profound way of thinking about how humans will add insight to computer programs for a long time to come, usually overlooked by those who think all jobs will disappear.

3. The human may be better at time management, and can tell the program when to spend more or less time on a move.  “Come on, Rybka, just recapture the damned knight!”  Haven’t we all said that at some point or another?  I’ve never regretted pressing the “Move Now” button on my program.

4. The human knows the “opening book” of the computer program he/she is playing against, and can prepare a trap in advance for the computer to walk into, although of course advanced programs can to some extent “randomize” at the opening level of the game.

Insofar as the above RybkaForum thread has a consensus, it is that most of these advantages have not gone away.  But the “human plus computer” needs time to improve on the computer alone, and at sufficiently fast time controls the human attempts to improve on the computer may simply amount to noise or may even be harmful, given the possibility of human error.  Some commentators suggest that at ninety minutes per game the humans are no longer adding value to the human-computer team, whereas they do add value when the time frame is say one day per move (“correspondence chess,” as it is called in this context.)  Circa 2008, at ninety minutes per game, the best human-computer teams were better than the computer programs alone.  But 2013 or 2014 may be another story.  And clearly at, say, thirty or sixty seconds a game the human hasn’t been able to add value to the computer for some time now.

Note that as the computer programs get better, some of these potential listed advantages, such as #1, #3, and #4 become harder to exploit.  #2 — seeing where different programs disagree — does not necessarily become harder to exploit for advantage, although the human (often, not always) has to look deeper and deeper to find serious disagreement among the best programs.  Furthermore the ultimate human sense of “in the final analysis, which program to trust” is harder to intuit, the closer the different programs are to perfection.  (In contrast, the human sense of which program to trust is more acute when different programs have more readily recognizable stylistic flaws, as was the case in the past: “Oh, Deep Blue doesn’t always understand blocked pawn formations very well.”  Or “Fritz is better in the endgame.”  And so on.)

These propositions all require more systematic testing, of course.  In any case it is interesting to observe an approach to the flip point, where even the most talented humans move from being very real contributors to being strictly zero marginal product.  Or negative marginal product, as the case may be.

And of course this has implications for more traditional labor markets as well.  You might train to help a computer program read medical scans, and for thirteen years add real value with your intuition and your ability to revise the computer’s mistakes or at least to get the doctor to take a closer look.  But it takes more and more time for you to improve on the computer each year.  And then one day…poof!  ZMP for you.

Addendum: Here is an article on computer dominance in rock-paper-scissors.  This source claims freestyle does not beat the machine in poker.

Steve Sailer November 5, 2013 at 7:20 am

Or maybe people lose interest in chess when it’s just computers playing against each other, just like we don’t find John Henry v. the Steam Shovel competitions exciting anymore. Humans have a remarkable talent for becoming bored.

Nelson Hernandez (Intagrand/Cato Freestyle team) November 5, 2013 at 8:48 am

Probably right for most people, Steve. But there is an irrepressible geek element out there, and God bless ‘em, they like creating awesome software which allows non-programmers to witness some of the best chess games that have ever been played. At this site (live games on hiatus for a couple of days) you can see what I’m talking about.

http://tcec.chessdom.com/live.php

Finch November 5, 2013 at 9:37 am

Working against that is the fact that computers have made world class opponents easily available to everyone, so training and good practice are more accessible and it’s easier to get good fast.

I think you might be right if you are talking about people outside of the chess community, but my impression is that the marked inferiority of humans doesn’t bother most chess players at all. People still have footraces despite easy access to cars.

Rahul November 5, 2013 at 7:59 am

I find the underlying assumption amusing: that chess might say anything about the general usefulness of humans.

For more than two decades now computers have been better at chess than the average person. Becoming better than the best human may be a commendable feat, but I doubt it says much from the general usefulness of humans.

david November 5, 2013 at 8:04 am

Even today, with more than a century of dramatic innovation and implementation as consumer goods, the horse is still better than the automobile at tasks as simple, and as crucial, as avoiding obstacles.

Still, that is poor comfort for the horse.

Brian Donohue November 5, 2013 at 8:16 am

Agreed.

Regarding the robots taking over, I just saw this: http://management.fortune.cnn.com/2013/11/05/brain-drain-us-manufacturing/?iid=HP_LN

RR November 5, 2013 at 8:34 am

I think I will really begin to worry when a computer leaves better comments at MR. But really seeing repetitive ideological (programmed?) comments being expressed by some of the commenters across various posts , perhaps the computers are already in action.

Z November 5, 2013 at 10:05 am

It is always the most ideological who bemoan ideology.

RR November 5, 2013 at 12:08 pm

Touche!

Nelson Hernandez (Intagrand/Cato Freestyle team) November 5, 2013 at 8:41 am

Tyler, the same holds true today as held true in Freestyle’s heyday of 2005-2008. If you have let’s say the top six chess programs analyzing the same position on equally strong multiprocessor hardware, and conveniently arrange their streaming outputs in such a way that one could compare their respective evaluations and principal variations at a glance, there is no question in my mind that a skilled freestyler could add value by simply identifying and exploring promising outlier moves/lines while his array of programs continued to crunch away. As you say above, the human could also manage time better than this array in most cases, taking more time in ambiguous positions where there were several viable candidate moves and less where the move ought to be near-instantaneous, as in a recapture of a major piece for example, or a position where all the engines in the array immediately see that one move is far better than any other choice in the present position.

This is particularly true in the endgame. Some programs are markedly better than others in this stage of the game. A person might be referring to his full array of chess programs right after exiting book, but be relying principally on one or two programs when the number of pieces left on the board had been whittled down.

But, in theory, all of these human judgment functions could be automated as well if you had powerful-enough hardware and a great programmer with freestyle experience and intuition. It hasn’t been done yet, but it certainly could be.

It would also be possible to marshal an engine array like I’ve described and wed it to a super-powerful opening book, and eventually 7-man endgame tablebases. No question in my mind you’d have the strongest chess entity in the world if you could put all that together and regularly update the engines in your array with the latest and greatest programs, which these days continue to improve on an almost day to day basis.

Steve Sailer November 5, 2013 at 8:46 am

People are still interested in the 100m dash at the Olympics, even though funny cars or cruise missiles can cover 100m a lot faster than human beings can.

The power of boredom is much underrated. All sorts of competitions have gone out of existence. For example, airplane racing around pylons was a spectacular sport in the 1930s, attracting huge crowds. Today, it’s just something for rich hobbyists to do. I can’t recall knowing the name of a champion airplane racer in decades. The technology just got too advanced to be interesting anymore.

Maybe the same will happen to chess, or maybe humans will figure out how to keep other humans from using computers to cheat and the game will survive, the way track and field has survived the motorcycle.

Perhaps, in the future rich people will pay ZMP workers to be part of their entourage, just like in royal courts of olde, or just like in Hollywood or the NBA.

Z November 5, 2013 at 10:14 am

Tennis and golf come to mind. Tennis, I think, hurt itself badly by allowing every player to become a human cannon. Serve and volley tennis, which favored strategy, has given way to power tennis, where supermen fire tennis balls at 150 mph at one another. Golf, in contrast, put the brakes on technology so it remained a game of skill. Demographics has a role here, but the use and abuse of technology has had an impact on both sport’s popularity.

Steve Sailer November 5, 2013 at 10:42 am

The reverse is probably more true. Federer is clearly a better tennis artist than my heroes of the 1970s, and his younger rivals might wind up being even better than him.

In contrast, golf has gotten less elegant and more pragmatic. Nobody hits tee shots anymore with the artistic trajectory of Jack Nicklaus (start out inches off the ground, then ascend to towering height, then drop straight down with hardly any roll). They’ve all used video technology to find their perfect launch angle for maximum distance. Off the tee, they play flog-it-and-find-it: bomb it off the tee and use one of your four wedges to gouge it out of the rough and on to the green. Curving approach shots left or right into the green isn’t fashionable anymore because it make more sense to hit irons straight and high.

I’m not being an old fogeyist: Woods and Mickelson are likely the greatest artists the game has ever known, but they (especially the cold-blooded Woods) have rationalized the game. Ironically, Jack Nicklaus, who intimidated his opponents with his steely Teutonic personality, now seems like a romantic who did self-destructive things (like hitting a 1-iron off the tee in the crux) because he had elegant but wrong theories about how to play the game.

Bob November 5, 2013 at 12:31 pm

The issue with golf is the same Formula 1 had for a while: Technology changed while the courses didn’t, so what once was challenging then became simple. You either send back the technology to keep the courses interesting, or change the courses to match the technology. A car can now go flat out on a turn that before was extremely technical? Weaken the car, or design a turn that challenges a car with modern aero.

You can design a hole so that hitting a rough will be more punishing than it is now. The problem is that you have regular people playing in the same courses as pros. They are too far away from us now.

Steve Sailer November 5, 2013 at 8:21 pm

Right. For example, grand old Merion proved an excellent test at this year’s U.S. Open despite being less than 7000 yards long, but the members had to put up for a long time with the course being rendered almost unplayable by mortals to get it ready to defend par against the world’s best.

Keith November 5, 2013 at 10:51 am

Z, I am with you on your tennis comment. I find it to be nearly unwatchable now. Steve’s comment is true that Federer is probably the best player ever, but he would probably still be the best ever and more enjoyable to watch if everyone used the old wooden tennis rackets.

Z November 5, 2013 at 11:57 am

Tennis is one step away from having the players use potato guns to fire tennis balls at one another. That would probably be more interesting. Contra Steve, Federer still had to possess the raw power to even get a chance to play. What makes Federer unique is he had skill and power, while just about everyone else is trying to exceed the speed of sound with their serve.

We have to allow for taste here, of course. Tennis has fallen out of favor for a few reasons, demographics being one other them. In the 60′s, 70′s and 80′s, Boomers were in the prime and able to play tennis. Then they got too old to play and moved onto golf. That has had something to do with it too. Still, the dullness of tennis is due in part to the abuse of technology.

The Anti-Gnostic November 5, 2013 at 2:36 pm

Yes. Tennis was huge. It’s interesting to see all the old condo, apartment, even office parks in my town with old, crumbling tennis courts that nobody’s used in years.

Steve Sailer November 5, 2013 at 8:28 pm

Sports go through phases. Remember how racketball was the sexy sport of 1975-1985? It seems unappealing now, but people couldn’t get enough of it then.

Golf isn’t really the old man’s game as assumed. Participation in golf is probably highest among youngish married men, who are a little too old for contact sports and maybe their wives’ cooking has padded on a few pounds so they don’t want to run a lot, but they still have fairly high testosterone levels. As T levels fall, the urge to play golf declines, although not as fast as the urge to play more violent or strenuous sports.

Alex Godofsky November 5, 2013 at 8:51 am

There doesn’t seem to be any reason that #2 couldn’t be automated.

Rahul November 5, 2013 at 9:23 am

Some of the top algorithms at the Netflix prize were using multiple algorithms to rate and then aggregating their answers in some way. I think such ensemble methods are already used and studied in the pattern recognition / classification community.

gwern November 5, 2013 at 10:38 am

Yes, far from being exotic or novel, ensembles are standard in machine learning. Bagging, boosting, model counting/averaging/expansion… Heck, one of the most common tools, random forests, is just a whole bunch of decision trees voting.

Michael D. Abramoff November 5, 2013 at 11:46 am

Yes, I don’t see either why deciding between competing algorithms would be better done by humans. Anyway we are widely using it in diagnostic algorithms.
More worrisome is that until now, at least humans were needed to design the filter or detector algorithms used as inputs to the machine learning algorithms. That is now going rapidly away also, where data driven detectors are created automatically, at least in my field.

http://www.ncbi.nlm.nih.gov/pubmed/21292586

Finch November 5, 2013 at 9:53 am

Multi-agent approaches have been tried in games like Go and poker. A quick Google Scholar search didn’t turn up something for chess, but I’d be surprised if it wasn’t out there. I think Nelson usefully discusses the possibility above at 8:41am. It seems pretty straightforward, although the devil’s in the details.

I think a lot of the weird or new stuff gets tried in other games as Chess is so well understood at this point that the payoff for programming and research time is generally in some new form of pruning or parallelism or minor enhancements to board evaluation. MCTS (Monte Carlo Tree Search) in Go is the most interesting thing to come of out computer game playing in ages, but nobody’s made it work very well in chess because chess is so tactical and because traditional methods have such a big head start. In Go the traditional methods are abject failures due to the greater complexity and more strategic nature of the game, so MCTS was able to get a foothold.

Tyle November 5, 2013 at 3:53 pm

True, and the same goes for 1, 3, and 4. I understood this to be Tyler’s point. “Think about why such a flip might be in the works…a human can add value in a few ways”…each of which seems like it could be automated.

Jack Sparrow November 6, 2013 at 1:52 am

It can’t be automated because feature selection is a critical component for any machine learning algorithm. So at some level humans will need to pitch in.

Jack Sparrow November 6, 2013 at 1:54 am

Though their contribution might appear as trivial as compared to that of the computer!

Sammler November 5, 2013 at 8:55 am

The cited paper is not of much value, because the search depth used to test the move quality is too shallow. For example, at the Thoresen Chess Engines Competition (now running its second season, see http://tcec.chessdom.com) Stockfish routinely searches with depths of 40 and up. Even given the older and lower-quality sample of games analyzed by Regan, depth 19 is likely vulnerable to moves which are too high-quality for the analysis engine, and thus will be labeled as suboptimal.

(TCEC, by the way, is an all-computer event played at long time controls.)

Kenneth W. Regan November 5, 2013 at 11:53 am

Sammler, indeed this is an issue. Matej Guid and Ivan Bratko defended their use of much shallower analysis for relative comparisons on grounds that it should still correlate most with overall quality of moves—there may not be a Platonic standard for “quality” but engines show convergence as they improve. To be sure, I am striving for absolute not just relative quality, but there may be enough signal to work with. As indication, here is my table of analysis run overnight on the just-completed nTCEC Stage 3. Seven of the ten engines finished tightly bunched, as befits their all having come through two earlier stages, but especially the last column distinguishes the two tailenders clearly and gives the second-best bump to the winner.

Rahul November 5, 2013 at 12:59 pm

Is there a reason why you chose depth 19 instead of 40 for the study?

Kenneth W. Regan November 5, 2013 at 7:38 pm

Long answer: the reason is time. The links show I’ve analyzed over 4,000 performances in all the major computer events since the 1990′s, plus Freestyle and Correspondence, making exactly 2,170,451 moves (1,836,780 retained), all since Sep. 1. To compare, in Stockfish 4′s second game in Stage 1 (the first went quick to endgame) against Hannibal, at Move 33 in position 2r5/3qb1k1/p2p2p1/Bp1Ppn1p/P1P1N3/1Q3P2/6PP/1R4K1 w – b6 0 33, TCEC shows Stockfish reaching depth 35 in 1:38 searching about 1,706,500,000 nodes. My compile of the same version, using 1 core of our department’s general-access scientific machine, depth 35 takes 23:42 going thru 1,943,595,325 nodes. The extra nodes and some delay may owe to my starting with a clear hash table, and I run 10 or so analyses in-parallel each on 1 core (which is 50% more efficient overall than multicore), so I could get on the TCEC timeframe. But then in 2 months I’d only have run the current TCEC games!

Short answer: the reason is time. Also, my full theory asserts that humans operate in a spectrum of depths, so for human games I may get better predictions from depths in the range I use, rather than depth 35. Plus I’ve tested allegations of cheating at Rapid time controls and/or using engines running on smartphones.

Other answer: the Multi-PV analysis needed for my full theory takes 20x more time. I’m still in process of applying it to the Freestyle data—I need to finish calibrating Stockfish’s “expanded” values to supplement my Rybka 3-based model and correct tilt toward Rybka 3 (which many Freestylers used too). The real mystery IMHO is why CEGT 40/120 games give much higher results than my similar number of computer world-championship and major-tourney performances. It’s struck me only this week that I can truncate many overlong CEGT games fairly by applying TCEC’s sensible draw rule to them, but this will take time.

Rahul November 6, 2013 at 12:14 am

Thanks. If you get access to a cluster perhaps you can do the depth 40 analysis to refute or bolster the criticisms like that by @Sammler.

At least fortunately, resolving it seems “only” a matter of throwing more computers at the problem not an analytical or data issue per se.

Sammler November 6, 2013 at 5:20 am

Thank you for a considered reply. I don’t find the table compelling, but given the very small sample that is too much to ask.

Could the hypothesis that shallow analysis is acceptable be tested by studying the convergence from even shallower analysis? For instance, if depth-19 testing of a depth-28 tournament like TCEC is valid, then the results should not differ much from those obtained at depth 17 or 15. Obtaining this convergence test is computationally cheap compared to what you are already doing.

Good luck in your work; it is quite interesting to the layman.

Kenneth W. Regan November 6, 2013 at 10:31 am

Indeed, that is exactly the stage we are doing, since my student and I have expanded our data format to include results at all depths (in MultiPv format). This also involved reprogramming Stockfish to clear hash on every “go depth ..” command, and using EPD files for other engines. We will also do some extensions for depths 21 and higher.

Kenneth W. Regan November 22, 2013 at 9:19 am

I have added the Stage 4 results to the same file. Only Naum 4.6 is out of perfect rank order, in both the AE and AE-difference columns.

Rahul November 5, 2013 at 9:29 am

Two points: First, how replaceable humans will turn out to be in practical settings won’t be judged by the quality of the computer competition alone but by the price point as well.

Second, in a major chunk of applications it won’t be the brainpower that decides but the sensors. Again, not the quality but the price-quality trade-off.

There’s a huge bunch of industrial applications where a powerful computer coupled to fancy sensors would get the job done but a human turns out to be cheaper. Sensors have gotten better and cheaper but not as much as computers.

john personna November 5, 2013 at 9:58 am

Isn’t Fukushima the reminder of what humans, computers, and robots can’t do (fix)?

Finch November 5, 2013 at 10:19 am

Irrational fear of science? The frigging tidal wave killed 10,000 people and the press focused on a comparatively minor industrial accident. We live in a strange world.

john personna November 5, 2013 at 10:37 am

A swing and a miss. I am a scientist by training (engineer by profession, retired by good fortune).

Finch November 5, 2013 at 10:57 am

Then you sure chose a strange example. I do not think it means what you think it means.

john personna November 5, 2013 at 11:06 am

I am very confident that people who respond in the cute-hit style have nothing. If they had an argument it would be easy to lay out. I mean, is Fukushima recovery not an ongoing project? Does it not have hard engineering problems? Were not a lot of those rooted in the need for humans to do certain vital plant tasks, but with radiation the humans were excluded? Did they not find that robots, while helpful, could not step in and operate as humans had done? Did not that all combine to make a disaster site which neither humans, nor robots, nor the combination can restore to health and safety?

Finch November 5, 2013 at 11:19 am

If that’s what you meant, you needed to say more. It was not possible to read that from your initial posting. The major Fukushima story is people are afraid of technology and their reactions to problems are way out of proportion. That’s the big takeaway. I took you to be saying what most of humanity would have meant if they wrote what you did.

john personna November 5, 2013 at 11:26 am

This is a thread on “what are humans good for.” Anyone with a scientific or engineering bent, who understood the Fukushima saga, would have been right onto what I was saying. He would not have gone off with “Irrational fear of science?”

Finch November 5, 2013 at 11:30 am

I suggest you reread what you wrote. There wasn’t much in the comment.

I agree robots are not plug and play substitutes for humans yet.

john personna November 5, 2013 at 11:34 am

You have come to accept what I wrote, but you want to stick with your “Irrational fear of science?” insult? Seriously?

Finch November 5, 2013 at 12:00 pm

Your comment at 9:58 am had none of the content of your comment at 11:06 am and appeared to be inciting irrational fear of nuclear power. You were misleading at 9:58 am if you actually meant what you subsequently wrote at 11:06 am. I have no problem with the statement that robots are not currently plug and play substitutes for humans. I don’t think that’s controversial.

If you want to draw a general lesson from Fukushima, you need to say “nuclear power: proven safer than the alternatives yet again.”

john personna November 5, 2013 at 12:01 pm

I think Finch that you inferred something I did not imply, and are now too stubborn to admit it.

Finch November 5, 2013 at 12:12 pm

You said “Isn’t Fukushima the reminder of what humans, computers, and robots can’t do (fix)?”

If the “what” in that sentence wasn’t “nuclear power,” and was instead “fully emulate humans with robots,” I apologize for misinterpreting you. That wasn’t the obvious interpretation. I inferred that you meant nuclear power.

john personna November 5, 2013 at 12:23 pm

Again, anyone familiar with the current problems at Fukushima would know what I was talking about. You are really defending the less-informed viewpoint, that Fukushima was just some event in the past, over, and so on. Honda Developing Disaster Response Robot Based on ASIMO

ed November 5, 2013 at 1:39 pm

John, most people aren’t familiar with “the current problems at Fukushima.” Not even most people who read this blog. If you assume that they are, you are in danger of being misunderstood, as happened here.

john personna November 5, 2013 at 1:58 pm

but they’re ready to attack any comment on a hair trigger.

Finch November 5, 2013 at 2:18 pm

As a person who cares about the environment and is concerned global warming may turn into a real problem, I’m sensitive to the common tactic of sowing fear, uncertainty, and doubt about nuclear power, which is the only clear solution several important environmental and geopolitical problems.

I’m still not convinced that’s not what you were trying to do, but if it really wasn’t I both sincerely apologize and encourage you to think more carefully about what you write and how it will be read.

john personna November 5, 2013 at 2:30 pm

Given that I was correct, I think I had every right to be caustic with you, Finch, following your emotional outburst.

I mean, I came into this knowing that this was not the first time humans, robots, jobs and their transitions, had crossed the MR news desk. You yourself commented on Only the Cyborgs Can Compete With the Robots. See also “Your robot anesthesiologist (the forward march of progress)”, “Designing robots to work with humans”.

And now still after that you want to hold out some hope that I’m the one being stupid and emotional?

Shame on you.

Finch November 5, 2013 at 3:10 pm

I’m still not convinced that’s not what you were trying to do (sow FUD), but if it really wasn’t I both sincerely apologize and encourage you to think more carefully about what you write and how it will be read, especially if the obvious interpretation is inflammatory.

I don’t know what more I can say, so I’m going to leave it at that.

john personna November 5, 2013 at 3:32 pm

It isn’t actually an apology when you pair it with an accusation (unsubstantiated) every time.

Dan Weber November 5, 2013 at 11:20 am

In many ways it would have been better for nuclear power had Fukushima killed a bunch of people on its own and then stopped being an event.

Instead, it has done something much worse than killing a lot of people: continued to inconvenience the living.

Human irrationality is amazing.

Finch November 5, 2013 at 11:24 am

I still find the German reaction appalling. Is global warming just not a thing anymore? Is anyone using their brain?

john personna November 5, 2013 at 11:28 am
john personna November 5, 2013 at 11:38 am

A better link, a summary from the WHO, which seems to suggest that despite some fairly long half-lifes, a localized problem. Dan Weber – with Cs-137, 30 year half-life, it isn’t really “human irrationality” keeping the issue around.

Dan Weber November 5, 2013 at 11:51 am

That’s not what I said. I was comparing the effects of actually killing a bunch of people and being over with the effects of inconveniencing a bunch of people for years. The first is clearly much worse than the second for similar values of “bunch.” But the dead don’t speak while the inconvenienced living (“I can’t eat seafood!”) can bitch to high heaven.

There’s the old story about the best way to disable an army isn’t to kill half of its soldiers, but rather just injure half of its soldiers, because the unwounded need to care for the living, while they would just leave their dead.

john personna November 5, 2013 at 12:00 pm

You are showing some strange emotion there, in “bitch to high heaven.” Isn’t it now just a real and ongoing cost (and reduction in seafood opportunity) for there to be radiation testing programs in place, as a result of Fukushima?

In that WHO report they mention that no seafood has shown up with radioactivity in Alaska, which means of course someone is testing Alaskan seafood as well. These are real costs, in dollars, in manpower, in management.

I mean, I get that people like to put down fear of nuclear power as an emotional thing, but you seem to be the one putting emotion before real and measurable costs.

Dan Weber November 5, 2013 at 12:27 pm

FWIW, you have convinced me you are an engineer.

Nate November 5, 2013 at 10:05 am

I’m the guy (from the tweet in the addendum) who interviewed the creator of PokerSnowie, the poker neural net. I’m sorry if I was unclear in my tweet–the creator believes that he–a pretty good poker player–could _not_ improve on the neural net’s decisions.

FWIW that claim surprised me, and I do think a freestyle team could beat the best poker bots.

Tyler Cowen November 5, 2013 at 11:05 am

Thanks, I will revise.

Nate November 5, 2013 at 11:39 am

Thank you!

Yancey Ward November 5, 2013 at 10:50 am

As long as the computers disagree with each other, a human input can help make better decisions. I see no reason for the depth of the disagreement to matter either since the very nature of the output allows the the human to see the disagreement the instant it arises.

john personna November 5, 2013 at 11:18 am

Maybe when humans disagree, a computer could help make better decisions. (Imagine a comments system with automatic fact-check)

Yancey Ward November 5, 2013 at 11:22 am

It would put you out of business here.

john personna November 5, 2013 at 11:43 am

My recollection is that I put a lot more links in, to supporting documentation, than you do. Examples above.

Bob November 5, 2013 at 12:35 pm

You could also create a program that asks chess engines to analyze a position, and make its own judgement calls when they disagree. It could compare success rates of different engines in different situations, or ask the differing engines to look deeper in their area of disagreement.

It’s not complicated at all.

Tyle November 5, 2013 at 11:03 am

“I’ve never regretted pressing the “Move Now” button on my program.” Then apparently you aren’t pressing it often enough.

Corporate Serf November 5, 2013 at 11:50 am

Kind of similar problem as “Which forum to read?” Should I just read Rybka forum, or read marginal revolution.

Ray Lopez November 5, 2013 at 12:37 pm

What is kind of cool is getting a freeware chess program, dialing down the strength by making it play faster, and beating the chess program. It can be done and it’s quite fun. PCs can blunder. As for this thread, PCs are good at some P/ NP problems such as 8×8 chess, due to the alpha-beta and other algorithms developed over the years for playing 8×8 chess. More info here: http://en.wikipedia.org/wiki/P_versus_NP_problem That does not mean PCs can always solve NP problems, see the above.

An Onyx Mousse November 5, 2013 at 12:48 pm

Much more interesting than the domain-specific problems is the developing art of “making decisions with computers”, and even more interesting than that is the technology of “teaching computers to make decisions through training from humans”. Imagine several different calculation engines, with a neural net choosing the best one based on complex criteria, e.g. “Fritz is better at endgames and Fritz is VERY sure it is correct here, while Rybka isn’t so sure.” Rather than becoming ZMP, with the right technology, the “average” humans will start interacting with computers to teach them ever more things – imagine Mechanical Turk set-ups where humans teach neural nets or learning machines to drive vehicles in complex situations, audit tax returns, write news articles, insert catheters, etc. With the right setup, relatively cheap non-programmer humans could start to be able to teach computers how to do things the same way we teach children, which will lower the cost of developing very task-specific automated systems. Teaching computers and robots could be made “labor intensive” rather than capital intensive. This will start to happen when demand & prices for skilled programmers gets high enough, and price of “average” labor gets cheap enough in real terms – the U.S. isn’t there yet, but we are heading in that direction. Of course once robots are good at all the “simple” tasks the cycle starts again, and many more people are now ZMP than before -but also much richer in real terms with lots of well-functioning automated systems everywhere.

Ray Lopez November 5, 2013 at 1:06 pm

Or pace the sci-fi dystopia “Elysium (2013)”, which reminded me of life here in the Philippines, lol. Life is good for the 1% like me. For everybody else, “it is improving”. BTW being 1% here is like being 10% in the USA–it is not a lot of money, relatively speaking.

Barkley Rosser November 5, 2013 at 4:45 pm

I am still waiting for the computer that can beat the world’s top Go players.

Oh, and Fukushima was about human error, not some computer program.

Kenneth W. Regan November 5, 2013 at 7:56 pm

Hi from me. I subscribe to a “Moore’s Law for Games” based on Laszlo Mero’s notion of “class unit”. One class unit means 75% expectation for the stronger player over the weaker, and in chess this is about 200 Elo points. Using a source from the 1990′s that pegged “bright beginner” in chess at 600 Elo (alas I’ve lost track of it, and never mind that young children’s ratings go clear down below 100), while 2800 has been the mode for world-champions’ ratings since Fischer until recently, chess has a “depth” of 11 class units. My source pegged Checkers at 10 and Shogi (Japanese chess) at 14—Shogi games are longer since captured pieces are recycled. With computers rated about 3200, they are at Mero level 13, which is corroborated by their being close but still not supreme at Shogi. Go does not use a similar rating system, and games are played at handicap so it’s hard to gauge 75% expectation from a level start, but my source estimated Go at level 25–40.

The mantra in chess is 2x speed = 50 Elo at depths in the teens, diminishing above 20 but Rybka’s creator said not as much as expected. So maybe going up one Mero level needs 16x–32x speed. So at the original 2x = 18 months pace of Moore, that’s 6–8 years, which makes sense since it’s 16 years since Deep Blue beat Kasparov (with about 2900 performance by my measures). Wikipedia notes a slowdown in Moore to 2x = 3 years at least since 2010, so now we’re talking 12–15 years per level. If the “25–40″ estimate is accurate, you would have to wait about 150 years. Well maybe another algorithmic advance to supplement Monte Carlo trials will crack it sooner…

Floccina November 6, 2013 at 10:50 am

Some jobs might be very vulnerable. I wonder if we still need pharmacists?

Comments on this entry are closed.

Previous post:

Next post: