Web/Tech

The great Ken Regan on AlphaZero

by on December 18, 2017 at 6:20 pm in Games, Science, Web/Tech | Permalink

A must-read for anyone who has been following this issue, Ken considers how close to God AlphaZero actually came:

We must pause to reflect on how clarifying it is that this single heuristic suffices to master complex games—games that also represent a concrete face of asymptotic complexity insofar as their size n-by-ngeneralizations are polynomial-space hard…

It may be that we can heuristically solve some NP-type problems better by infusing an adversary—to make a PSPACE-type problem that hits back—and running AlphaZero.

That sort of thing.  And don’t neglect the comments.

Star Wars and net neutrality

by on December 18, 2017 at 1:49 am in Film, Web/Tech | Permalink

That’s why the latest Star Wars trilogy is so dark: It’s looking more and more like real life, where the credits never roll and problems can always recur…The Last Jedi premiered a few hours after we learned that the FCC had reversed its stance on net neutrality; we all sang “Yub Nub” back in 2015 when that vote went one way, but we know now that the war wasn’t won.

Here is the Ben Lindbergh piece on the pessimistic themes in Star Wars.  Ted Cruz also serves up some unusual remarks.

I have some tricks for doing this, which I have scrawled in the inner part of the margin of the manuscript.  Here is Matt Levine on Bitcoin:

I half-joked yesterday that “perhaps the cost of bitcoin storage — keeping your private key in a vault, worrying about hackers, etc. — is so high that arbitrageurs need to charge $1,000 for a month of it,” but maybe it’s the right explanation? Everything I read about bitcoin storage is utterly exhausting. “A private key printed out on a sheet of paper, cut into pieces, and distributed among family members who don’t know how to put it back together; an encrypted file loaded on a USB stick and buried in the backyard; a password committed only to memory;” a private key engraved on a metal plate and stored in a safe; a safe deposit box at a bank; an account at an exchange that gets hacked and loses its customers’ bitcoins. Buying bitcoin futures is a way to get exposure to bitcoin and avoid the bitcoin-storage problem: You never have to store bitcoins because you never own bitcoins; you just get paid dollars for the amount that bitcoin goes up. But the storage problem doesn’t go away; you just offload it to the arbitrageur who provides you the bitcoin exposure. Maybe the arbitrageur needs to charge you $1,000 to cover her storage costs. If you think these markets are efficient, then the gap between the futures and the spot is telling you how much — in out-of-pocket expenses, in theft risk, in psychic pain — it costs to store bitcoin.

Here is more from Matt:

If I told you that there was an asset that is an excellent store of value even in inflationary conditions, how much of your gold portfolio would you reallocate to that asset? One percent? Three percent? Fifty percent? All of it? Sure, whatever, if you believed me. But we are just assuming that bitcoin actually fulfills that function, in order to decide its valuation. Bitcoin has had a pretty good run, but so far it’s a short one; there’s no historical experience of bitcoin retaining its value in periods of global financial crisis or rich-world inflation or even just, you know, people not talking about bitcoin for a minute. So far the evidence that bitcoin is a good store of value consists of the fact that bitcoin’s price keeps going up. That is not bad evidence! But it is not a ton of evidence to build a store-of-value valuation around.

If Matt told me, I would allocate at least two percent.

That is a new paper by Aghion, Akcigit, Hyytinen, and Toivanen, here is one brief excerpt:

In particular we see that IQ is by far the main characteristic for the probability of becoming an inventor in terms of the share of variation it explains, followed by parental education.  These two groups of variables account for 66% and 16% of the overall variation captured by our model.  In contrast, IQ plays a relatively speaking much more minor role for becoming a medical doctor or a lawyer.  Parental education is the main explanatory variable for the probability of becoming a medical doctor or a lawyer (40% and 53%), with base controls and parental income also playing clearly more important roles than for inventors.

The paper offers many other points of value, and you will note the data are from Finland.

That is the title of my latest Bloomberg column.  Rather than giving you the main argument, let’s instead cut to the end:

The real story of bitcoin is a heartening one of community. Less than 10 years ago, the bitcoin asset was worth virtually nothing, but a small group of people believed in it and worked tirelessly to promote it, and now the whole world is watching. It’s a tale at least as old as Christ and the Apostles. Maybe the bitcoin believers are as much of a miracle story as that of the brilliant inventor Satoshi.

The thing is, I don’t always believe in miracle stories of community, not in these days of declining governance and possibly fraying social order. Yet I’ve become emotionally involved in tracking the bitcoin price, perhaps because I realize that if one such miracle of “ex nihilo” creation can be sustained, others are on the way. I don’t think bitcoin is a bubble, but every morning I wake up doubting.

More broadly the piece considers what should be the appropriate price for Bitcoin.

“Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm”

The game of chess is the most widely-studied domain in the history of artificial intelligence. The strongest programs are based on a combination of sophisticated search techniques, domain-specific adaptations, and handcrafted evaluation functions that have been refined by human experts over several decades. In contrast, the AlphaGo Zero program recently achieved superhuman performance in the game of Go, by tabula rasa reinforcement learning from games of self-play. In this paper, we generalise this approach into a single AlphaZero algorithm that can achieve, tabula rasa, superhuman performance in many challenging domains. Starting from random play, and given no domain knowledge except the game rules, AlphaZero achieved within 24 hours a superhuman level of play in the games of chess and shogi (Japanese chess) as well as Go, and convincingly defeated a world-champion program in each case.

In other words, the human now adds absolutely nothing to man-machine chess-playing teams.  That’s in addition to the surprising power of this approach in solving problems.

Here is the link, via Trey Kollmer, who writes “Stockfish Dethroned.”  Here is coverage from Wired.  Via Justin Barclay, here is commentary from the chess world, including some of the (very impressive) games.  And it seems to prefer 1.d4 and 1.c4, loves the Queen’s Gambit, rejected the French Defense, never liked the King’s Indian, grew disillusioned with the Ruy Lopez, and surprisingly never fell in love with the Sicilian Defense.  By the way the program reinvented most of chess opening theory by playing against itself for less than a day.  Having the white pieces matters more than we thought from previous computer vs. computer contests.  Here is the best chess commentary I have seen, excerpt:

If Karpov had been a chess engine, he might have been called AlphaZero. There is a relentless positional boa constrictor approach that is simply unheard of. Modern chess engines are focused on activity, and have special safeguards to avoid blocked positions as they have no understanding of them and often find themselves in a dead end before they realize it. AlphaZero has no such prejudices or issues, and seems to thrive on snuffing out the opponent’s play. It is singularly impressive, and what is astonishing is how it is able to also find tactics that the engines seem blind to.

Did you know that the older Stockfish program considered 900 times more positions, but the greater “thinking depth” of the new innovation was decisive nonetheless.  I will never forget how stunned I was to learn of this breakthrough.

Finally, I’ve long said that Google’s final fate will be to evolve into a hedge fund.

1. Moore’s Law plus the internet makes smart people smarter, and stupid people less smart.

2. Manipulable people can be reached with a greater flood of information, so over time as data on them accumulate, they become more manipulable.

3. It is often easier to manipulate smart people than stupid people, because the latter may be oblivious to a greater set of cues and clues.

4. Social media bring smarter people together with the less smart more than used to be the case, Twitter more so than Facebook.  Members of each group are appalled by what they experience.  The smarter people see the lesser smarts of many others.  The less smart people — who often are not entirely so stupid after all — can see how manipulated the smarter people are.  They also see that the smarter people look down on them and attack their motives and intellects.  Both groups go away thinking less of each other.

4b. The smarter people, in reacting this way, in fact are being manipulated by the (stupider) powers that be.

5. “There is a performative dimension that renders both sides more rigid and dishonest.”  From a correspondent.

6. Consider a second distinction, namely between people who are too sensitive to social information, and people who are relatively insensitive to social information.  A quick test of this one is to ask how often a person’s tweets (and thoughts) refer to the motivations, intentions, or status hierarchies held by others.  Get the picture?  (Here is an A+ example.)

7. People who are overly sensitive to social information will be driven to distraction by Twitter.  They will find the world to be intolerably bad.  The status distinctions they value will be violated so, so many times, and in a manner which becomes common knowledge.  And they will perceive what are at times the questionable motives held by others.  Twitter is like negative catnip for them.  In fact, they will find it more and more necessary to focus on negative social information, thereby exacerbating their own tendencies toward oversensitivity.

8. People who are not so sensitive to social information will pursue social media with greater equanimity, and they may find those media productivity-enhancing.  Nevertheless they will become rather visibly introduced to a relatively new category of people for them — those who are overly sensitive to social information.  This group will become so transparent, so in their face, and also somewhat annoying.  Even those extremely insensitive to social information will not be able to help perceiving this alternate approach, and also the sometimes bad motivations that lie behind it.  The overly sensitive ones in turn will notice that another group is under-sensitive to the social considerations they value.  These two groups will think less and less of each other.  The insensitive will have been made sensitive.  It’s like playing “overrated vs. underrated” almost 24/7 on issues you really care about, and which affect your own personal status.

9. The philosophy of Stoicism will return to Silicon Valley.  It will gain adherents but fail, because the rest of the system is stacked against it.

10. The socially sensitive, very smart people will become the most despairing, the most manipulated, and the most angry.  The socially insensitive will either jump ship into the camp of the socially sensitive, or they will cultivate new methods of detachment, with or without Stoicism.  Straussianism will compete with Stoicism.

11. Parts of social media will peel off into smaller, more private groups.  At the end of the day, many will wonder which economies of scale and scope have been lost.  And gained.  Others will be too manipulated to wonder such things.

12. The “finance guy” in me thinks: how can I use all this for intellectual arbitrage?  Which camp does that put me in?

13.  What bounds this process?

Doug’s new book Clashing over Commerce: A History of US Trade Policy is the greatest book on trade policy ever written, bar none. and also a splendid work of American history more generally.  So I thought he and I should sit down to chat, now I have both the transcript and audio.

We covered how much of 19th century American growth was due to tariffs, trade policy toward China, the cultural argument against free trade, whether there is a national security argument for agricultural protectionism, TPP, how new trade agreements should be structured, the trade bureaucracy in D.C., whether free trade still brings peace, Smoot-Hawley, the American Revolution (we are spoiled brats), Dunkirk, why New Hampshire is so wealthy, Brexit, Alexander Hamilton, NAFTA, the global trade slowdown, premature deindustrialization, and the history of the Chicago School of Economics, among other topics.  Here is one excerpt:

COWEN: Here goes. The claim that 19th century American growth was driven by high tariffs. What’s your take?

IRWIN: Not really true. If you look at why the US economy performed very well, particularly relative to Britain or Germany or other countries, Steve Broadberry’s shown that a lot of the overtaking of Britain in terms of per capita income was in terms of the service sector.

The service sector was expanding rapidly. It had very high productivity growth rates. We usually don’t think as that being affected by the tariff per se. That’s one reason.

We had also very high productivity growth rates in agriculture. I’ve done some counterfactual simulations. If you remove the tariff, how much resources would we take out of manufacturing and put into services or agriculture is actually pretty small. It just doesn’t account for the success we had during this period.

COWEN: Is there any country where you would say, “Their late 19th century economic growth was driven by tariffs?” Argentina, Canada, Germany, anything, anywhere?

IRWIN: No. If you look at all those, once again, in late 19th century, they were major exporters, largely of commodities, but they did very well that way. You know that Argentina was one of the richest countries in the world in the late 19th century. It really wasn’t until they adopted more import substitution policies after World War I that they began to fall behind.

Definitely recommended, and here is Doug’s Wikipedia page.

Some people think so, in the associated video clip Joe Stiglitz says Bitcoin should be banned.  Here is some FT skepticism from Jean Tirole.

I used to think Bitcoin was a bubble, but I no longer hold this view.  If nothing else, put all the more complicated factors aside and think of Bitcoin as competing for some of the asset space held by gold and also to some extent art.  Gold, too, in its hedging functions is a “bubble,” though not a bubble.  It is hard to ship, but has some extra value because it is perceived as a focal asset and one that does not covary positively in a simple way with the market portfolio.  The same is true of Bitcoin, yet that kind of focality-based “bubbliness” can persist for centuries.  Note by the way that gold has become less of a hedge, partly because inflation has been low and partly because China and India dominate the gold market more than a few decades ago.  So new and better hedges are needed.  And what a backstory Bitcoin has, making it a strong competitor in this regard.

I am not saying that is the Bitcoin story, it is simply a Bitcoin story, a minimalist account that can appeal to skeptics.  And you can buy this story and still think the current price is either too high or too low.

This estimate claims there is $241 trillion of wealth in the world, make of that what you will (there is something nonsensical about such aggregate measures because they are not traded against anything).  If you imagine people wish to hold one quarter of one percent of that in crypto form, that gets you to about $600 billion in value.  Currently crypto assets (on good days) hover near $300 billion in market capitalization.  Is that so crazy?  I genuinely don’t know, but that is one way of thinking about market cap in this sector.

I will continue to watch with interest.

1 – My favorite unpopular blockchain ideas: 99% of corporate experiments regarding blockchains are better handled with Apache Kafka and multiple archivers. Anything that attempts to be a fast, global ledger has to accept the reality that global ordering is a limitation, not a feature, and instead use logical clocks. The intersection between blockchain enthusiast and distributed system researchers is close to zero. When we look back 100 years, Bitcoin itself will be seen as far more relevant in retrospect than blockchain technologies.

That is from MR reader Bob.

That is the new and excellent history by Leslie Berlin, substantive throughout, here is one good bit of many:

In March 1967, Robert and Taylor, jointly leading a meeting of ARPA’s principal investigators in Ann Arbor, Michigan, told the researchers that ARPA was going to build a computer network and they were all expected to connect to it.  The principle investigators were not enthusiastic.  They were busy running their labs and doing their own work.  They saw no real reason to add this network to their responsibilities.  Researchers with more powerful computers worried that those with less computing power would use the network to commandeer precious computing cycles.  “If I could not get some ARPA-funded participants involved in a commitment to a purpose higher than “Who is going to steal the next ten percent of my memory cycles?”, there would be no network,” Taylor later wrote.  Roberts agreed: “They wanted to buy their own machines and hide in the corner.”

You can buy the book here, here is one good review from Wired, excerpt:

While piecing together a timeline of the Valley’s early history—picture end-to-end sheets of paper covered in black dots—Berlin was amazed to discover a period of rapid-fire innovation between 1969 and 1976 that included the first Arpanet transmission; the birth of videogames; and the launch of Apple, Atari, Genentech, and major venture firms such as Kleiner Perkins and Sequoia Capital. “I just thought, ‘What the heck was going on in those years?’ ” she says.

Here is praise from Patrick Collison on Twitter.

Nicholas Kozeniauskas, a job candidate from NYU, has a job market paper on that topic:

Recent research shows that entrepreneurial activity has been declining in the US in recent decades. Given the role of entrepreneurship in theories of growth, job creation and economic mobility this has generated considerable concern. This paper investigates why entrepreneurship has declined. It documents that (1) the decline in entrepreneurship has been more pronounced for higher education levels, implying that at least part of the force driving the changes is not skill-neutral, and (2) the size distribution of entrepreneur businesses has been quite stable. Together with a decline in the entrepreneurship rate the second fact implies a shift of economic activity towards non-entrepreneur firms. Guided by this evidence I evaluate explanations for the decline in entrepreneurship based on skill-biased technical change, changes in regulations increasing the fixed costs of businesses and changes in technology that have benefited large non-entrepreneur firms. I do this using a general equilibrium model of occupational choice calibrated with a rich set of moments on occupations, income distributions and firm size distributions. I find that an increase in fixed costs explains most of the decline in the aggregate entrepreneurship rate and that skill-biased technical change can fully account for the larger decrease in entrepreneurship for more educated people when combined with the other forces.

This is one of the more important papers of this job market season.

That is the topic of my latest Bloomberg column.  Here is one excerpt:

An additional reason for skepticism stems from the nature of crypto assets. The word “cryptocurrency” is far more common than “crypto asset,” but it’s a misleading term. Bitcoin, for instance, is used only rarely in retail transactions, and for all its success it isn’t becoming more important as a medium of exchange. Bitcoin thus isn’t much of a currency in the literal sense of that term. There is a version of bitcoin, Bitcoin Cash, that changed the initial rules to be better suited as an exchange medium, but it isn’t nearly as popular.

If you think of these assets as “cryptocurrencies,” central bank involvement will seem natural, because of course central banks do manage currencies. Instead, this new class of assets is better conceptualized as ledger systems, designed to create agreement about some states of the world without the final judgment of a centralized authority, which use a crypto asset to pay participants for maintaining the flow and accuracy of information. Arguably these innovations come closer to being substitutes for corporations and legal systems than for currencies.

Put in those terms, an active (rather than merely supervisory) role for central banks in crypto assets is suddenly far from obvious. Consider other financial innovations: Does anyone suggest that central banks should run their own versions of ETFs or high-frequency trading? Is there a need for central banks to start managing the development of accounting and governance systems?

Central banks are too conservative anyway, which of course is how they should be.  Don’t forget:

…consider a simple question: Would any central bank have had the inspiration or taken the risk of initiating the bitcoin protocol in the first place?

Keep in mind, I’ve favored net neutrality for most of my history as a blogger.  You really could change my mind back to that stance.  Here is what you should do:

1. Cite event study analysis showing changes in net neutrality will have significant and possibly significantly negative effects.

2. Discuss models of natural monopoly, and how those market structures may or may not distort product choice under a variety of institutional settings.

3. Start with a framework or analysis such as that of Joshua Gans and Michael Katz, and improve upon it or otherwise modify it.  Here is their abstract:

We correct and extend the results of Gans (2015) regarding the effects of net neutrality regulation on equilibrium outcomes in settings where a content provider sells its services to consumers for a fee. We examine both pricing and investment effects. We extend the earlier paper’s result that weak forms of net neutrality are ineffective and also show that even a strong form of net neutrality may be ineffective. In addition, we demonstrate that, when strong net neutrality does affect the equilibrium outcome, it may harm efficiency by distorting both ISP and content provider investment and service-quality choices.

Tell me, using something like their framework, why you think the relative preponderance of costs and benefits lies in one direction rather than another.

Consider Litan and Singer from the Progressive Policy Institute, they favor case-by-case adjudication, tell me why they are wrong.

Or read this piece by Nobel Laureate Vernon Smith, regulatory experts Bob Crandall, Alfred Kahn, and Bob Hahn, numerous internet experts, etc.:

In the authors’ shared opinion, the economic evidence does not support the regulations proposed in the Commission’s Notice of Proposed Rulemaking Regarding Preserving the Open Internet and Broadband Industry Practices (the “NPRM”). To the contrary, the economic evidence provides no support for the existence of market failure sufficient to warrant ex ante regulation of the type proposed by the Commission, and strongly suggests that the regulations, if adopted, would reduce consumer welfare in both the short and long run. To the extent the types of conduct addressed in the NPRM may, in isolated circumstances, have the potential to harm competition or consumers, the Commission and other regulatory bodies have the ability to deter or prohibit such conduct on a case-by-case basis, through the application of existing doctrines and procedures.

4. Consider and evaluate other forms of empirical evidence, preferably not just the anecdotal.

5. Don’t let emotionally laden words do the work of the argument for you.

6. Offer a rational, non-emotive discussion of why pre-2015 was such a bad starting point for the future, and why so few users seemed to mind or notice as the regulations switched several times.

7. Don’t let politics make you afraid to use your best argument, namely that anti-NN types typically develop more faith in an assortment of government regulators in this setting than they might express in a number of other contexts.  That said, don’t just use this point to attack them, live with and consistently apply whatever judgment of the regulators you decide is appropriate.

If you are wondering why I have changed my mind, it is a mix of new evidence coming in, experience over the 2014-present period, relative assessment of the arguments on each side moving against NN proponets, and the natural logic of the embedded trade-offs, whereby net neutrality typically works in a short enough short run but over enough time more pricing is needed.  Of course it is a judgment call as to when the extra pricing should kick in.

Here is what will make your arguments less persuasive to me:

1. Respond to discussions of other natural monopoly sectors and their properties by saying “the internet isn’t like that, you don’t understand the internet.”  If someone uses the water sector to make a general point about tying and natural monopoly, commit internet error #7 by responding: “the internet isn’t like water!  You don’t understand the internet!”

2. Lodge moral complaints against the cable companies or against commercial incentives more generally, or complain about the “ideology” of others.  Mention the word “Trump” or criticize the Trump administration for its failings.  Call the recent decision “anti-democratic.”

3. Cite nightmare or dystopian scenarios that are clearly illegal under other current laws and regulations.  Cite dystopian scenarios that would contradict profit-maximizing behavior on the part of the involved companies.  Assume that no future evolution of regulation could solve or address any of the problems that might arise from the recent switch.  Mention Portugal as a scare scenario, without explaining that full internet packages still are for sale there, albeit without the discounts for the partial packages.

Are you up to the challenge?

If I read say this Tim Wu Op-Ed, I think it is underwhelming, even given its newspaper setting, and the last two paragraphs are content-less, poorly done emotive manipulation.  Senpai 3:16 is himself too polemic and exaggerated, but he does make some good points against this piece, see his Twitter stream.

Net neutrality defenders, as of now you have lost this battle.  I’d like to hear more.