Popular Science: A pilot A.I. developed by a doctoral graduate from the University of Cincinnati has shown that it can not only beat other A.I.s, but also a professional fighter pilot with decades of experience. In a series of flight combat simulations, the A.I. successfully evaded retired U.S. Air Force Colonel Gene “Geno” Lee, and shot him down every time. In a statement, Lee called it “the most aggressive, responsive, dynamic and credible A.I. I’ve seen to date.”

What’s the most important part of this paragraph? The fact that an AI downed a professional fighter pilot? Or the fact that the AI was developed by a graduate student?

In the research paper the article is based on the authors note:

…given an average human visual reaction time of 0.15 to 0.30 seconds, and an even longer time to think of optimal plans and coordinate them with friendly forces, there is a huge window of improvement that an Artificial Intelligence (AI) can capitalize upon.

The AI was running on a $35 Raspberry Pi.

AI pilots can plan and react far quicker than human pilots but that is only half the story. Once we have AI pilots, the entire plane can be redesigned. We can build planes today that are much faster and more powerful than anything that exists now but the pilots can’t take the G-forces even with g-suits, AIs can. Moreover, AI driven planes don’t need ejector seats, life-support, canopies or as much space as humans.

The military won’t hesitate to deploy these systems for battlefield dominance so now seems like a good time to recommend Concrete Problems in AI Safety, a very important paper written by some of the world’s leading researchers in artificial intelligence. The paper examines practical ways to design AI systems so they don’t run off the rails. In the Terminator movie, for example, Skynet goes wrong because it concludes that the best way to fulfill its function to safeguard the world is to eliminate all humans–this is an extreme example of one type of problem, reward hacking.

Imagine that an agent discovers a buffer overflow in its reward function: it may then use this to get extremely high reward in an unintended way. From the agent’s point of view, this is not a bug, but simply how the environment works, and is thus a valid strategy like any other for achieving reward. For example, if our cleaning robot is set up to earn reward for not seeing any messes, it might simply close its eyes rather than ever cleaning anything up. Or if the robot is rewarded for cleaning messes, it may intentionally create work so it can earn more reward. More broadly, formal rewards or objective functions are an attempt to capture the designer’s informal intent, and sometimes these objective functions, or their implementation, can be “gamed” by solutions that are valid in some literal sense but don’t meet the designer’s intent. Pursuit of these “reward hacks” can lead to coherent but unanticipated behavior, and has the potential for harmful impacts in real-world systems. For example, it has been shown that genetic algorithms can often output unexpected but formally correct solutions to problems [155, 22], such as a circuit tasked to keep time which instead developed into a radio that picked up the regular RF emissions of a nearby PC.

Concrete Problems in AI Safety asks what kind of general solutions might exist to prevent or ameliorate reward hacking when we can never know all the variables that might be hacked? (The paper looks at many other issues as well.)

Competitive pressures on the battlefield and in the market mean that AI adoption will be rapid and AIs will be placed in greater and greater positions of responsibility. Firms and governments, however, have an incentive to write piecemeal solutions to AI control for each new domain but that is unlikely to be optimal. We need general solutions so that every AI benefits from the best thinking across a wide range of domains. Incentive design is hard enough when applied to humans. It will take a significant research effort combining ideas from computer science, mathematics and economics to design the right kind of incentive and learning structures for super-human AIs.

Soon I will be recording a podcast-only, no live attendance, no live video Conversations with Tyler with Michael Orthofer.  Michael runs the site Literary Saloon and is perhaps the world’s most productive book reviewer and book review blogger, with a focus on foreign fiction translated into English.  Michael is a deeply devoted infovore, and I expect this to be one of the most interesting conversations in the series.

Here is my short review of Michael’s big book on world literature: “If you measure book quality by the actual marginal product of the text, this is one of the best books written, ever.  Reading the manuscript in draft form induced me to a) write an enthusiastic blurb, and b) order about forty items through Amazon, mostly used of course.  The book is basically a comprehensive guide to what is valuable and interesting in recently translated world literature, a meta-book so to speak, with extensive coverage of most of the countries you might want.”

Here is the New Yorker profile of Orthofer:

“I can’t imagine not doing it,” Orthofer told me. “A day in which I don’t read or write, I have trouble falling asleep.” His goal is to read a book a day, though he confesses that this is “unrealistic.” He works on weekends, too, and has written four novels that are in the drawer. His main interests, according to the site, are inline roller-skating in Central Park and building snow sculptures, some of which are big enough that he carves staircases inside them to get to the top. When he tires of working, he steps out to a library or bookstore, “to see, be around books.” Last year, and this year, he worked through Christmas.

OK, so what should I ask Michael?  Comments are open.

Addendum: Here are previous installments of Conversations with Tyler.

It can be incredibly frustrating when a virtual assistant repeatedly misunderstands what you’re saying. Soon, though, some of them might at least be able to hear the irritation in your voice, and offer an apology.

Amazon is working on significant updates to Alexa, the virtual helper that lives inside the company’s voice-controlled home appliance, called Amazon Echo. These will include better language skills and perhaps the ability to recognize the emotional tenor of your voice.

Researchers have long predicted that emotional cues could make machine interfaces much smarter, but so far such technology has not been incorporated into any consumer technology.

Rosalind Picard, a professor at MIT’s Media Lab, says adding emotion sensing to personal electronics could improve them: “Yes, definitely, this is spot on.” In a 1997 book, Affective Computing, Picard first mentioned the idea of changing the voice of a virtual helper in response to a user’s emotional state. She notes that research has shown how matching a computer’s voice to that of a person can make communication more efficient and effective. “There are lots of ways it could help,” she says.

The software needed to detect the emotional state in a person’s voice exists already. For some time, telephone support companies have used such technology to detect when a customer is becoming irritated while dealing with an automated system. In recent years, new machine-learning techniques have improved the state of the art, making it possible to detect more emotional states with greater accuracy, although the approach is far from perfect.

Here is the full story.  Here is my recent New Yorker piece on how talking bots will affect us.

The robot administers a small pin prick at random to certain people of its choosing.

The tiny injury pierces the flesh and draws blood.

Mr Reben has nicknamed it ‘The First Law’ after a set of rules devised by sci-fi author Isaac Asimov.

He created it to generate discussion around our fear of man made machines. He says his latest device shows we need to prepare for the worst

‘Obviously, a needle is a minimum amount of injury, however – now that this class of robot exists, it will have to be confronted,’ Mr Reben said on his website.

Here is more, with pictures of (slightly) injured humans, via the excellent Mark Thorson.

A reader has been asking me this question, and my answer is…no!

Don’t get me wrong, I still think it is a stimulating and wonderful book.  And if you don’t believe me, here is The Wall Street Journal:

Mr. Hanson’s book is comprehensive and not put-downable.

But it is best not read as a predictive text, much as Robin might disagree with that assessment.  Why not?  I have three main reasons, all of which are a sort of punting, nonetheless on topics outside one’s areas of expertise deference is very often the correct response.  Here goes:

1. I know a few people who have expertise in neuroscience, and they have never mentioned to me that things might turn out this way (brain scans uploaded into computers to create actual beings and furthermore as the dominant form of civilization).  Maybe they’re just holding back, but I don’t think so.  The neuroscience profession as a whole seems to be unconvinced and for the most part not even pondering this scenario.

2. The people who predict “the age of Em” claim expertise in a variety of fields surrounding neuroscience, including computer science and physics, and thus they might believe they are broader and thus superior experts.  But in general claiming expertise in “more” fields is not correlated with finding the truth, unless you can convince people in the connected specialized fields you are writing about.  I don’t see this happening, nor do I believe that neuroscience is somehow hopelessly corrupt or politicized.  What I do see the “Em partisans” sharing is an early love of science fiction, a very valuable enterprise I might add.

3. Robin seems to think the age of Em could come about reasonably soon (sorry, I am in Geneva and don’t have the book with me for an exact quotation).  Yet I don’t see any sign of such a radical transformation in market prices.  Even with positive discounting, I would expect backwards induction to mean that an eventual “Em scenario” would affect lots of prices now.  There are for instance a variety of 100-year bonds, but Em scenarios do not seem to be a factor in their pricing.

Robin himself believes that market prices are the best arbiter of truth.  But which market prices today show a realistic probability for an “Age of Em”?  Are there pending price bubbles in Em-producing firms, or energy companies, just as internet grocery delivery was the object of lots of speculation in 1999-2000?  I don’t see it.

The one market price that has changed is the “shadow value of Robin Hanson,” because he has finished and published a very good and very successful book.  And that pleases me greatly, no matter which version of Robin is hanging around fifty years hence.

Addendum: Robin Hanson responds.  I enjoyed this line: “Tyler has spent too much time around media pundits if he thinks he should be hearing a buzz about anything big that might happen in the next few centuries!”

Looking for something to do this weekend in New York? Story, a concept shop that completely changes theme every few months, has relaunched this week as a Mr. Robot-themed space. In addition to being a retail shop selling an assortment of gadgets, accessories, and Mr. Robot-themed wares, there’s an “Evil Corp” ATM at the front of the store that will dispense real money (up to $50) if you figure out the four-digit code. The clues are hidden around the store, and we’re told they’ll probably change often.

Here is the full story, with many photos and an address.  To think that they closed Tower Records and Borders for this…sigh.

I found this discussion of interest:

I believe I’ve sketched out an idea that enables all transfers to verify the recipient is not a prohibited person without communicating any distinguishing information to the government while optionally leaving an audit trail useful for prosecution.

  • When an individual applies for state-issued identification, let them choose a public-private key pair. Make the public key part of their identification card, and the private key remain private. Not even the issuing state would know it. Add their public key to a whitelist.

  • At the same time, perform a background check to determine if the individual is a prohibited person. If so, add their public key to a blacklist.

  • Publish the blacklist in bulk and make updates available daily. We all have access to the Internet. We can do this. Regularly update the blacklist according to adjudicating events associated with the definition of “prohibited person.” For example, at the time of conviction of a felony, and individual’s public key gets added to the blacklist.

  • Firearms sellers, private and commercial, must maintain a copy of the blacklist up-to-date at the time of sale. Perhaps use a blockchain of sorts and/or share via BitTorrent or some other distributed service.

  • At the time of sale, the recipient provides their public key to the seller. Seller verifies the public key is not on the blacklist. The seller constructs a secret cryptographic nonce and encrypts it with the recipient’s public key. Recipient decrypts with their private key and returns the nonce in plaintext to the seller to confirm their public-private key pair is valid. This form of handshaking is common place and may be automated.

  • If the recipient is on the whitelist and not on the blacklist, the transfer may proceed.

  • Optionally, the seller may record a log of the recipient’s public key, perhaps encrypted with their private key. On the event of a warranted search, the seller may choose to decrypt their log entry to reveal the identify of the recipient.This leaves the audit trail which we may be legislatively require.

Thus, no direct communication with the state on the event of a transfer is needed which prevents the creation of a registry. Prohibited persons cannot acquire firearms without unlawful behavior from the seller which satisfies the aims of universal background checks. We already have all the cryptographic primitives and communications infrastructure needed to implement this and verify its integrity.


That is from kermudgeon on Reddit, via N.  Some of the comments are quite good as well.

An all-cash deal where the combined entity is trading down on the news is basically the exact opposite of bubble behavior.

That is from Conor SenGrodaeu remarked:

Microsoft had to move quickly before the ECB bought it

Increasingly, says Professor Crystal, whose books include Making a Point: The Persnickety Story of English Punctuation,” the period is being deployed as a weapon to show irony, syntactic snark, insincerity, even aggression

If the love of your life just canceled the candlelit, six-course, home-cooked dinner you have prepared, you are best advised to include a period when you respond “Fine.” to show annoyance

“Fine” or “Fine!,” in contrast, could denote acquiescence or blithe acceptance

“The period now has an emotional charge and has become an emoticon of sorts,” Professor Crystal said

And this:

Researchers at Binghamton University in New York and Rutgers University in New Jersey have also recently noted the period’s new semantic force

They asked 126 undergraduate students to review 16 exchanges, some in text messages, some in handwritten notes, that had one-word affirmative responses (Okay, Sure, Yeah, Yup) Some had periods, while others did not

Those text message with periods were rated as less sincere, the study found, whereas it made no difference in the notes penned by hand

Here is the full Dan Bilefsky story (NYT).

“We’re living through an historic glut of stolen data,” explains Brian Krebs, who writes the blog Krebs on Security. “More supply drives the price way down, and there’s so much data for sale, we’re sort of having a shortage of buyers at this point.”

…But cybercriminals’ most crucial adaptation in recent years has little to do with their technical tools and everything to do with their business model: They have started selling stolen data back to its original owners. To keep cybercrime profitable, criminals needed to find a new cohort of potential buyers, and they did: all of us. At the heart of this new business model for cybercrime is the fact that individuals and businesses, not retailers and banks, are the ones footing the bill for data breaches.

Here is the full Josephine Wolff piece.

From Geoff Kaufman and Mary Flanagan:

The present research investigated whether digital and non-digital platforms activate differing default levels of cognitive construal. Two initial randomized experiments revealed that individuals who completed the same information processing task on a digital mobile device (a tablet or laptop computer) versus a non-digital platform (a physical print-out) exhibited a lower level of construal, one prioritizing immediate, concrete details over abstract, decontextualized interpretations. This pattern emerged both in digital platform participants’ greater preference for concrete versus abstract descriptions of behaviors as well as superior performance on detail-focused items (and inferior performance on inference-focused items) on a reading comprehension assessment. A pair of final studies found that the likelihood of correctly solving a problem-solving task requiring higher-level “gist” processing was: (1) higher for participants who processed the information for task on a non-digital versus digital platform and (2) heightened for digital platform participants who had first completed an activity activating an abstract mindset, compared to (equivalent) performance levels exhibited by participants who had either completed no prior activity or completed an activity activating a concrete mindset.

Here is also the press release, and for the pointer I thank Charles Klingman.

I have a short New Yorker piece on that question, here is one bit from it:

If Siri is sometimes sarcastic, could heavy users of the Siri of the future become a little more sarcastic, too?

For companies, there are risks associated with such widespread personification. For a time, consumers may be lulled by conversational products into increased intimacy and loyalty. But, later, they may feel especially betrayed by products they’ve come to think of as friends. Like politicians, who build up trust by acting like members of the family only to incur wrath when they are revealed to be careerist and self-interested, companies may find themselves on an emotional roller coaster. They’ll also have to deal with complicated subjects like politics. Recently, Tay, a chat bot from Microsoft, had to be disabled because it began issuing tweets with Nazi-like rhetoric. According to Elizabeth Dwoskin, in the Post, Cortana, another talking Microsoft bot, was carefully programmed not to express favoritism for either Hillary Clinton or Donald Trump. A product’s apparent intelligence makes it likable, but also offers more of an opportunity to offend.

Here is another:

And there are ways in which just knowing that bots exist will change us. If the bots are good enough, we won’t be able to distinguish them from actual people over e-mail or text; when you get an e-mail, you won’t necessarily be certain it’s from a human being. When your best friend writes that she’s also “looking forward to seeing you at the baseball game tonight,” you’ll smile—then wonder if she’s busy and has asked her e-mail bot to send appropriate replies. Once everyone realizes that there might not be a person on the other end, peremptory behavior online may become more common. We’ll likely learn to treat bots more like people. But, in the process, we may end up treating people more like bots.

Do read the whole thing.

Here is an excerpt from a longer post, which also includes a summary:

Here are some of the most interesting ideas in the book:

1. Mind speeds: I had not previously spent much time thinking about how our brain’s hardware affects the speed at which we think. As it happens, our minds are spectacularly slow compared to what’s feasible with other materials! Better hardware, as well inequalities of hardware across individuals, will likely drive many parts of em society.

2. Death in the time of copies: An individual’s relationship to death is much different when you can make and store copies of yourself. Given how much of our current lives and societies are wrapped in who dies / how they die / when we die – a world where death is less central has major implications for identity, values, and relationships.

3. Security concerns are paramount: Theft (making copies of you without your permission) thus becomes almost more of an issue than death. As such, laws and cultural taboos will shift with security becoming more central to em value systems.

4. Less democratic: In a short period of a time, a well run non-democratic regime can outperform your average democracy. However, in the modern human world, these regimes often implode on themselves before they can dominate the rest of the world. But in the em world, things will move so fast (economic doubling rates are incredibly fast, every month or two!), that the rewards to short bursts of effective non-democratic regimes may be very high.

5. Religion: I tend not to think of robots as religious, but Robin makes the case that the utility of religion (nicer hard-working people) and the values of the em world (more farmer like) should lead to increased religiosity.

6. Increased utility: The sheer number of ems, coupled with their high mind speeds – as well as the likelihood that there lives will be ok in terms of meaning and happiness – suggests that the transition to an em world will be a positive utility move.

You can order the book here.  Here is my earlier review.

Which search terms correlate with support for which politicians?  Why not at least ask this question?

John Kasich. Places that like Kasich are richer in some fairly policy-wonkish search terms: “net cost,” “renewable portfolio standard,” the economist Joseph Stiglitz, Financial Times writer Martin Wolf, and Vox writer Dylan Matthews. These terms have a ring of plausibility. They might be good fodder for small talk…if you are talking with a Kasich supporter!

But then there are terms that I don’t entirely understand: Route 73 and Haven Pizza. Maybe someone can explain those to me. It is also true that with billions of search terms to choose from, occasionally a correlation will arise by chance. These might be false positives.

Ted Cruz. Many Cruz-related search terms are related to domestic life of a certain kind: family photos, felt Christmas stockings, scentsy plug ins, balloon animals, Baby Trend car seats, and DIY cribs. Easy enchiladas are particularly Cruz-y. Mmmm, enchiladas. And udder covers…I wasn’t expecting that one. Maybe the Cruz campaign could start distributing Cruz-themed udder covers!

Donald Trump. Note that the correlations are weaker. That could be because Trump support is broad-based in the Republican Party. Or it could be that the connection between the voter and the Google-searcher is indirect (i.e. they are different individuals who live near one another).

That is from Sam Wang, via the keen-eyed Jordan Schneider.  And what about the Democrats?

Near Clinton supporters it’s cheap bedroom furniture, Nicki Minaj fans, and pink hoverboard shoppers. And “career in” – Google auto-complete as a job counselor!

And the strongest correlate with Bernie Sanders support?: “candied nuts,” next in line is “best oatmeal,” ladies and gentlemen that is proof this is not just data mining and false correlations.  The list is dominated by recipe terms, and “corn syrup substitute” is number four!  Oh where oh where is Martin Wolf?

I considered the question of when one should just stop reading, and here is Robert’s take:

I read full-time to edit The Browser, and I abandon a hundred articles for every one that I finish. I generally stop if I hit “eponymous”, or “toxic”, or “trigger warning”, or “make no mistake”. Summary labelling of anything in an article as “complex” means that the writer does not understand or cannot explain the material. I don’t often read beyond headlines that use the words “surprising”, “secret”, “really”, “not” or “… and why it matters”. Any headline ending in a question mark is a bad sign. I know writers don’t usually write their own headlines, but the headline represents a best effort to say what is useful in the article by a sympathetic person who has been paid to read it.

Robert is one of the best readers I know.