Defer to the Algorithm

A BuzzFeed article predicts that Twitter will soon move from a time-ordered feed to an algorithmic feed, one that shows you tweets that it predicts you will like before it show you lesser-ranked tweets. Naturally, twitter exploded with outrage that this is the end of twitter.

My own tweet expresses my view ala Marc Andreessen style:

It is peculiar that people are more willing trust their physical lives to an algorithm than their twitter feed. Is the outrage real, however, or will people soon take the algorithm for granted? How many people complaining about algorithmic twitter don’t use junk-email filters? I want ALL my emails! Only I can decide what is junk! Did junk email filters ruin email or make it better?

Facebook moved to an algorithm years ago. At the time, the move caused complaints but I think algorithmic feed has made Facebook more relevant, especially in recent years when the algorithm has gotten quite good. The profits agree with my assessment. Many people don’t understand that there is no serious alternative to an algorithmic feed because most people’s uncurated feeds contain well over a thousand posts every day. It’s curate or throw material out at random.

Think of the algorithm as an administrative assistant that sorts your letters, sending bills to your accountant, throwing out junk mail, and keeping personal letters for your perusal. The assistant also reads half a dozen newspapers before you wake to find the articles he thinks that you will most want to read that morning. Who wouldn’t want such an assistant? Moreover, Facebook has billions of dollars riding on the quality of its assistant algorithms and it invests commensurate resources in making its algorithm more and more attuned to our wants and needs.

It’s not simply that the algorithms are good and getting better it’s that the highest productivity people will use their human intelligence to complement machine intelligence. That means trusting the machine to curate millions of items, bringing only the most important to your attention, and then using human intelligence to take action on the most important items. By trusting the machine intelligence to filter, you can open yourself up to a much wider space of information. I have many more friends on Facebook than I have IRL because I trust the algorithm to bring me only the best of my friends on any given day. A twitter algorithm will mean that I can follow more people without being overwhelmed. Even when the filter is imperfect, you are more likely to discover something of importance from 100,000 items imperfectly filtered to 100 than from 1000 items perfectly filtered to 100.

As Tyler argued in Average is Over, the future belongs to people who can defer to the algorithm.


"It’s not simply that the algorithms are good and getting better."

I have a phd in CS, and have been working on the kind of large-scale, distributed data processing software systems (the kind that would drive the twitter feed, for example), for more than a decade now. I don't recognise the word "Algorithm" in that sentence.

Algorithms are getting better? That makes very little sense. What you describe as a facebook algorithm is literally thousands or tens of thousands of algorithms doing a massive number of things. Do you mean software systems are getting better? Is "algorithm" just a proxy word for "software"?


Why not use that instead of repurposing a word that has a well known meaning? It sounds ridiculous to me.

If inwrote about microeconomics and used "money" as a proxy for prices indices and unemployment etc you'd think i sound ridiculous. This sounds like that.

Perhaps "algorithm" has meaning recursively, and the stack can still be referred to as a thing.

I won't belabour this, but imagine if I wrote "the fed uses the Money to keep people in jobs, and thus it should be fine to let the treasure or private companies use the money to help manage their risk".

Or replace money with " economy ". Either way, its pretty obvious the person writing doesn't actually know much about the topic.

Wrapping up self-driving cars and twitter feeds, etc as just " algorithms" and then saying "algorithms are getting better" like that is somehow interesting is the same thing. It becomes very clear to me that Alex knows little about software systems and really doesn't (cant?) have anything interesting to say about them.

Even the first sentence "time ordering" is a type of algorithm

Or replace money with ” economy “. Either way, its pretty obvious the person writing doesn’t actually know much about the topic.

Money isn't economy. However... isn't a stack of algorithms algorithm? I mean ... I do a lot of programming and it seems to me that what Alex wrote is ultimately a correct use of the word. I would have used it too...

Thats my point: money isn't an economy, and Facebook feed ordering isn't an algorithm or even a stack of algorithms. There are a/b tests, load balancing, network conditions, etc. An algorithm us a series of steps in a calculation or computation, and not a huge complex software system.

I'm also a computer science PhD - including having the development of a major networking protocol as my PhD thesis under my belt (YPP if you've heard of it). I have to say Ian Smith is being too prickily here, the usage was largely correct for a layman like Prof. Tabarrok. Not sure why you're making such a big bruh-haha about it.

Never heard of YPP, but again, I work in Distributed Systems. I am, honestly, trying to figure out what the hell he means by "Algorithm". I guess it means software, but not all software. Sorting a twitter feed by time is somehow not "algorithmic" (clearly contrasted in the first sentence), but whatever Twitter will move to is "algorithmic". I get there's a distinction; I'm not sure what it is.

The closest I can gather is that an "algorithm" in this lay sense is a sort of automated, time-adaptive machine intelligence. That's pretty far from the definition you (or I, or anyone else) learned in maths or CS.

It was for interfacing between an end user and the feeling of being on a mountain top via a chocolate coated mint pattie. But anyway not your field I guess.

"Algorithm" may not be the best word to use in the post, but it's not wrong. A couple definitions of "algorithm" that I see in (non-CS) dictionaries:

"a step-by-step procedure for solving a problem or accomplishing some end especially by a computer"
"A precise step-by-step plan for a computational procedure that possibly begins with an input value and yields an output value in a finite number of steps."

I realize that "algorithm" is usually used to describe a procedure to exactly solve a well defined problem, like sorting a list of numbers by value, but by analogy, it doesn't seem too crazy to say that "a method to sort posts by estimated importance" is an algorithm. Likewise, path finding algorithms exist for graphs (like a breath-first search) and simple systems that can be represented by graphs (like mazes), so by analogy the path finding method used by a self driving car is an algorithm. So Alex's algorithms do have analogies with the algorithms we're familiar with.

I think that Alex's distinction between "software" and "algorithm" is the degree of human interaction. Microsoft Excel is software because you manually enter the content and you get the output. Post ranking would be an algorithm because you don't input the content (besides selecting the source of the input -- i.e. the people you follow) but you still get the output. Similarly, a self-driving car would be controlled by an algorithm since, besides choosing the destination, the car reacts to its environment, which is outside of your control.

Is this the best use of the word "algorithm"? I don't thinks so, but it does look self-consistent to me, which is something.

Thats my point: money isn’t an economy, and Facebook feed ordering isn’t an algorithm or even a stack of algorithms. There are a/b tests, load balancing, network conditions -

Huh? Are you sure you know what you are talking about? Seems to me PhD in CS doesn't mean anything these days...

Actually a better analogy would be the word "economy". Much like "algorithm" and "software system", people use the word "economy" when they should use "catallactic system" or "catallaxy".

Sire! The pedants are revolting!

This seems like a whole lot of babble. I do machine learning and programming for a living and the post's word choice re. 'algorithm' was quite unremarkable. There _are_ better algorithms for recommendation, and they have a hell of a lot more data to train over.

It's not babble. The first sentence contrasts a "time-ordered" feed with an "algorithmic feed". Sorting things by time is an algorithm. Twitter already has an algorithmic feed. The way " algorithm" is used here is inconsistent and incorrect.

Please excuse my autistic friend, he does this from time to time.

I think the idea being transmitted here is that of 'mechanical' or 'autonomous' function (Church-Turing) not to the specific class of "effectively computable" functions apart from those that are well-defined but not 'algorithmic' e.g. Chaitin's constant

Ian, I agree with you. This sums up the sloppy thinking and defensiveness when called on it that drives this place (both among commenters and main posters) Beyond that, AT's point is trivial. 'Here are some positive examples of X, therefore all implementations of X are positive.'
It appears average is not yet over.

You tell a robot car where to go and it uses AI to get there.

Twitter will tell you where to go.

One is a tool. In the other, you become the tool.

A more important note might be that Twitter's algorithm is crap. When I went (testing yesterday) to the lightning icon, "news" tab, the top story was "a seal walks into a restaurant."

This tells me that they have a very superficial approach, looking at total links to news sites, with no good categorization on what is actually news, let alone what matches my interests.

They need less hoopla more AI

Well technically it's a recommendation system algorithm that's making the decisions. It being distributed doesn't change the underlying idea behind "the algorithm." Or is there something I am missing?

Allowing an algorithm to control your news feed is more like letting a car decide where to take you, not how to get there

I thought the exact same thing. Maybe people don't want to filter posts according to predicted liking. It seems like the kind of thing that could cause confirmation bias, similar to reasons that people object to Google causing search bubbles.

Yes. Some of us actually WANT to see post out of the usual.

How do you think we'll monetize the self-driving car? You ask us to take you to McDonald's, and we take you to Burger King because they paid us to do it. Think that won't happen?

Controlling and monetizing the user experience is the whole point of these devices and apps.

But that's actually not a bad thing because its probably more likely the advertisers, or companies that pay for this sort of thing actually know what the better option is for you. This way people will make decisions made more efficiently by having them made for them. Defer to the algorithms.

"Allowing an algorithm to control your news feed is more like letting a car decide where to take you, not how to get there." This gets at my concern. Facebook decides what I need/want to see. I suspect this is determined not by any consideration of what I want, but by consideration of how best Facebook can monetize my interaction with the site. If FB wanted to produce a news feed tailored to me, this would be very simple. All they would have to do is ask me. They don't.

I am reading Ted Cruz's book for this reason. All of my "feeds" say he is hated by everyone in DC because [insults of different kinds]. So I personally broke from those feeds to see what it's all about.

Exactly. I want machines to help me do. I do not want machines to help me think.

My analogy would be having a conversation among friends. You can either go to the bar and sit through the whole talk with the others, or later ask somebody for the highlights. The latter strategy will take less time. But you will still choose to go to the bar sometimes, precisely because having a conversation with your friends, on any topic, is what you derive the enjoyment from in the first place. In addition, we sometimes enjoy being given more than just the highlights (there are people who sit through whole "conversations with Tyler" episodes rather than just reading the short blog post). Twitter is (was?) that bar niche. If what you want is the highlights you go to Buzzfeed or something - there are already curated best of twitter lists.

These points all make sense, however many people (myself included) use Twitter primarily as a news source. I selected the news organizations and pundits, politicos, economists etc. specifically because I want to know what they are saying, regardless whether I click or not. I don't want to miss items, and I sure as hell am not here for someone deciding what conversations I'll find helpful. I long ago abandoned Facebook and others because they didn't allow for this set up (and they are just plain exhausting). I guess catering to people like me might not be best for business, so I don't hold this against them for this reason, it's just disappointing that I'm going to lose out on this excellent resource.

I use Facebook but only the chronological newsfeed. Any Friends who post pish or whom I no longer care about I hide from my newsfeed. Any websites whose articles I know I won't be interested in (9gag, boredpanda, et cetera) I can hide, even if friends I follow post linking to them.

Facebook constantly prompts me to switch to "Top News" and has hidden "Recent Stories" in a corner of their mobile app.

It is also frustrating that even on Recent Stories, Facebook still refuses to show all posts from pages (not friends) that you like. Last weekend I missed five posts from the football team I support regarding a pitch inspection and subsequent postponement of that afternoon's match. Luckily I got notice through a friend. I liked those pages because I wanted to hear what those people/organisations say, it's frustrating when Facebook go out of their way to make it difficult for me to do so.

This is also how I feel. I specifically want to see everything in my twiiter feed in real time. I hand select who i follow based on their individual signal to noise ratio. There are many things I miss in my facebook feed because it curates it for me and it drives me crazy.

Quite. It's the very reason why Facebook is now of no use to me.

Algorithms are only of use when they serve you, not control you.

Plus1 This article is stupid. I can accept the general existence of algorithmic software without feeling I should subordinate every choice to it. I have spent a lot of time carefully deciding what material is on my feed and judging by the awful recommendations that Twitter makes I suspect I have a much better idea of what I like than they do. Seperate but equal objections include worrying that this would also exacerbate polarisation, one of the worst tendencies of social media, and undo the levelling effect social media provides by replacing the interesting with the empty, vacuous, popular nonsense that already fills our world.

One more person here for whom Fcebook"s curation is unnecessary and annoying (I don"t use Twitter.). I have maybe 50 friends, the vast majority of whom don't post at all on a given day, and I have no interest in posts from bands, movie stars, or media companies. To make me happy, just show me everything my friends post, in chronological order.

I empower "while you were away" with current Twitter, but have kind of a grudging relationship with it. It has information, but a narrow view of conversation.

I think Twitter's problem is they want to be a simple thing, and are not comfortable with the simple thing they are. So of course happy users become unhappy.

Seems to me Twitter could offer more optional views, more metaphors, and let users pick. What would be the downside? The default new user view could be as guided as they want, but a billion experienced users are going to have different goals.

Oh that must have been around the time when I quit using Facebook. It turned from a useful means of keeping in touch with friends and family to I quite don't know what.

The twitter 'algorithm' that is salient is the one filtering out rape reports during New Year's celebrations by gangs of recent 'refugees'. And yes it will be popular among heavy twitter users whose worldview is challenged by the reality.

Same here. On FB I disliked the feeling I was out of control of what I saw. Perhaps twitter will do it better, although I doubt it.

In addition, FB had a tendency to creep into real life. In random situations the thought "this would make a food FB post" would occur. It would disrupt the moment. Once I realized this was happening I closed my account.

"food FB post" = "good FB post", of course.

Twitter may know the sort of post I "like," re-tweet, or reply to, but that certainly doesn't mean I want to read them all first. Besides, there are many I read with interest without any action. And the real-time function is especially convenient for ongoing conversations.

Bottom line: if they change it in the manner described, I will use it far less.

Twitter could evade all criticism by having a button that let you toggle from algorithm feed to chronological and back, but they won't. And this is why it is rightly resisted and they are not trusted.

+1. but I suppose we need to be nudged along and freedom of choice makes us unhappy, right economists?

Another naive, techno utopian post. What was it a few years back, 3-D Printing?

Google's algorithms certainly haven't 'improved' - they're ranked now by their advertising value [to goog, not what you're necessarily trying to find]

Here's a more reasoned post on the topic:

"..the future belongs to people who can defer to the algorithm"


Try this: the future belongs to people who can interrogate algorithms & hold them accountable.

Tabbarok is a wanna be engineer/mathematician who became an economist and now likes to go around pretending to be a math/CS nerd. I get worked up about this

but then I bite into a York Peppermint Pattie AND I GET THE SENSATION OF BEING ON A MOUNTAIN TOP.

Taste the Sensation.

Isn't the whole issue that of who is making the relevant choices about what is important to them and how they keep random inputs in some type of context? To me the idea of linking self-driving cars (will be great for general transportation but please keep it out of my race car) and how some information stream is provided to me being similar in any meaningful way just seems silly.

Big data, average is over, great stagnation, Straussian, autonomous - just put the buzzwords wherever it suits you best.

Until it is done algorithmically, of course - then one can pine for the days of artisanally crafted posts written by a person using the buzzwords instead.

Though maybe not in Prof. Tabarrok's case - 'I have many more friends on Facebook than I have IRL because I trust the algorithm to bring me only the best of my friends on any given day.' Which is truly a bizarre way to frame 'friendship' - but makes it another fine buzzword to use.

If curation is the value, and we believe that human+computer has the best curation product, I would love to see twitter directly monetize that. Allow premium feeds, take a cut, and enforce some rational markets by establishing some sort of 'curation property rights' and ensure one premium feed isn't just a copy of another premium feed.

It is still worth noting that the algorithmic judgment may reinforce existing superstar effects in following. Perhaps the engineers at Twitter can avoid having a problem where new users fail to generate followers, but the combination of algorithmic curation and preferential attachment certainly poses a possibility of destruction of the news feed (and therefore much of Twitter's value). This is less of an issue on Facebook where networks are more "local" in some sense.

New thought on this post. If we're deferring to the algorithm isn't it more than the average is over. Isn't economics over as a social science and it's merely a subfield of software engineering or computer science?

The comparison between driverless cars and feed algorithms fails on several levels:
1) Driverless car algorithms have much clearer metrics to obey.
2) Driverless cars will be thoroughly tested for the stability of their algorithms.

Feed algorithms are opaque, subject to social engineering (by both Twitter/Facebook and their sponsors/advertisers), and can easily be destabilized by a cocky programmer who wants to show off.

It's not impossible that someone could invent and employ a stable, well-reasoned feed algorithm, but it hasn't happened yet and I doubt Twitter will be the one to do it.

When I bite into a York Peppermint Pattie I HAVE THE SENSATION OF BEING ON A MOUNTAIN TOP!

Taste the Sensation

> It’s not impossible that someone could invent and employ a stable, well-reasoned feed algorithm,

I think it is no exaggeration to say that such an algorithm would, for all practical purposes, be full-blown artificial intelligence.

The kind of task we are talking about here is essentially a software model of each user's thought processes. Even then it will be far from perfect because we are works in process. Even if the model could somehow start off close to the real thing it would immediately begin to diverge. It would require constant 'training' by virtue of a full-feed of all our sensory inputs 24x7 in order to stay in sync.

FWIW, it's the exact same problem as targeted advertising on the internet. You know, the old "we only show you ads for stuff you will be interested in" fantasy. And then reality turns out to be that once you buy a pair of shoes you constantly see ads for those shoes for the next two months.

The reality of ad targeting is that it isn't about showing you products you are interested in, but rather in profiling you to figure out which of your buttons to press in order to manipulate you in to buying whatever products their clients are selling. That is an achievable goal that works with little to no AI, just big profiling databases.

I am concerned that whatever twitter develops will end up doing something similar: the equivalent of fully personalized clickbait manipulating people into believing whatever agenda twitter's paying clients are pushing today. ("We've always been at war with Eastasia" etc.)

"As Tyler argued in Average is Over, the future belongs to people who can defer to the algorithm."

That's nice. I just don't want to live there.


“In another hundred and twenty days the building of the Integral will be completed. The great historic hour is near, when the first Integral will rise into the limitless space of the universe. One thousand years ago your heroic ancestors subjected the whole earth to the power of the United State. A still more glorious task is before you: the integration of the indefinite equation of the Cosmos by the use of the glass, electric, fire-breathing Integral. Your mission is to subjugate to the grateful yoke of reason the unknown beings who live on other planets, and who are perhaps still in the primitive state of freedom. If they will not understand that we are bringing them a mathematically faultless happiness, our duty will be to force them to be happy. But before we take up arms, we shall try the power of words.

“In the name of the Well-Doer, the following is announced herewith to all Numbers of the United State:

“Whoever feels capable must consider it his duty to write treatises, poems, manifestoes, Odes, and other compositions on the greatness and the beauty of the United State.

“This will be the first cargo which the Integral will carry.

“Long live the United State! Long live the Numbers!! Long live the Well-Doer!!!”

I think Twitter and Facebook's problem is the lure of Social Justice. When people use it, they want to use it. When they decide they are going to snoop on everyone and delete things that they find morally reprehensible, people will go elsewhere.

Facebook does this much more openly than Twitter and unlike Twitter is doing fine. Although my views are largely opposite to yours I am deeply concerned by the extent to which soft and hard censorship is advancing into online spaces (technically private but vital to everybody). It seems that the much lauded user participation in the various forms of social media is only valued when it serves as an echo chamber.

I am not sure that Facebook is doing it more than Twitter. Look at Milo Yiannopoulos's problems. If I remember correctly Twitter also introduced secondary boycotts during Gamergate, that is, they deleted accounts of people who were *close* to people they did not like. Not that they had said or done anything offensive themselves.

Use of Facebook is declining. I think that young people want a safe space where they can be moderately annoying. Not quite the full 4Chan experience, but not what they are getting either. That is too much like High School with teachers breathing over their necks and permanent damage being done to their job prospects.

For Twitter and Facebook, they might start out at The Doors, but in the end they want to be the Ed Sullivan Show. That is where the money is. So what happens when The Doors meets the Ed Sullivan Show? They let the morally outraged, the Social Justice Warriors, set the terms. It is just a pity they are not as honest, with themselves or about the facts or about their agenda, as the Christians were.

When I bite into a York Peppermint Patty I GET THE SENSATION OF BEING ON A MOUNTAIN TOP.

Defer to the Algorithm

Taste the Sensation.

Great demonstration of how someone who knows nothing about a topic can still manage to tie it into their favorite online political pet peeve (in your case "Social Justice").
If you've got enough ex-dittohead friends and family, you'll see that Facebook (at least) does very little to police the topic based on *any* notion of "Social Justice", "Political correctness" or whatever you want to call it. Facebook's real issue is the feed has basically become a recycling of whatever was floating around the meme boards (4-chan, reddit, tumblr, whatever) last week. Facebook is happy with this as they've ran out of things for people to spend more than half an hour on otherwise. And the farmville crowd isn't playing like they used to. This leads to the early adopters being annoyed, but huge growth in everyone else.
Twitter seems to have capped out the number of users who are willing to go through the effort of finding people who are worth following on their own. The goal with this is to grow their user base by making it easier and more friendly for non-power users. But, like Facebook, the effect is people who have put the most effort into min-maxing their feeds get annoyed because there's no way (short of mind-reading) that any algorithm will pay back the effort they've already put into it.
The other issue is you miss out on the immediacy where you are typing to someone who is also online at the same time and replying which is one of the other cool things about Twitter. My hope is they tweak the algorithm so that some of the less interesting posts are pushed down but it still favors people that I have wanted to read alot or who are online and active.
However, given the company's track record and the diversity of people who use Twitter, I think they won't succeed in pleasing anyone too much.

MyName February 7, 2016 at 9:41 pm

Great demonstration of how someone who knows nothing about a topic can still manage to tie it into their favorite online political pet peeve (in your case “Social Justice”).

I thoroughly enjoy a well thought out and sensible response that directly replies to something I have said. A pity you did not take that option. I made at least three points about declining use. You chose to criticize the one specifically marked as my opinion. Interesting.

If you’ve got enough ex-dittohead friends and family, you’ll see that Facebook (at least) does very little to police the topic based on *any* notion of “Social Justice”, “Political correctness” or whatever you want to call it.

Facebook has a long standing de facto policy of deleting conservatives. They deleted pages that supported George Zimmerman. Zuckerberg was actually caught on a live mic discussing censoring Facebook with Angela Merkel so that bad news about immigrants would not spread. OK, some of that is reasonable. I can see why they would not want to host the White History Month page. But they are way beyond that. The fact that you do not notice Facebook's Social Justice policies just reflects where your politics lie. Fish don't notice water.

Facebook’s real issue is the feed has basically become a recycling of whatever was floating around the meme boards (4-chan, reddit, tumblr, whatever) last week.

If that is what people want, that is what people want. That is hardly a disaster. The fact that employers now know to check people's Facebook past is likely to be a bigger problem.

People want a safe space to be a bit of an idiot. If Facebook doesn't provide that they will go elsewhere.

Flying cars to Mars. More likely the algorithm is designed to sell you something.

Your first error is in thinking self-driving cars are a popular idea with the general public.


'Your first error is in thinking self-driving cars are a popular idea with the general public.'

Then it is a mistake that Mercedes is making, particularly with models they cannot build enough of (a Finnish subcontractor is building 50,000 additional cars, and company executives are no longer able to 'order' this model as a company car).

Now it may be that the general public is not actually all that interested in buying a Mercedes, but Daimler has a fairly good record at making money while remaining, more or less, at the forefront of automotive engineering.

I don't know what I think about it - but just yesterday, along the 462, the car simply drove itself for about 15 kilometers, without hands on the steering wheel or feet on the pedals. The driver does need to dial in some information - max speed for the stretch or the minimum distance allowed for the vehicle in front, for example. The car also parks itself, but though that is becoming more commonplace these days.

Don't know what the general public thinks about it - the cars that offer such features tend not to be defined as 'lower priced mass market.'

The responses of an algorithm are only as good as the inputs.

A car-driving algorithm has inputs to use to generate responses. Twitter has no inputs that tell it what tweets I liked, only what tweets it showed me.

I think that you under-estimate how much information Twitter can get out of your behavior. How quickly do you read a Tweet? How many times do you read it? How often do you respond to that person or to Tweets containing these key words.

"people are more willing trust their physical lives to an algorithm than their twitter feed"

Who are these people (most people in the real world think that self-driving cars are a loony idea)?

'Who are these people'

Anyone who has flown on an Airbus passenger jet in the last couple of decades, for example.

Not to defend Prof. Tabarrok (who truly requires no defense), but sometimes the commenters here seem to be really, really unaware of the world we live in.

prior_test February 7, 2016 at 1:22 am

Anyone who has flown on an Airbus passenger jet in the last couple of decades, for example.

Virtually all the crashes of the Airbus have been linked to problems in their software:

The type's first fatal accident occurred on 30 June 1994 near Toulouse on a test flight when an Airbus-owned A330-300 crashed while simulating an engine failure on climbout, killing all seven on board.[43] Airbus subsequently advised A330 operators to disconnect the autopilot and limit pitch attitude in the event of an engine failure at low speed.[184]
The type's second fatal accident, and first while in commercial service, occurred on 1 June 2009 when Air France Flight 447, an A330-200 en route from Rio de Janeiro to Paris with 228 people on board, crashed in the Atlantic Ocean 640–800 kilometres (346–432 nmi; 398–497 mi) northeast of the islands of Fernando de Noronha,[188] with no survivors. .... Investigators later determined that the inadequate response of the pilots to both a loss of airspeed data from malfunctioning pitot tubes and subsequent autopilot disengagement resulted in Flight 447 entering into an aerodynamic stall.[190]
On 7 October 2008, Qantas Flight 72, an A330-300, suffered a rapid loss of altitude in two sudden uncommanded pitch-down manoeuvres while 150 km (81 nmi; 93 mi) from the Learmonth air base in northwestern Australia. After declaring an emergency, the crew landed the aircraft safely at Learmonth.[200] It was later determined that the incident, which caused 106 injuries, 14 of them serious, was the result of a design flaw of the plane's Air Data Inertial Reference Unit and a limitation of the aircraft's flight computer software.

People are quite right to be cautious about software.

'Virtually all the crashes of the Airbus have been linked to problems in their software'

Except for the notably worst crash, as noted in the very link you posted - 'Investigators later determined that the inadequate response of the pilots to both a loss of airspeed data from malfunctioning pitot tubes and subsequent autopilot disengagement resulted in Flight 447 entering into an aerodynamic stall.'

That Air France accident is truly remarkable, in that two trained airline pilots not only could not work together, but that they literally dropped the machine they were piloting into the ocean, as noted here -

'On 5 July 2012, the BEA released its final report on the accident. This confirmed the findings of the preliminary reports and provided additional details and recommendations to improve safety. According to the final report,[4] the accident resulted from the following succession of major events:

* temporary inconsistency between the measured speeds, likely as a result of the obstruction of the pitot tubes by ice crystals, causing autopilot disconnection and reconfiguration to alternate law;

* the crew made inappropriate control inputs that destabilized the flight path;

* the crew failed to follow appropriate procedure for loss of displayed airspeed information;

* the crew were late in identifying and correcting the deviation from the flight path;

* the crew lacked understanding of the approach to stall;

* the crew failed to recognize that the aircraft had stalled and consequently did not make inputs that would have made it possible to recover from the stall.[229]

These events resulted from the following major factors in combination:[4]

* feedback mechanisms on the part of those involved made it impossible to identify and remedy the repeated non-application of the procedure for inconsistent airspeed, and to ensure that crews were trained in icing of the Pitot probes and its consequences;

* the crew lacked practical training in manually handling the aircraft both at high altitude and in the event of anomalies of speed indication;

* the two co-pilots' task sharing was weakened both by incomprehension of the situation at the time of autopilot disconnection, and by poor management of the "startle effect", leaving them in an emotionally charged situation;

* the cockpit lacked a clear display of the inconsistencies in airspeed readings identified by the flight computers;

* the crew did not respond to the stall warning, whether due to a failure to identify the aural warning, to the brevity of the stall warnings that could have been considered spurious, to the absence of any visual information that could confirm that the aircraft was approaching stall after losing the characteristic speeds, to confusing stall-related buffet for overspeed-related buffet, to the indications by the Flight Director that might have confirmed the crew's mistaken view of their actions, or to difficulty in identifying and understanding the implications of the switch to alternate law, which does not protect the angle of attack.'

Essentially all the failures in that crash, from the non-replacement of defective parts to incredibly incompetent training procedures to utterly ineffective piloting, were human errors, not software ones.

People are right to be worried about trusting systems that can never be perfect - in the case of Air France 447, the software absolutely worked as designed. Unfortunately, not one of the human components functioned as intended.

And speaking from long association with military pilots - the lack of training in the most basic aspects of flying an aircraft remains inconceivable to me, but then, Air France is a commercial company, and obviously, their decision making concerning flying with defective equipment and utterly inadequate pilot training was the sort of thing that no software designer can ever influence (though we are to trust that companies are much better judges of safety than regulators, at least in the narrative presented endlessly at this web site).

Just to emphasize the point - the pilots literally dropped the Airbus they were piloting into the ocean.

It is amazing how you have managed to read that wrong. The computer lied to them. They were sure the computer knew what it was doing. It did not. They did not spot that it was lying to them in time. They did not know what to do when the computer did something incredibly stupid and dangerous without telling them. When they did something, they were too late.

The computer crashed that airplane. The pilots were simply too trusting of the computer and too untrained in what to do when the computer does not work properly.

Flying has never been safer. The fatal crash rate for the Airbus A300 series of planes is 0.47 per million flights (see Even when software makes mistakes, one has to compare it to the counterfactual world in which we are more exposed to human error. Flying was more dangerous even 30 years ago than it is today and the role that human error has played in most crashes is extremely well documented.

If it is content that I read regularly I prefer time-order items, otherwise I have to go through too many repeats. If the volume is high it is unlikely that the site keeps history of clicks.

"Show undeleted and uncensored posts sorted by most recent" is just another algorithm. It is not holy, merely simple.

I would say the future of those who "defer to the algorithm" belongs to those who give you "The" algorithm. Google shows you hits in the order of maximum expected revenue, to their best approximation; is that what is best for you?

Missing the point. Twitter views its timeline as a way to engage people with ads since their not clicking on the discover feature. There is also some concern that it will be used to automate filtering of controversial topics as a way to "shadowban" users without the moderation overhead of messing with the terms of service.

I don't know why you think FB has gotten better recently. For the last several months it has been showing me posts of people I don't know (and it should know I don't know) because someone I do know liked them. Why in the world would I want to see my cousin's college roommate's baby pictures? Something is going seriously wonky.

Is it possible at all that there are any dimensions to consider beyond "algorithms bad" and "algorithms good"? Such that one might prefer algorithms for certain tasks and humans for others?

"Don't fix what's not broken" is a nice saying and all, but what else are the people who run twitter going to do? They have to do something to justify their paychecks.

While I agree the consequences of trusting an algorithm appear more severe for cars, these are still apples and oranges. A Twitter algorithm is arbitrary and takes choice away where choice does not need to be taken. I have yet to see a selection algorithm that matches my own sequence of viewing things, and they definitely hide/bury items of interest. On the other hand a driving algorithm takes away choice where it is necessary - or irrelevant - because "choice" is not an element of driving. Such an algorithm can operate by a universally-agreed rules of the road. Choice is not part of driving in most cases - doing the expected action is key to safety. I shouldn't have a choice to cut lanes, run stop signs, assert rights at yield signs, etc. If there were such universal rules for Twitter then an algorithm might make sense.

Twitter already censors trending topics for political reasons. Why would you trust their algorithm to "bring only the most important items to your attention?"

What if there are heterogeneous effects, and people who didn't like Facebook algorithm (o/) ended up migrating to Twitter?

I'm personally unimpressed by Facebook's algorithms (as opposed to Gmail's spam filter or Amazon's). (a) The personalized ads I see on the right of my timeline are completely uninteresting, I've never find anything useful in them. This is good for me (no impulse purchases) but speaks badly on the algorithm. (b) One of the main reasons I migrated to Twitter was that Facebook became incredibly boring. The algorithm populated my TL with posts of people I don't interact at all, and never say anything I find interesting. I never like or comment their posts, nor do they like or comment on mine, yet they have a prominent and completely unjustified place in my TL. If Twitter is going to become that, I don't want it.

Exactly. Facebook's feed is based on what others prefer, and if you don't interact with it, it can't figure out what your preferences are. Which is why most people's Facebook feed, and news feed, is obnoxious as hell.

Alex however thinks that simply because there are "algorithms" involved in both cases, then you can't assume that one will be good and the other will be bad, even if they are applied to very different conditions and trying to do very different things.

That's like saying sociology is just as respectable and trustworthy as astro-physics because they both use "math".

There are some things beyond the comprehension of economists. Not, however, according to Alex. Comparing "algorithms" that drive cars to "algorithms" that pick twitter posts you may prefer over others is like comparing apples to bowling balls. They are both round, but hardly the same thing. Both "algorithms" in these cases are based on machine learning, but one is adapting to physical constraints, and the other is trying to guess psychological preferences.

I don't see why the stance that one wold produce good results and the other wouldn't, would be contradictory stances.

It all depends on what twitter is trying to do: if it's providing you with more tweets from people you regularly visit rather than simply chronologically, that might be ok. If its providing you with tweets others visit more on twitter, then it would end up like the disastrously idiotic Facebook news feed...where every other feed is some SJW BS article that makes you want to throw up, rather than want to read.

Since twitter is expressly trying to be more like facebook, I am pretty sure that they are going to screw up the timeline.

Comments for this post are closed