Professionalism vs. amateurism

Here was one MR reader request, from Philip W:

Professionalism vs. Amateurism, the merits and demerits of each. And the relationship of these to science, or “science.” How large is the role of “common sense” in your way of thinking about the world? Should we wish that policymakers would have more professionalism, or more common sense?

Amateurism is splendid when amateurs actually can make contributions.  A lot of the Industrial Revolution was driven by the inventions of so-called amateurs.  One of the most revolutionary economic sectors today — social networking — has been led by amateurs.  Maybe it is stretching the concept, but you can interpret Bill Gates and Steve Jobs as amateurs too.

Amateurs are associated with free entry and a lot of experimentation.  Barbecue quality is very often driven by amateurs, and in general amateurs still make contributions to food and cooking.   The difficulty of maintaining productive amateurs is one of the reasons why scientific progress periodically slows down.  Specialization, however necessary it may be, can make big breakthroughs harder at some margin.  (There is a good recent paper on this.)  This is one aspect of the division of labor which Adam Smith did not fully grasp, though he hinted at it.

Through computers, and the internet, the notion of amateurs working together is becoming more important.  This includes astronomical searches and theorem-proving, plus collection and collation of data, and Wikipedia; this is Shirky’s “cognitive surplus.”

On the latter part of the question, what is “common sense”?  Most common sense, if one can call it that, is a highly refined product of a lot of trial and error.  The real question is how to refine one’s common sense.

Policymakers need more of a sheer willingness to do the right thing, even if it means sacrificing reelection.  Selection mechanisms, however, do not much favor that bravery.  For a sane, well-adjusted person, the job is neither fun nor well-paying, so the job attracts people who love being in office and thus who fail to do the right thing.

When specialization proceeds very, very far, the difference between a professional and an amateur is sometimes no longer well-defined.

Comments

A big factor in the usefulness of amateurs is getting rid of barriers to entry. When computers were super expensive things that only government agencies, universities, and large companies had, there wasn't much opportunity for amateurs to start writing programs. After PCs became widespread, that changed--suddenly, a bright 15 year old could be writing interesting and useful programs. And IMO that drove an incredible explosion of innovation in the world of software.

Access to information (blogs, podcasts, iTunes U/OCW, published papers online) decreases the barriers to entry in a field. If you want to learn about something, to the extent you can learn by reading, you can just do it. That's an amazing change in the world, one whose implications we'll spend years sorting out.

Access to conversation with like-minded others also decreases the barriers to entry. It means you can talk with practitioners in the field you're interested in, and that amateur communities can grow up and mature into communities of practitioners.

And the third piece of this is access to tools. There are dozens of programming languages for which there are high-quality tools free for the downloading. Similarly, all kinds of plugins and programs for doing all kinds of stuff, from image processing to statistics to modeling of various kinds, are available online for free or commercially for relatively low cost. This is really crucial: most people who could make useful contributions to microbiology aren't going to be building their own microscopes.

Pushing against this are the forces that want to re-instate or increase those barriers: Guilds protecting their turf, national security types wanting to make some kinds of information and science off-limits to all but a pre-screened few, and academic publishers trying to keep their business model in place. But it's hard to see them winning anytime soon, though I expect we'll see a backlash against all this openness of information in the future, and it's possible they'll eventually win.

"Policymakers need more of a sheer willingness to do the right thing, even if it means sacrificing reelection."

Honestly, why should anyone pay attention to the recommendations of a man who believes that policymakers are elected?

http://en.wikipedia.org/wiki/Tin_foil_hat

Seriously?

http://www.usajobs.opm.gov/

Recent work in behavioral ecology on cognition and heuristics has big implications for "common sense". We are evolved beasts with stunningly efficient sub-consious mental tools for processing the kinds of information that were important in our evolutionary proving grounds. As we have become urban animals we have increasingly replaced that environment with one of our own making that has radically different characteristics. Money and economics are man made systems that do not exist in nature and run counter to most of our moral heuristics. Until we broadly recognize this reality, increasingly well documented, and institutionalize protections, economics and money will continue to be tools of subjugation rather than liberation. Previous eras of prosperity have all occurred in political economies that implemented power structures to prevent the abuses of normal human heuristics for opportunistic financial gain.

Common sense: the collection of all prejudices acquired by age 18.

... theorem-proving ...
I do not know of any amateur who could dream of proving a real theorem. Mathematics is probably the most professionalized field in existence. To be able to even understand statements that might become theorems, one needs to thoroughly grasp centuries of math, math that has increased exponentially in complexity and breadth over time.

neil, maybe tyler was obliquely referring to the amateur who claims to have proven p != np and whose proof is, as i understand it, being considered with some respect and possibility from "the establishment."

(in his case, his argument is very long and sophisticated - he's an outsider but not in my mind an amateur.)

There's a history of outsiders doing neat things. c.f. Perelman.

Perelman was no amateur. He was solitary, eccentric and semi-retired but no amateur.

Yes, I know. Outsider, not amateur.

Yes, no, maybe.

I think that the post has conflated "amateur" with "someone contributing compute time".

As soon as "common sense" is mentioned in contrast to "professionalism", I thought about mathematics. Common sense says that there is no real problem with the set of all sets, but about a hundred years ago, we figured out that it ain't so. As a friend of mine once put it (roughly). "You start out with a proposition. Your intuition tell you if it is true. Then you work the proof. If it turns out your intuition was wrong, you adjust your intuition."

So you have a field in which it takes a almost a decade to take someone with good talent and train their intuition sufficiently to be productive, and in the end, what do they do? The same basic trial & error as someone plotting the next revolution in their garage. (I spent 10 years working at AMD & IBM--the process REALLY IS the same there, as well.)

The professional's advantage are two. 1) The amount of time spent means that his intuition is highly refined. 2) He has access to much greater resources to pursue his ideas. Note that mathematics is no longer purely cerebral--compute power is a major factor now in achieving many proofs. The amateur's advantage is that his intuition is highly unrefined (by comparison). But in mathematics in particular, the space accessible to those with limited training is played out, and taste has precious little to do with the validity of a proof.

And I find your definition of common sense lacking.

I would add a third advantage to your list. 3) A professional does his work full-time, so he has fewer competing claims on his time. An amateur has to put work and family first, and only then can he relax and think about his hobby.

I think the role computing power plays in math is exaggerated. Aside from several highly publicized examples (4 color theorem, e.g.) where computers are used for checking cases, I haven't seen much computer use at all in proving theorems. My sense is that computers - particularly algebra systems like magma - are tools for checking conjectures against known but hard-to-compute examples, but are quite a way from being used to create proofs.

Let me add that I have a better feel for things on the topology/geometry side of the industry. For all I know, computers might be essential in combinatorics or graph theory (as Scott says), but that would surprise me.

Hmmm - I was specifically trying to say that amateurs don't necessarily need a lot of training *or* a lot of compute power to be useful in combinatorics, so I seem to have been unclear in my phrasing. I don't know of any mathematical field where progress simply cannot be achieved unless you have a lot of spare compute cycles to throw at things, although there are particular problems that seem to need that (for example, just counting all the knots that can be drawn with fewer than N crossings, which is far from being the most interesting problem in knot theory.) I don't think Polymath, for example, was primarily a compute-intensive effort.

This is by no means always true, although it's not quite as easy for an amateur to contribute here as it is to build, say, a useful new piece of software. The statements of lots of problems in combinatorics and graph theory are completely accessible, as are many useful techniques. A couple of years ago a number of professionals and amateurs launched the "Polymath project", a successful attempt to essentially crowdsource the proof of an open and nontrivial conjecture in combinatorics. That group (with an ever-shifting membership) has continued with further projects of varying success, but it's certainly the case that highly-motivated amateurs have contributed meaningfully.

With that said, it is undoubtedly the case that the main lines of number theory or algebraic geometry require a formidable amount of training these days. But from my reading, neither field seems to be moving particularly slowly because of that, because the number of open and interesting questions seems to be growing even faster than the amount of math that has been discovered - there is no lack of soft targets.

If common sense is distilled experience from your own life, then as the world changes more quickly, common sense will decline in usefulness. Fields where a lot has changed over the years will be fields where the older practitioners' common sense isn't as useful, and both newcomers and amateurs may be able to do better. Fields where things are the same over many years mean that common sense acquired from years of experience is more valuable.

Somewhere in there is a role for intelligence or wisdom or something, in figuring out how to apply experience from the previous world to understand the new one, without letting the old experience lead you off a cliff when it's no longer applicable. It seems like there ought to be a whole field of study involving generally how to do this.

This is not the first time you've stated that policymakers should 'do the right thing' and forgo being re-elected. I think most would agree in the short term and obviously the elected will still have great (higher paying) careers after leaving office. However, would this not create a super virus of politicians willing to hold the line on any dumb policy the electorate desires. That'd be the next species to evolve I think.

Democracy is the art of running the circus from the monkey cage
Mencken

Why would they care about what the electorate desires if they're willing to forgo being re-elected?

I find Tyler's answer thought-provoking but not altogether satisfying. My feeling is that professionalism generally requires one kind of blindness or another as a professional qualification, and that this has very serious consequences for how the world works today--especially in the professional worlds of law and "high" finance. Along these lines it is interesting to see how, for example, Michael Lewis celebrates people in The Big Short who existed somewhat as outsiders.

Another, less rarefied, way of putting my question would be this: what role should smart people's bullshit detectors be playing in making sense of our very complex political economy today? The pure professional answer is, "None--people's detectors just can't cut it in today's world, as they are adapted for a different kind of era." I emphatically reject that answer.

Still, policymakers (and, yes, the elected ones are actually very important) may not be sufficiently professional to be able to effectively evaluate policy alternatives offered to them by insiders. In this way, one man's technocracy is another man's capture. There are plenty of "populists" among our legislators who say that what we need is more common sense, and often their claims seem offensively naive, and yet something roughly in that style may well be necessary.

Hard to say what to do about all this, of course.

Thank you for expressing skepticism at "common sense." I think the proper characterization is "common sense" is knowledge that hasn't been subjected to careful scrutiny in the context of the available evidence. (Which is different than saying it's always wrong or even that the current status of careful scrutiny of available evidence isn't sometimes misleading.)

It is worth noting, if Bill Gates is included, that Linus Torvalds was, when Linux was begun, absolutely an amateur, as were most people contributing to the body of work we call Linux. (Technically, the name only attaches to the kernel.)
Now, it is mostly professionals maintaining these things, but I think there is an interesting study in there, somewhere, tracing the evolution from ami to pro.

In a post several months ago Tyler linked a paper describing the general progression from ami to pro. Its purpose was modeling industrial revolutions, but the ideas should be applicable to any field that matures from hobby to specialized profession.

Some Kuhn background would greatly increase the quality of this post.

Diversity of thought involves a lot more than "experimentation". It involves differences in background, training, and world view.

Diversity also helps to avoid failure due to blind spots -- selection theory can help you think about the role of diversity in exploring all of the dimensions of a potential adaptive space.

Similarly, diversity also provides a wider range of criticism -- criticism which can make a research program more robust.

Etc.

Tyler,

Any chance you could expand this into a discussion of amateur athletics? Specifically, the merits of professional development leagues (MLB's minor leagues) verses the NCAA Division I. For sports like football and basketball, the NCAA is essentially running a developmental league for the NFL and NBA. It is clear this set up is good for the NCAA, but I've always wondered whether it would be better for all parties (schools, professional leagues, and athletes) if the charade of "amateur athletics" was put to an end. Under such a system, the NBA and NFL would establish minor leagues or developmental leagues (like the big European soccer clubs) and athletes could choose to go pro out of high school or be real student athletes.

I always thought 'common sense' describes the knowledge of the commoners, which is often extremely useful, but at times blindingly wrong.

The ruling class has a vast store of knowledge (we see much of it on display here) but can also still be limited by what they do not know. That said, it's likely that despite his knowledge of the of the politics of Rome and the motivations of his capitol's ruling class, the King cannot build a fire or cook a meal, and likely he cannot tie his saddle onto his horse. He's never needed that knowledge.

In my view, common sense is refined by successfully confronting the problems faced by the common people. Have you changed a tire? It's just common sense that you have to loosen the lugnuts before you jack up the car.

As for professionalism vs. common sense in one's work, I'll give you an example from my field of engineering (lifted from the book 'The Introspective Engineer', and recovered from memory after about 15 years, so forgive me if there are any innacuracies)

The author of the book, a freshly-minted Civil engineer at his first employer, is given a task that requires him to specify the type and number of temporary supports for a wall that is to be constructed. He notes that typically in the past, the supports used would be 6"x6" lumber, but he does his job well and discovers that if he specifies 8"x8" lumber instead, he can order fewer of the supports, and net he will save the customer money. He feels very proud of a job well done until the general contractor calls his firm asking to speak to the idiot who sent him 8"x8" support beams. Apparently, the larger beams are too bulky / heavy to be handled by the workers and the net change is going to cost his customer a fortune in labor. After the unfortunate event, the author has gained a little common sense, and he's a better engineer.

It seems hard to imagine that the story doesn't generalize to almost any field.

That’s the wonderful thing with feedback.
However, as a economist, your only feedback will concern whether your results is consistent with current theory – which is consistent with previous theory - … - which, in the end, don´t even pretend to be realistic. But that’s ok – as long as It predict things in the real world – which it doesn’t – but… (I guess it should be some argument here but I still haven’t heard it - maybe someone can enlighten me)

Are Tyler and Alex professional or amateur public intellectuals?

Do professional economists fairly evaluate the contributions of amateur economists? (I'm not trying to pick a fight. I don't feel like I know the answer to the question.)

Aaron:

A related question: How hard is it in various subfields of economics, and in other disciplines, for an amateur to get out to an "edge" where he can do useful work? In some fields, the accessible questions may already have a lot of full-timers working on them, or may require hard-to-get equipment. In others, it takes years to understand enough to be able to do anything useful once you get to an edge. In still others, you can ramp up relatively quickly and there are a lot of available edges that aren't already crowded with people frantically publishing in hopes of one day getting a job. Probably only one of these situations offers many opportunities for amateurs.

A related question, in some sense, is whether there's a path for amateurs to become professionals. In some jobs, doing the job is the critical part--you can be a programmer without any degree or certification at all, for example. In others, there's simply no entry into the field without the credential--even if you've done great work, you're not going to be allowed to treat patients without going through some kind of formal certification process for an MD or NP or PA or something. How hard would it be to become an economics professor at a decent university without a PhD in economics, or at least in some plausibly related field like math or political science?

The Universe is Expanding, We learn new things, The Universe Shrinks, We Forget.

A interesting aspect is that amateurs scale much better than professionals. Professionals often have high stakes in getting a breakthrough attributed to themselves. This impedes collaboration and knowledge sharing. The risk of another guy (or firm) scooping your next-great-idea is high. The stakes are high too. Hence many professionals are likely to be secretive. Look at proofs in math for example. If you have a new idea for a proof etc. I doubt many would post it on an open forum.

Amateurs on the other hand collaborate much better and very early too. Open source projects are a great example.

Comments for this post are closed