More sectors bad at finding talent (from the comments)

Surgery (and many medical specialties, esp. highly compensated ones) should be on the list of ‘Bad at finding best talent.’ There’s no way to show aptitude for a surgical specialty before medical school, and there is no mechanism for good surgeons to rise to the top, and bad surgeons to be identified and punished. If you make it into a surgical residency, you will succeed, even if you faked your way into med school and your surgical success rate is terrible. There is essentially no mechanisms to make sure aging surgeons learn the newest techniques, and no checks on waning competency. It is only because the training is so long and difficult that it isn’t a complete disaster.

Policing should also be on the list. It’s another job where, like being a surgeon, once you’ve made it into the profession, you have to fail spectacularly to be kicked out. At least half the police officers I know shouldn’t be allowed to carry firearms, much less have the power of life and death over ordinary citizens.

That is from Kevin, based on my earlier post on this question.

Comments

Surfing in this evidence-free zone is quite exhilarating!

In this case it is using the structure of the profession as evidence of professions likely to be bad at selecting for talent.

It is not perfect or ideal evidence but it is not mere anecdote.

Carthage?

To be fair, the structure of the profession is not evidence that the profession is actually bad at selecting talent. It is an a priori basis for hypothesizing that the profession is bad.

And your truly rigorous comment to go with it.

> In a Mayo Clinic study with the American College of Surgeons, 8.9% of participating U.S. surgeons reported the belief that they’ve made a major medical error within the last 3 months — and 1.5% believe their error resulted in a patient’s death

https://catalyst.nejm.org/medical-errors-preventable-deaths/

Hedge fund managers, of course, never make mistakes. Nor do TV executives, CEOs and economists.

So that is what we have become: Death, Destroyer of worlds.

Tennis, btw, is actually pretty good at finding talent.

If it suffers at all, it's mainly because
1. you have to start really, really early. (High school facilities have nothing to do with it).
2. zero-sum competition between sports for a fixed pool of athletic talent. For women, this is less of a factor. (How many sports offer the prestige and money that tennis does?) Plus, tennis is a truly global sport, so it's casting a wider net than most sports.

Everyone thinks tennis is a country-club-gated sport, but it's not true at all in terms of who ends up becoming a top professional. Most have come from middle, lower-middle, and often poor backgrounds (e.g. the Williamses, Sharapova, Agassi, Ashe, Connors). It's actually pretty easy to gauge the promise of a very young (even 5 yr old) tennis player, and the tennis community is pretty good at nurturing or referring extraordinary talent up into the academies where they'll get all kinds of needs-based support depending on their talent.

As far as minorities go, tennis is pretty good compared to most any other sector. Again, it helps to be international. But also, tennis is probably the most socially progressive sport on almost any dimension. As just one of many, many examples, Althea Gibson broke the color barrier before Jackie Robinson did.

If you are measuring global talent recognition then Tennis is comfortably the best sport. The world top 20 represents 16 countries and 5 continents on the men's side, with 14 countries on the womens. Asia isn't represented in either top 20, but has 2 in the top 34 for both genders.

Apart from Africa (excluding SA/Zim) pretty much every region of the world has developed a top quality tennis player. I can't think of any other discipline (sport or other) where that would be the case.

Apart from Africa (excluding SA/Zim) pretty much every region of the world has developed a top quality tennis player. I can’t think of any other discipline (sport or other) where that would be the case.

Basketball? Obviously, North America has developed lots of "top quality" players, but so has South America (several Brazilians and Argentinians), Europe (Dirk Nowitzki, the Gasol brothers, etc., etc.) Asia - (Yao Ming), Africa - (Serge Ibaka, Hakeem Olajuwon, Joel Emiid, etc.) The biggest possible exception would be Australia, where, I think, the biggest stars until this year would likely be Andrew Bogut or Patty Mills. You could quibble as to whether either was a "top quality" player, though Bogut was a former number 1 draft pick and NBA champion, and Mills is an important contributor to a top team and was an important player on a championship team. Ben Simmons, likely rookie of the year this year, is plausibly the future best player from Australia.

Basketball is pretty good, but by region, I really meant sub continental region. There have been top 20 players from India, Thailand, all across South America, the Caribbean. At least 50 countries would have had a male or Female 20 top player in the world.

I would guess the Basketball equivalent is All Star (roughly 20 players a year) How many countries would ever have had an all star? 20?

On Football it's the same, there are hundreds off professionals in the premier league, which gives opportunities for plenty of different nationalities, but if you were to list the top 20 players in the world, they would almost all be Western European or South American.

Being an All-Star in the NBA is only a very rough estimate for being a top-20 player in a given year, because, especially for the fan vote, being on a winning team with, say, Lebron Jame or Michael Jordon, or having been good in the past, is a good way to get on the team even if you're not plausibly a top-20 player. So, it is both over-inclusive and under-inclusive. But, you can get at least 20 countries listed. Here's a list of all-stars from countries other than the US, based just on people I knew were from outside the US, so there may be others:
Giannis Antetokounmpo – Greece; Joel Embiid – Cameroon; Marc Gasol, Paul Gasol, Wally Szczerbiak – Spain; Al Harford – Dominican Republic; Tim Duncan – Virgin Islands; Dirk Nowitzki, Detlef Schremph – Germany; Tony Parker – France; Luol Deng – (South) Sudan; Steve Nash – South Africa (Canada); Yao Ming – China; Manu Ginobili – Argentina; Mehmet Okur – Turkey; Zydrunas Ilgauskas – Lithuania; Andrei Kirilenko – Russia; Peje Stojakovic – Croatia; Dikembe Mutombo – DR Congo; vlade Divac – Serbia; Rik Smit – Netherlands; Hakeem Olajuwon – Nigeria; Patrick Ewing – Jamaica;

Soccer? Manchester United all by itself has almost more diversity than the entire United Nations.

Most tennis players are the children of tennis coaches. That would suggest that in fact tennis selects from a very small pool of players. For instance the top US tennis players are: Isner (6'11 so a special case), Querrey (mother was a tennis coach), Johnson (father was a tennis coach), Sandgren (mother), Tiafoe (father was a janitor at a tennis court), Harrison (father), Escobedo (father played pro tennis), etc.

Taylor Fritz (mother was a tennis player who married a tennis pro).

"Most tennis players are the children of tennis coaches".

Probably not true.
In the top 25 for the men, I count 8: Nadal (uncle), Dimitrov, Zverev, Thiem, Goffin, Querrey, Murray, Mannarino. For the women... it might be just 1? (Ostapenko) Though there are some edge cases there.

Now compare that to the rate someone selected randomly from the world would have a parent as a coach

What would that tell you? (Serious question)

How do you insert the free market into the field of surgeons?

Verrrrrrrrrry carefully.

Performance related pay?

Have surgeons bid with the guarantee of X months of further life to the patient? No live, no fee?

Most of that is beyond the surgeon's control. For instance, say you go in for a kidney transplant; your life span is much more heavily impacted by your adherence to the post-op instructions (e.g. what meds to take, what weight to maintain) than by anything the surgeon does. Worse some of the risk factors happen to be things where it is easy for surgeons to discriminate (e.g. black patients tend to be less responsive to a number of anti-hypertensive agents and have increased mortality for anything kidney related).

We can start getting complicated about coding the patient before the procedure to try to measure the actual impact of the surgeon, but then you can game the system by upcoding before you cut. Like with many things, ungameable metrics are in very short supply.

This gets worse as surgeons go through a natural career lifecycle. They start off relatively inexperienced and have higher mortality rates, they become proficient at their surgeries and often specialize within their specific discipline (e.g. doing only hip replacements), and then they approach retirement (when they are less likely to learn new procedures). Even at any given stage, surgeons have vastly different mortality rates when learning a new procedure. Grading all of this is hard. You do not want to penalize surgeons just for being new to the job and we already have enough trouble keeping surgeons from retiring with useful years in the OR ahead of them.

We already have a hard time treating to the patient and not the numbers (e.g. blood pressure, infection rates, readmission rates); all of these options make that worse.

Sorry, all of this was meant to be in reply to Alistair.

Let's say we have two roughly equal sized towns with average police forces. Suppose we fire both forces and rehire them. Those above average are hired for of the towns and below average for the other..

Revisit the town 1, 5 and 10 years later. As a social scientist, what measures would you use to determine which town got the above average and which the below average force? If you are unable to rely upon any measures to say with confidence which town got which, is there any meaning to the idea of talent in this context?

Indeed. What is measured? The number of infractions that are posted? The number of arrests? How many doors were knocked down?

Similar with surgeons. If your beat is a very rough part of town, good policing will have an effect over a generation. As well as bad policing. The best way to get good numbers is to get an easy beat. An emergency surgeon who has moments to decide how to proceed will inevitably make mistakes.

There are lots of occupations where short term numbers are easy to fix. Property management is one. You can defer expensive maintenance repairs and show exceptional numbers for 4-5 years, long enough to get a promotion and plaudits. The poor sob that follows you will have to spend every penny you saved two or three times over when a simple repair turns into a major renovation to fix rot or equipment replacement.

In theory I think surgeons should be able to be objectively measured for quality. You can tally stats on the operations they perform, control for procedures by difficulty and even control for the underlying health of the patient.

Policing, however, seems highly localized to me. An officer who knows his beat very well after working it 5-10 years may be pretty bad if he is suddenly transferred to someplace that's radically different.

Not so much.

The biggest factors impacting patient outcomes are under the patient's control: how soon do they get moving, how aggressively to they do rehab, how regularly do they take their medication, how well do they adhere to their diet, and even if they have friends visit. It is pretty hard to tease out what is the surgeon and what is the patient.

It is, however, pretty easy to screen patients. Non-poor patients are vastly more likely to heal better. They are vastly more likely to follow directions. They are more likely to sue you, but in general you can do much better by having patients than being a better doctor.

Even if we control for patients, it is pretty hard to measure the surgeon's impact as a lot of supporting healthcare providers - nurses, aides, social workers, etc. all have huge impacts.

There's no need to screen patients. If two surgeons do a lot of one type of operation (say 100 or more) and have the same general mix of patients then the average outcomes should be relatively close *unless* the difference can be attributed to the actual quality of the surgery. Assuming they both have the same average then there's no 'talent' problem in the surgeons (unless the average for both is pretty horrible).

The hiring of police is often most often about social engineering then hiring the most qualified. The process of hiring and promoting police officers is so denigrated with political clout and vague liberal notions of racial and gender correctness. It is amazing as many people apply for the job as they do.

But any you find the current state so awful is free to apply for the job. Is Kenin signing up?

But those who find the current state so awful are free to apply for the job of police officer. Is Kevin signing up?

Sorry. typing on a phone, at an airport, without my reading glasses, isn't easy for me.

Is how it works now? "Hey, you don't like how much money politicians take, become one" or "I am sorry the doctor amputated the wrong lef, but if you had cut your leg off yourself, it wouldn't have happened". "You don't like how cop X beat up helpless people, you are welcome to become a cop and not beat people up. Problem solved."

Yes, that is how it works. If you don’t like it do something about it. Place your life on the line, dedicate yourself to a path. If a given field has so much difficulty attracting the type of people you desire ask why. Ask why is the market failing. Of course you can just say that police are scum. Is that how it works?

"Of course you can just say that police are scum. Is that how it works?"
Some of them demonstrably are (some of the politicians, too, some of the doctors, too, etc.). And the "type of people" I desire is honest, law-abbiding people. I remember a time when Conservatives used to say things like "simple honesty is not too much to demand of men in government. We find it in most. Republicans demand it from everyone."

Place your life on the line

If you want to place your life on the line, become a lumberjack, who is many times more likely to be killed at work than a cop. http://nailheadtom.blogspot.com/2018/01/the-chances-of-being-killed-at-work.html

"he hiring of police is often most often about social engineering then hiring the most qualified"

That's interesting because in most places the process begins with a mandatory civil service or police officers' exam. Do you think 'most qualified' would mean you simply rank all the scores highest to lowest and start hiring from the top and work your way down until you filled up all the slots?

That assumes the exam is actually a good measure of talent or 'qualified' but you and I both know there's no studies anywhere that looks at individual police offers' performance against their scores on the exam(s). Also if you think about it almost no jobs anywhere hire using an exam. You may be asked about your SAT scores or grades in a job interview....some places like Home Depot have a screening test to see if you have some basic knowledge about the industry but almost no jobs hire simply based on who has the highest score. The fact then that there's a test as even part of the hiring process seems to indicate it's a lot more about 'most qualified' than one would think.

You just miss the point, again. Test are an incomplete method for selecting candidates. How does race norming the test, including the psychology portion, improve the test.? The Rand corporation did a study on candidates for police jobs. White candidates tend to hold police officers in high regard, view the job as a calling, have family members who are police officers, and see it as a way to help people. African American candidates held the job in lower regard and tended to view it as a civil service job or government job. Hispanics weren’t addressed.

Promotions exams are sold as being on job specific skills needed. Understanding of criminal code, department procedures.

Still the test is for basic physical condition, basic intelligence, psychological screen, background check, all without violating constitutional rights. Then check race, sex, political clout, and re-adjust scores. Put them on the streets and blame them social ills. Make asinine comments about them being monsters.

Then wonder why the quality of candidates may not be optimal

Then there's stuff like this: http://nailheadtom.blogspot.com/2018/01/photographs-of-evil-people.html

Are you that dense? Police officers sometimes work undercover. The need to protect their identity protects them and civilians that they may have worked with. The courts and society have recognized this repeatedly because it is common sense. Or do you want criminals to have the home address of the police who arrest them? The names of the schools their children attend? You are an ass.

Police officers commit crimes. Politicians commit crimes. School teachers commit crimes. Clergy commit crimes. Soldiers commit crimes. Supervisors at auto plants commit crimes. Brain dead bloggers commit crimes. Human beings are flawed. Hatred for the police seems to have blinded you to reality.

1. Tests are not 'race normed' if by that you mean something like "blacks get 15 extra points on their score'.

2. Tests invite you to think in terms of the score. The high score is supposedly the best qualified person and anything else is 'politics' that subtracts from that ideal. But exactly what test are we talking about? What evidence is there that test works to find the best candidates (the higher the better) versus just filtering out unqualified (i.e. anyone who can't get a 70+ is out of the pool but scoring an 85 doesn't make you better than someone who scored 79)?

3. " White candidates tend to hold police officers in high regard, view the job as a calling, have family members who are police officers, and see it as a way to help people."

The flip side is white candidates feel the job is their personal entitlement, part of their family's tradition and therefore they are entitled to the job and family members on the force are obligated to help them get it and keep it regardless of their individual fitness for the job. Likewise a Hispanic/Black person without this entitlement mentality may view it as a job which is hard to get so they better keep it by doing it well.

This 'just so' story can be told from either the positive or negative angle for any particular candidate. How exactly do you test for this and then establish your test works.

Surgical assistants are much more exposed to standard market signals than surgeons, and they often perform the bulk of the surgery. The prestigious hospitals are generally teaching hospitals though, so they use over-worked interns instead.

I once asked a university medic why they didn't test would-be surgeons for their hand-eye co-ordination and fine motor skills. He gulped and said that he supposed that they hoped that clumsy clots wouldn't apply to be surgeons.

As usual I got far more sense from a vet. When I asked how they dealt with students who were unsuited - by virtue of clumsiness or anything else - to be vets, I was told "Easy: we fail them on the practical exams."

Air Forces have known for some time what makes a very good fighter pilot - someone with good eyesight who is also a loner, with asocial tendencies, and a long history of fights. Which is not a surprise as being a successful fighter pilot is a lot like sneaking up on people in a dark alley and stabbing them in the back before they know you are there.

They have resisted making this obvious in their selection. In fact the US Air Force famously tried to insist that every pilot should have a college degree. Back in the days when they were training more pilots than college graduates. Western air forces still try to select for things that look a lot like a proxy for Class. Even the Royal Air Force which has long been opposed to anything that looks like Britain's traditional ruling class.

Luckily the West makes very good air planes with even better electronics so whether this is a better policy or not has never been put to the test.

When I was in med school I opted out of surgery for that and some other reasons. Students electing to go into surgery have a few years to vet themselves, and teaching docs vetting them as well. Then multiple post-grad years doing the same.

Most surgery is relatively benign and death is very unlikely. A doc will be monitored by his/her peers and hospital all along the way. And any complication and certainly an unexpected death will come into review.

When I was in hospital leadership positions we tossed out a few surgeons not doing the best job.

What about doctors in general? My better half is a specialist that treats a common treatable/controllable chronic condition. She has taken over patients when other doctors left her practice who have been essentially untreated (ie treated incorrectly) for years. The patients, for the most part, don't realize that they should have been getting improving.

Surgeons, if anything, have a bit more of a control as often they get referrals from other doctors. She knows the surgeons that do a good job treating her patients, and send them to those surgeons.

Very true. Word of mouth from patients, referring docs and other surgeons works with this arcane market.

As an internist referring to surgeons I would not use a new surgeon for about 2 years until proven worthy.

One problem is that many disciplines require different talents at different phases of learning. I myself am a physicist by training. Early education strongly selects (both by self-selection and by filtering others out) for technical prowess in calculus. However, after you learn to do calculus fast and without errors, other skills turn out to be more important. In actual research, work ethic, some sort of creativity, social skills (very little research today is done in isolation) and deep, intrinsic interest in the field are more important. I'm also convinced that almost anyone can learn the calculus part with enough time and effort while the skills needed later in the career are more difficult to acquire. This is why the selection process ends up being quite bad in the end with many people doing PhD that are not really suited for research and are not even that interested in the field.

"many people doing PhD that are not really ... that interested in the field": bonkers, isn't it? You might wonder why the relevant Physics department would admit them.

Few professions make worse selection decisions than University Professor.

I've got a masters in a STEM field. We had professors who clearly hated it, were incapable of speaking loud enough, were incapable of being understood because of their accent, resented being asked to explain something a second time, and/or took months to bring themselves to grade a simple quiz for a 15-person class.

About two-thirds of them were good, however. But that grade works out to a D.

Thank God for good textbook writers.

+1

I am also a Physicist.

I think what you're describing is more general. In any field that requires talent in multiple areas, people get held back by the part of their portfolio that is relatively worst.

If you take an outfielder who runs like a deer, hits the ball a country mile, makes great plays on fly balls and steals bases with wild abandon, but can't hit a curveball, he will see nothing but curveballs and wash out of the league. If he learns to hit the curveball, the rest of his game will come into play and his career will progress until some other limiting factor enters the picture.

"Technically good at solving research problems but has no real vision for the field" is a problem people encounter late in the process, after clearing the initial hurdles like calculus.

It seems that based on these examples, there is poor allocation of talent in industries where there are high barriers to entry, insufficient mechanisms to generate competition, high demand relative to supply, and poor availability of information about the relative merits of different providers of talent. Sounds to me like the market for any other good or service.

Talent is rare, and the sectors that excel at allocating talent in the end have a small number to allocate.

In my industry talented people are fully occupied and paid very well. The middle does just fine, and the very untalented either fall out or teach.

The consumer has no way of knowing except by reputation and experience. Sometimes very costly experience.

"the very untalented either fall out or teach" - Another reason talent allocation in teaching is so poor, inability to distinguish the qualified, passionate, talented teachers from those who are washed out from other industries?

Culture in surgery actually does police poor performance - they treat bad residents absolutely savagely.

I added this discussion to http://www.textbooksfree.org/Education%20Libraries.htm#Curriculum_Stuff_
under the title of Talent Optimized.

What do you call a doctor who received the lowest passing marks on his/her medical exams and almost failed their residency? A surgeon.

Medical professions are good at policing their guild members. They do it clandestinely, out of the public eye. In Israel, patients can choose the surgeon so much real (and fake) information is being circulated. Prestige is dis proportionally rewarded.

Actually, Justin Dimick at University of Michigan has shown that, using video of the surgeries themselves, you can objectively measure surgical quality.

https://www.ncbi.nlm.nih.gov/pubmed/26077990
https://www.ncbi.nlm.nih.gov/pubmed/28686537

--bruce

Paging Heather Parsons... https://scholar.harvard.edu/sarsons/publications/interpreting-signals-evidence-medical-referrals

Clearly surgery qualifies as a field that is poor at selecting the best talent. You can generate plausible, if politically incorrect, stories for why women may be disproportionately punished in some fields, but not surgery, where fine motor skills should be paramount. Perhaps field that require low levels of agreeableness such as, litigators, compliance professionals and negotiators, would fit that bill?

Viewing this question through a gender lens, although not always applicable, is an interesting approach that could yield many benefits.

Comments for this post are closed