The growth of on-line tests in assessment (average is over)

This continues to be a growing trend:

T-Mobile asks job applicants to take this [problem-solving, for a customer] test before inviting them for an interview because the company has found powerful correlations between the online assessments and success on the job. High scorers tend to resolve customer calls about 25 seconds faster than those who receive low scores. That means they can handle one more call a day and about 250 more a year.

More generally:

Companies are using these tests to evaluate skills and personalities for job openings at every rung of the career ladder, from bank teller to C-suite executive. They are not merely on-screen versions of decades-old paper employment tests. They are built on the power of big data: Creators have harnessed a massive trove of results to help companies pinpoint the kind of worker who might thrive in a particular job.

The legion of tests is only growing:

Some tests evaluate a specific skill, such as how quickly and accurately someone can make change from an onscreen cash register or program software in the Java coding language. Many tests incorporate simulations of scenarios one might encounter on the job. Marriott International, for example, shows housekeeping applicants a photo of a landscaped area at one of its hotels and asks candidates to determine what’s wrong with it. (Perhaps a gardening tool was not put away properly). In one of CEB’s tests for a supervisory role, applicants might have to demonstrate how they would talk to an employee who was coming in late and missing important meetings.

We are entering a new “meritocracy,” at least for people who test well, especially on-line:

Providers say the tests hold the promise of leveling the playing field for job applicants by removing the chance of bias that comes with a traditional résumé screening. The tests can’t distinguish, for example, if a candidate didn’t attend a top-tier college, is currently unemployed or is a woman or minority.

“In many cases, algorithms can trump instinct on staffing,” said John Boudreau, a professor in the business school at the University of Southern California, adding that decades of research have found that tests can serve as reliable barometers of certain personality traits, such as conscientiousness.

The full story, by Sarah Halzack, is here.

Comments

note also the growth in companies that advise you on how to answer such tests (i.e., selling insider information on what the personality tests are looking for)

one wonders who wins the arms race

The general problem with personality tests as opposed to IQ tests is that you can use IQ to figure out the wanted answers to questions. Since IQ correlates pretty positively with most kinds of job performance, that's not so bad, but it does raise worries about positions where a person who is both smart and unethical could be extra trouble.

IQ tests have the problem where a smart person, or persons, can collude to answer an IT test. You can solve this by having testing centers, or doing live testing, but then you would lose out on much of the cost savings to begin with

a well-designed personality test has genuinely ambiguous multiple choices, where there's no benefit to bribing the whiz kid to answer for you.

I agree I just did one for a job - the questions far and by large did not have a correct answer, just more and less desirable regions in the continuum and even there some of them are non-obvious what they are looking for. FWIW, I scored 100/100 on the g-loaded part of the test so it's not that I lack the IQ to figure out the personality test (though I will give you that: it's easy enough to manipulate stuff like Myers-Briggs)

The venerable old psychological tests like the MMPI from the 1940s are quite good at grinding down the test-taker into not trying to outsmart it, but do you think they could stand up Amy Chua-style Tiger Mothering?

Here's an interesting bit from Wikipedia on the MMPI:

"One of the biggest criticisms of the test is the difference between whites and non-whites. Minorities tend to score five points higher on the test [i.e., more psychopathic]. Charles McCreary and Eligio Padilla from the University of California, Los Angeles state, "There is continuing controversy about the appropriateness of the MMPI when decisions involve persons from non-white racial and ethnic backgrounds. In general, studies of such divergent populations as prison inmates, medical patients, psychiatric patients, and high school and college students have found that blacks usually score higher than whites on the L, F, Sc, and Ma scales. There is near agreement that the notion of more psychopathology in racial ethnic minority groups is simplistic and untenable."

http://en.wikipedia.org/wiki/Minnesota_Multiphasic_Personality_Inventory#Criticisms

Well, of course, indeed ... It would be simplistic and untenable to to assume that a homicide rate 7X or 8X higher can be considered evidence of more psychopathology among blacks on average, so this 71-year-old test must be fundamentally broken in some unfixable fashion.

Steve,

Then the difference between races should disappear if, say, you are giving the test to inmates serving sentences for serious crimes. In that case the population has already been self-selected in favor of psychopathy. If the difference doesn't disappear, then that indicates the test is not reliable...at least not reliable outside of white populations.

When I had real jobs, I worked at a few firms that did a lot of testing. I've sat for a lot of personality tests as well as IQ tests. My observation is that once you get into high average IQ, the personality tests fall apart. The test taker starts gaming their answers. Until that point, they provide results consistent with observation, but nothing you could not figure out in an interview. I'm not a skeptic of personality testing. You just have to accept their limitations.

As an aside, I don't know if they still do it, but in the 1990's the U-Haul Company administered a basic math and vocabulary test to all applicants. For managers they administered the Wonderlick and another aptitude test. From what I understand, it was not used in hiring decisions. The results were not shared with the hiring managers. I always wondered what they did with the results. Given the size and age of the company, the data would be useful.

This kind of thing will probably work fine for people answering phones. For executives? Give me a break. Any test that matters can be gamed by people at that level.

It's like how there are laws about how cabdrivers can't discriminate but those don't apply to Uber or Lyft because they are, like, digital.

Hollywood and Silicon Valley are massive Democrat donors. They are getting something for their money.

On the other hand Lawyers are too. But clearly the Catholic vote is not what it was.

It has the advantage of eliminating some types of discrimination in favor of a more rectifiable kind, if done right. The hiring committee can't be subconsciously biased against you because you're black if they never get a chance to evaluate your race in the hiring decision.

The law going back to Griggs v. Duke Power in the early 1970s says, more or less, that the burden of proof is on employers to prove that their objective hiring tools don't inadvertently statistically discriminate against protected classes such as blacks and women.

I've noticed a growing trend to assume that discrimination law doesn't apply to Silicon Valley and its products because they are digital or disruptive or something.

Anyway, it's a great era to be in the test design business because everybody assumes that the test designers of the past must have been racist idiots, so all we have to do is hire a smart new test designer and all that bad old disparate impact will vanish. And when that doesn't happen, then we just have to hire a different smart new test designer. And on and on.

One thing about Silicon Valley is companies don't have a very long life expectancy. If you're in the basis of shaking down corporate boards for discrimination lucre than it helps if the company is going to be around for a while. Expose a scandal against GE and they're donating to your Social Justice Foundation and hiring your diversity consultants for decades. There's not as much incentive to go after a startup that might only be around for a few years.

Right, it's like how the 2000-year-old Roman Catholic Church was a sitting duck for sex abuse lawsuits, while Hollywood has barely been touched. Sure, it's the same guys pulling the same stuff decade after decade in the entertainment industry, but the contractual fluidity makes the challenge to the plaintiffs' lawyers of getting their hand in a deep pocket more complicated.

So what's your point? The large size or long history of an institution ought to be a valid defense again malfeasance? A mitigating factor? "Oh we are GE or the Vatican so we are soft targets please look elsewhere?"

What's stopping you from suing Hollywood? If you win I'm sure we will cheer.

@Rahul: Deliberate ignorance to try and score cheap theatrical points is not very attractive. It is obvious that the point being made is race hustlers target stable established firms.

Everyone chooses their fight. So what? EPA targets biggest polluters with a vengeance. Prosecutors are often going for the big guys to make a statement. Tons of other examples. This isn't about race. Sailer's church sex abuse example makes that amply clear.

"This isn’t about race."

Disparate impact racial discrimination lawsuits are about race.

That might be changing.

http://bits.blogs.nytimes.com/2014/03/19/jesse-jackson-confronts-silicon-valley/?_php=true&_type=blogs&_r=0

Jesse has tried to shake down Silicon Valley before, with only minor payoffs, and he's not getting any younger and mentally quicker. Most of the energy in shaking down Silicon Valley these days seems to be focused on payoffs for women. The NYT reporter Cain Miller has a running series on attractive young women oppressed by the "alpha male culture" of Silicon Valley. If you read closely, most of the young ladies featured in the NYT appear to be what past ages would call adventuresses -- not really programmers, just women who want to be where rich men are.

Similarly, V. Stiviano is a down-market version of the return of the adventuress.

Apple, Google, Microsoft, Yahoo, Dell, IBM, SAP, Oracle, Intel, even Facebook. All these companies have been around for years and they most likely will still be here 10 years from now even if things to badly for them.

GM & the Roman Catholic Church are poor examples. Consider, say, Circuit City, Borders vs. any of the Big Three automakers. But for the bailout some or all of them would now be in the same garbage bin as Circuit City or Boarders Bookstores.

If those tech companies were discriminating based on race, there's plenty of incentive for lawyers to go after them and so-called brick and morter companies are hardly guaranteed to stick around as long as actual bricks do!

It isn't an assumption. It's all based on precedent. Is there adverse case-law that strikes down testing by a Google-like company for a highly skilled position?

Hiring labor is not exactly identical to hiring a highly skilled algorithm designer. Steve Sailer may not realize it, but I guess the courts aren't all that stupid.

The Detroit Symphony Orchestra used to hold blind auditions. The candidates sat behind a screen so no one could see them.

They were still kicked around for discrimination and was forced to hire a token African American.

Yeah, that was as a condition of continued state funding--kind of like a state school being required to implement the narrowest type of affirmative action possible. Another outcome was that DSO created an African-American Fellowship Program for young people who would probably never have had exposure to classical music. Its grads have joined major orchestras across the country. As a white guy from the Detroit suburbs, I don't think that's a bad thing, considering Detroit is both one the poorest blackest cities and most segregated metro areas in the country.

As a white guy from the Detroit suburbs, I don’t think that’s a bad thing, considering Detroit is both one the poorest blackest cities and most segregated metro areas in the country.

Depends on how you look at it. Why might Detroit be one of the poorest and also segregated cities in America? Well when Coleman Young first stood for election, he pledged to disband the anti-Street Crime unit. Which he didn't actually do because the White liberals who ran the city up to that point panicked and disbanded it first.

So police shootings are up, what?, 2000 percent? How's that Black Power thing working out?

So from where I sit, it looks like the voters of Detroit voted to drive out the White population and for a government determined to take everything of value from White people and give it to Black people. Especially the Black politicians who ran the city. As it turned out. So naturally, the DSO, being one of the last things of value in the city, established and funded by White people, should be forced to hand over as much of value as possible to Black people. The question is whether the politicians have learned anything. How many White musicians can they drive out before the DSO is not worth anything any more?

But if you think the poor segregated people of Detroit are passive victims of circumstances, perhaps you might like to vote to incorporate your suburb in Detroit's city limits?

White people were leaving Detroit in droves the 15 years before Coleman Young become mayor. Their abandonment of the city tipped the balance enough to make Coleman Young electable, which only accelerated white flight. Anyway, one can make Detroit's decline a black power thing, a union thing, a death of manufacturing thing, whatever you like. Some like you think it is 100% racial and nobody is going to change your mind about that anytime soon.

The point is that the DSO is a joke as a Detroit institution if it doesn't even try to reflect the makeup of its hometown or promote classical music among the young people who live there. It might as well be the Oakland County Music Club for Asians and White People, Detroit branch. You may believe that fostering black people's participation in it will ruin the institution, which should be the purview of whites until such time as "the blacks" take it upon themselves to create some musicians worthy of DSO standards. It is a viewpoint very much in line with many of the white baby boomers of the area, but most people are moving beyond that. Younger folks -- those under 40 -- don't see the world through the lens of our parents' black fear.

They were leaving but not in droves until the schools were forced to "desegregate". As you would expect. But you are right - a weak response to the rioting and crime was driving White people out before Young got elected. That hardly contradicts my point.

Well it may be a Black Power thing. I think so. I also think it is a Union thing. Although I note other cities with Unions have not done anywhere near as badly. It is clearly not a death of manufacturing thing. Because places like Pittsburgh also saw a death of manufacturing. But Pittsburgh has been repeatedly voted the most livable city in America. Pittsburgh had Unions too. So what does 66% White Pittsburgh have that 80% Black Detroit does not?

The point about the DSO is that it is a joke if it does not try to maintain standards. It is a joke if it is just an employment boondoggle for the local voters. The way to promote classical music is to play it. Well. If you want it to become another Tammany Hall-style vote bank, well, it will not retain its reputation for long. I believe that rejecting the qualified for the unqualified will ruin the institution. As similar programs have ruined similar institutions everywhere else the idea has been tried.

I think you will find younger people are not constrained by the guilt their parents felt. It is not fear of Blacks that bothered them. It was guilt about Alabama. The young don't feel that. And they are less inclined to accept excuses. But we will see.

"Providers say the tests hold the promise of leveling the playing field for job applicants by removing the chance of bias that comes with a traditional résumé screening. The tests can’t distinguish, for example, if a candidate didn’t attend a top-tier college, is currently unemployed or is a woman or minority."

If only the Fire Department of New York had put its hiring exam online then it would have been spared a $98 million settlement over disparate impact:

http://online.wsj.com/news/articles/SB10001424052702304017604579447270579768720

alternatively it could pay market wages so that there isn't a huge oversupply of applicants, over which they can apply some discretionary choice of rationing procedure (which would, coincidentally, produce a disparate impact).

Volunteer fire departments don't pay any wages, and they are notoriously staffed largely by white male volunteers, often the relatives of other white male volunteer firemen, as Emily Bazelon complained in Slate during the Ricci case. (Bazelon is a "writing fellow" at Yale Law School, which doesn't have anything at all to do with her grandfather David Bazelon being chief judge of the First District Court of Appeals from 1962-1978):

http://isteve.blogspot.com/2009/06/gown-v-town-in-ricci-case.html

eeoc wouldn't care about volunteer fire dept ethnic composition, though, surely

the settlement terms are pretty instructive: the courts hammered out and approve an exam, so the exam process stays. but the settlement forced the FDNY to admit fire academy graduates based on residency in the neighbourhood - it's an elaborate attack on friends-and-family advantages in navigating the application system, albeit under the banner of race, since those seeking the advantages want to keep them too. The exam was merely a bargaining chip, since it couldn't really go away.

Pure racial quotas would be more efficient: just take the highest scorers on the exam by race.

Or take the Bush route - if an out right racial quota is unacceptable, take the top test scorers in each neighborhood. Most neighborhoods are fairly segregated. Most fire services impose a residence requirement. You could use the latter to act as a stalking horse for the former without being too blatant about it.

The cynical would note that simply providing access to a demo version of the ERP (CRM, etc.) system a company uses would allow the company to not even bother to train people on that software, as they search for future employees who already understand the interface willing to work for an 'entry level' position.

I'd like to see the study that shows what if any of this works.

Should they be testing their mid and upper management and just publishing those numbers?

"because the company has found powerful correlations between the online assessments and success on the job"

I keep reading about the difficulty of replicating the results of studies in the academic world. One might expect the commercial world to have similar problems replicating their results over time... but, then, maybe they don't have to. The commercial world is about sales not rigor.

It is much easier to gather a lot of useful data on call center employees than on students that you can use as basis for forecasting...

Those wrist monitors do a great job.

http://www.dilbert.com/strips/comic/2014-05-11/

Nobody here works with ERP/RM software, much less works creating it, do they?

The example from the T-Mobile is something which a couple of SAP Web Dynpros would handle (German only - http://de.wikipedia.org/wiki/Web_Dynpro ). Using a test system for web access, and a dataset sourced from the productive environment, this would be fairly trival. (Though ABAP knowledge is probably required at this point - Java seems on the way out, if the wikipedia article is to be believed.)

And the amusing thing is that people already experienced with such systems are going to score better than beginners - but since experience won't count per se, a company gets to hire more experienced workers at entry level wages.

In other words, just another step on the CEO compensation ladder - but that's fine, since CEOs are never actually subjected to such objective assessment. That would be unfair, as CEOs possess a certain something which is of such rare essence that it cannot be measured by mundane standards as handling an ERP software front end.

It's unlikely to be all that helpful to measure familiarity with a random ERP package when you try to estimate future job performance.

I think you may not be all that familiar with the ERP market regarding large corporations, or T-Mobil.

To put it a bit differently - the world's largest business software company is headquartered in Walldorf.

'* Overall market growth of just 2.2% and the top ten vendors owning 64% of the worldwide ERP market is leading Gartner to predict further consolidation of the industry.

* SAP had just over $6B in total ERP software revenue in 2012, leading the worldwide market with 24.6% market share. Oracle Oracle had $3.12B and Sage, $1.5B in software revenues for 2012. Oracle’s market share was 12.8%, and Sage, 6.3%. The following graphic shows worldwide ERP market share for 2012.' http://www.forbes.com/sites/louiscolumbus/2013/05/12/2013-erp-market-share-update-sap-solidifies-market-leadership/

Do particularly note that prediction about consolidation - though niche ERP applications will continue to exist, the difference between revenue of 6 billion dollars and the following are notable -

'- Workday grew 114.7% in 2012, increasing revenue from $88.6M in 2011 to $190.3M in 2012.

- Cornerstone OnDemand grew 61.5% in 2012, increasing revenue from $58.4M in 2011 to $94.3 in 2012.

- WorkForce Software grew 39.8% in 2012, increasing revenue from $11.8M in 2011 to $16.5M in 2012.

- NetSuite grew 34% in 2012, increasing revenue from $139.7M in 2011 to $187.1M in 2012.'

Ah, the point was on job performance - well, that is exactly what T-Mobil was rating.

And they weren't using a random software package, nor were they likely to be testing applicants on anything but their current system. (Though who knows - corporations aren't rational actors either, and it just could be that the consulting contract for the online assessment system just happened to end up resting in the hands of someone who couldn't even explain what Dynpro means - signed by an executive was equally unware that a couple of developers could crank out such a system in a short period of time, without requiring much in the way of additional equipment or resources.)

The military has used these kind of tests for generations, and validated them against training and job performance. For example, during the 2004 Presidential campaign, I interviewed retired top military psychometricians to understand the officer qualifying exams that George W. Bush and John F. Kerry took in the 1960s to get into the Air Force Reserve and the Navy, respectively:

https://vdare.com/articles/this-just-in-kerrys-iq-likely-lower-than-bushs

On January 17, 1968, Bush took the AFOQT. This AFOQT then consisted of 13 subtests that were aggregated into five composites.

Here are Bush`s percentile scores by composites:

Pilot Aptitude 25th percentile

Navigator Aptitude 50th

Officer Quality 95th

Verbal Aptitude 85th

Quantitative 65th

The percentiles are based on the scores of Air Force Academy candidates during 1955-1960.

This baseline group would appear to be fairly comparable to the Naval OCS applicants against whom Kerry scored at the 50th percentile. How did Bush do? In estimating his IQ, we can probably throw out his high score (the 95th percentile on Officer Quality) and his low score (25th percentile on Pilot Aptitude) because those tests don`t measure IQ very directly. Instead, we should concentrate on his Verbal Aptitude (85th percentile), Quantitative (65th), and Navigator Aptitude (50th). In fact, those three are fairly similar in subject matter to the three parts of the Naval OQT that Kerry took: Verbal Analogies, Arithmetic Reasoning, and Mechanical Comprehension, respectively, on which Kerry scored just slightly above the 50th percentile.

The Officer Quality score was derived by combining Bush`s score on the 60 item Quantitative Aptitude subtest, the 60 item Verbal Aptitude subtest, with the 100 item Officer Biographical Inventory. The latter was a personality test that asked about "past experiences, preferences, and certain personality characteristics related to measures of officer effectiveness." It inquired into enthusiasm for sports and hunting, and was only vaguely correlated with IQ.

(A retired Air Force test psychologist told me that this section was later dropped because women did very poorly on it, and urban and suburban youths didn`t do as well as country boys. "It was politically incorrect, but"—he recalled wistfully—"It was a predictor of success as an officer.")

Judging from his scoring at the highest percentile possible on Officer Quality, Bush must have absolutely nailed the Officer Biographical Inventory test, as you might expect coming from his ultra-competitive family.

In contrast, his not having any flying experience dragged down Bush`s 25th percentile score in "Pilot Aptitude." He would have scored poorly on the Pilot Biographical Inventory and on Aviation Information, two of the seven subtests for this composite. Many of the other subtests focused on three dimensional imagination capacities, such as the "Visualization of Maneuvers" component. These are valuable mental skills, no doubt, but not ones called upon much in the Oval Office.

So, if you take the average of Bush`s percentile scores on the three composites most similar to the test Kerry took, Bush scored at the 67th percentile, a little better than Kerry`s 50th percentile.

When NBC's Tom Brokaw asked Kerry about Bush apparently outscoring him according to my research, Kerry gave a gracious answer on camera. But, according to Brokaw, Kerry returned to the subject after the filming was over and said, "I must have been drinking the night before I took that military aptitude test.”

I was going to say that all this test data suggesting that Kerry's IQ (~115) is probably even lower than Bush's IQ (~120) is ancient history, except that the 70-year-old Kerry is currently the Secretary of State.

And Kerry is performing as Secretary of State like an aged man who earned at Yale a slightly lower GPA at Yale (76 out of 100, or a C in political science) than George W. Bush did (77 in history).

That, Sir, is an insult to C students everywhere!

So, what you are saying is these tests still need some work?

No, the 1960s tests that Bush and Kerry took seem pretty accurate about their cognitive abilities. And yet, the country is stuck with Kerry trying to carry out the cognitively challenging job of secretary of state while he's in his 70s.

I'd expect that these tests because less effective measures in the long-run as applicants learn to game the system. They won't help bring about a "meritocracy" if relatively more privileged/less "meritous" applicants become able to receive application test coaching similar to the present situation with e.g. SAT prep courses.

The SAT and similar tests are pretty much a known quantity. Those employment tests are less well specified and easier to change in reaction to test teaching...

Good point, but I'm not sure this eliminates the concern entirely. These tests are only useful if you can be sure they are correlated with job performance, and to be sure of that you need to do a study like the ones mentioned in the original post. Those studies are costly, and I'm not sure I expect employers to perpetually face those costs and redesign the tests in response to interview prepping.

The tech industry has been doing this for some time, and you're seeing books like http://www.amazon.com/Cracking-Coding-Interview-Programming-Questions/dp/098478280X popping up that are _awfully_ reminiscient of SAT prep materials.

How do they know that the applicant actually took the test himself, unaided? Do they care?

Theoretically, they could have the PC's camera record the face of the test taker taking the test.

How naive!

It's called ProctorU (http://www.proctoru.com/), and it's being used by colleges across the country in an increasing number of online courses to do their midterms and final exams.

Wonder how it'd deal with virtual web cams. Essentially for anyone decently nerdy it's not very hard to spoof a webcam video stream. Cheating shouldn't be too hard.

Is there a reason not to care? Some times there is.

http://en.wikipedia.org/wiki/Michael_Oher

Oher eventually increased his 0.76 grade point average (GPA) to a 2.52 GPA by the end of his senior year so he could attend a Div. I school by enrolling in some 10-day-long internet-based courses from Brigham Young University. Taking and passing the internet courses allowed him to replace Ds and Fs earned in earlier school classes, such as English, with As earned via the internet.[6] This finally raised his graduating GPA over the required minimum.[1]

Maybe they think it is a good sign of an enterprising character? The German Army used to like officer candidates who had been forced to repeat the odd subject and had a slight discipline problem.

Now he is a miilionaire, so I certainly hope he cheated. If we really want student athletes we could do that, but it wouldn't be by doing more of the same.

Considering the actual purpose of these tests in normal work is probably to avoid the total disasters, identification may be the most important aspect.

He is a millionaire *now*. But for how much longer? NFL players have a very poor record of financial management. Especially if they are not White.

I don't know if he cheated or not, but improvement at that rate makes Flo Jo's improvement look pathetic.

College football has long been a farce. But there is still some idea that the players ought to be students. That they are not there just to play football but that they are students who also play sport. Obviously that is not the case in places like North Carolina. Nor some others I could name.

The purpose of these tests is clearly to do what IQ tests used to - find the motivated and smart. I think they will find that the Courts will not tolerate them for long. Although America may have turned a corner on disparate impact. It would be sensible to try it again in a few years.

Comparison with Flo Jo is not apt. She had been training for years before the dramatic improvement for high level competitor. Michael Oher's improvement is off a base where he didn't care.

Speaking of the German Army . . .

http://straighttogo.com/clever-and-lazy/

I divide my officers into four groups. There are clever, diligent, stupid, and lazy officers. Usually two characteristics are combined. Some are clever and diligent — their place is the General Staff. The next lot are stupid and lazy — they make up 90 percent of every army and are suited to routine duties. Anyone who is both clever and lazy is qualified for the highest leadership duties, because he possesses the intellectual clarity and the composure necessary for difficult decisions. One must beware of anyone who is stupid and diligent — he must not be entrusted with any responsibility because he will always cause only mischief.

Best comment on the thread.

+1

Reminds me of the quip that the two key virtues for a good sysadmin are laziness & hubris.

And yet Obama is lazy and, just ask him, clever - without any sign that he possesses the intellectual clarity and the composure necessary for difficult decisions.

For a programming test, they can ask followup questions in an interview ie. why did you do it this way? How else could you do it?

It's still possible to game the system, but you can separate the clueless from the clueful.

this is just more evidence employers, in an attempt to increase productivity and realization of the uselessness GPAs, are instead using IQ-type tests that more accurately predict employee performance, such as the SAT, wonderlick, or custom apps.

More evidence to me that nobody knows anything. But almost everything i see is these days. YMMV.

I have never seen the use of 'on-line' anywhere, ever. Its 'online'. Except in this instance it isn't.

The writer in the Washington Post article means 'computer based assessment'. This is > 20 years old. And the overheated marketing hooks you breathlessly copy/paste make it read like its 1994.

And this gets your heart rate up?

Codility is a system for evaluating programmers that has been around for a while. It has various flaws, but does a good job in separating the clueless from the clueful.

One time a company required that I do Codility's test. I spent a few hrs doing practice tests and then the actual test itself.

Eventually the company told me that I passed the test but that they had no openings that matched my seniority.

Do they work? A lot of coders seem to not have learned it by degree so I can see the motivation.

Hiring software developers is still incredibly broken. Most people are using the same technique they would have done in the 1980s.

Instead, most teams create a hiring process that prioritizes finding clones of themselves.

Yet here we are with lessening business dynamism as outlined in the very next post. Why have all the increases in so-called meritocracy been so very fruitless for the economy as a whole? Are the countervailing forces of technological stagnation put forth by our host just that powerful?

Women and minorities free to enter the general workforce/educational framework, lack of capital controls, ever freer trade, the weakening of unions, lowering of most taxes, etc. Management science generally, Big Data such as that in this post, at-will employment law, ever higher educational spending and attainment, the Flynn effect, etc. Why is none of this making much, if any, difference?

Is it IP laws that severally hinder economic competition and therefor growth? Perhaps if Social Security was eliminated the US could then claim increasing or simply steady dynamism and innovation? What the heck is going on?

I do not drop a great deal of comments, but after reading through
a ton of responses on The growth of on-line tests in assessment (average is over).

I actually do have some questions for you if it's
allright. Is it only me or do a few of these responses come
across like they are left by brain dead individuals?
:-P And, if you are posting at other online social sites, I'd like to keep up with anything new you have to post.
Would you list of every one of your public sites like
your twitter feed, Facebook page or linkedin profile?

Hi there! I'm at work browsing your blog from my new iphone!
Just wanted to say I love reading your blog and look forward
to all your posts! Carry on the outstanding work!

Comments for this post are closed