Does signaling also help you to do better?

That is the conclusion from a new paper by Rebecca Diamond and Peta Persson (pdf), on Swedish data, here is part of the abstract:

Despite the fact that test score manipulation [by teachers] does not, per se, raise human capital, it has far-reaching consequences for the beneficiaries, raising their grades in future classes, high school graduation rates, and college initiation rates; lowering teen birth rates; and raising earnings at age 23. The mechanism at play suggests important dynamic complementarities: Getting a higher grade on the test serves as an immediate signaling mechanism within the educational system, motivating students and potentially teachers; this, in turn, raises human capital; and the combination of higher effort and higher human capital ultimately generates substantial labor market gains. This highlights that a higher grade may not primarily have a signaling value in the labor market, but within the educational system itself.

Again, the result is that “encouragement effects,” or alternatively “writing off effects,” are stronger than many of us might think.  Tell people enough times that they are a certain way, and eventually they will start to believe you.  I would say this is evidence for my “beasts into men” theory of education, though other interpretations are not ruled out.

For the pointer I thank Ben Southwood.

Comments

Okay, but this appears to be related to teachers inflating the scores of decent students who scored lower than their classwork would suggest.

"This highlights that a higher grade may not primarily have a signaling value in the labor market, but within the educational system itself."

This does not surprise me in the least.

What this paper says to me, at first look, is that teachers like kids who are good at school and kids who are good at school have better outcomes. Is this supposed to be supportive of education or damning?

> [T]est score manipulation has far-reaching consequences for the beneficiaries... raising earnings at age 23. Getting a higher grade on the test serves as an immediate signaling mechanism... in turn, rais[ing] human capital.

I don't see how they're demonstrating the case that the manipulation beneficiaries actually have higher human capital. Yes, they may have higher long-run earnings, but how do we know that's not simply from the long-run effects of signaling. Get a better test score, use that to get biased grades in high school, use that to get accepted at a more prestigious college, use that to get access to better jobs, then get paid more. At no point in that hypothetical did the subject actually develop more human capital. It's quite possible, even likely, that they simply kept leveraging each step's signaling boost into another signaling advantaged position.

Isn't this a little like the results showing that workers who graduate in recessions have lower earnings decades later. It's not that they're actually less competent. It's just that one low-prestige job leads to another, ad infinitum.

The recession year graduates analogy seems very instructive.

It would also explain at least some of the serial academic underachievements of certain segments of society - start off just a little behind because your parents were less inclined/able to support your academic progress, and the effects could rapidly multiply over the course of 12-20 years in education.

Yes, I am sure that a few hundred years from now, humanity will look back and realize how important the discovering of signaling was.

It's a better world when everyone is a winner.

What happened to the Up Side of Down? Does this mean that a little assistance provides an inducement to do better in life rather than become a lackey? What's next, unemployment benefits induce workers to find a job! Who knew?

Signaling may not be what's happening here; rather, it's reinforced self-confidence (or delusion, take your pick). If someone is repeatedly given reinforcement that she is smart, she will come to believe it. Hence, "smart" people who work together at a think tank provide mutual reinforcement of just how smart they are (especially compared to those "dumb" people at the other think tank). Of course, this sometimes leads to arrogance, which is fine as long as those "smart" people aren't put in a position of power. If they are, we all pay a high price for their smartness. It may seem counter-intuitive, but reinforced self-confidence closes the mind to new ideas, new ways to approach a problem. That's why "smart" people in a communist country, just like the "smart" people at the think tank, repeat the same mistakes again and again. Delusion is a powerful force, especially when everyone around suffers from it.

I think the reinforced self confidence thing can be a huge deal for at least some students.

While there are legitimately dumb students and others with a variety of learning disabilities, essentially "normal" students can get behind for all sorts of reasons - perhaps they had a bad teacher for a year or two or just didn't try in the early years. Thinking that they're just not good at something, they see little hope of doing well and just don't try very hard (circular). Also, it's not very interesting to pay attention when you're completely lost 90% of the time. In trying to reach these students, I apply something I learned from a post-colonial theory professor (a much hated subject by the anti-SJ Warriors). She would say basically "if you're confused, this means you're starting to learn something - if you're not confused at least a little bit, then you're too overconfident and probably misunderstand completely". Now, please ignore the ways that one might fantacize that this was all to lead us woefully astray in some manipulative way - it was absolutely critical to being able to enter into such a completely foreign land of concepts enroute to credibly engaging with the perspectives and critiques available in the field.

Here's how I reapply it. I tell students not to worry if they don't fully understand things, and in fact that I fully expect most of them to be at least a little confused the first time the see new material. Rather, do not allow the sense of confusion to cause you to lose focus, and to apply all their senses maximally to try to establish whatever amount of meaning they can glean from it: listen to me speak, connect this with words and pictures in the book, read or follow along in your head in your internal voice - then make note of anything that you don't understand but which you misunderstand concretely enough to ask questions.

Listen, look, think with me, allow your brain to naturally step in and create the incomplete links, just prioritize staying focused and on task and do not be deterred by any sense of confusion - there will always be time to ask questions soon after (and I award them with meaningless points for asking such questions as well). I repeat it at least once or twice before entering into any new material that I know will be challenging, staring down the students I know it's most relevant to (the ones who are (were) genuinely lost). I would estimate that it makes a HUGE difference for about 50% of students, with especially outsized effects on students who are basically smart but were previously deterred by being behind - they are embracing comfort with the unnkown and accepting that incomplete comprehension is vastly superior to zero comprehension.

Lol. I also encourage students to be aware of teaching methods and to please inform me if they think we're ever covering anything useless. The only input I've had so far is from one student who came to class the next day, informing me that his grandfather threw out his textbook, having called it useless. He's now on a self-guided independent learning track, doing whatever he wants in class as long as it's roughly in line with the subject ...

There's hope! Thanks for your service (no, I'm not being ironic). I have a thing about schools and boys, the latter suffering failure rates far higher than girls, the consequences of which will be felt for years to come not only by them but everyone in the communities where they reside. Schools are failing boys. Having spent lots of time with boys teaching them baseball, I learned more than they did. Not about baseball, but about boys and how they learn, or don't learn. People learn in one of three ways: listening, seeing, and doing. For many boys, they learn little or nothing from listening, only a little bit by seeing, and mostly by doing, the latter my grandmother called learning the "hard way". Yet, in most schools the primary way of teaching is by the listening method (i.e., by lecturing). The teacher might as well be speaking in Mandarin. For the boys, it's reinforced failure. Reinforced success, the subject of Cowen's post, can be equally as negative. My friend's daughter attended their state's flagship university her freshman year, she and her parents drawn to the school because of the state's scholarship program that provides tuition-free education for top high school students. Indeed, many of the students at the flagship university are there on the same scholarship program. At the end of her freshman she, she told her parents she would continue at the flagship university if her parents wanted, but she wanted them to know she could make straight As and never open a book because the students weren't being challenged - everyone already knew these students were "smart" and all of them deserved As. She transferred to another school where she has been challenged.

Behavioural problems are definitely a much bigger deal among boys (duh). To a number, all who remain completely impermeable to the above strategy are boys. I managed to reach a couple of them with very adult discussions of "this is your habit, this is what you're being like, I understand reasons that it seems like a sensible thing to you even if you haven't thought it through, but please think it through, because it you cannot take it upon yourself to find the mental willpower to break this habit then you're going to have a lot of problems in life."

However, most boys naturally hear "blah blah blah blah blah blah blah" when such lectures start. It is only my third year as a teacher and I don't have much of any formal training ... in time I guess you gather up all sorts of little tricks that suit your personality well.

Out of the more disruptive ones, I managed to crack another by pleading "if you don't want to try, don't care, don't have any interest, I cannot force you to want to try or to be interested in things you don't want to do, but please, since you seem to have so much interest in your friends, consider that every time you disrupt them that you are making this choice for them - it is not fair that you take away their choice of whether or not they want to try by disrupting them so often." Once he's not disrupting constantly, at least he's occasionally paying half attention.

I think the biggest problem with the remaining subgroup is that the implicit (not intentional) attitude is "whatever you say, I'm not going to do it." Yeah, like, I get that - if you don't explain to me WHY, I don't do it, I don't care if you're the King of England or my mother, and I need a reason. But these kids are impermeable to reason (just immature?). I try to reason with them saying "hey, looks like you want a lot of independence. If you want independence in the future, you're going to have succeed in school first, or your options will be limited. We're not trying to control you because we want power over you or anything like that, we're trying to get you to learn stuff. If you can accept doing what we ask, you will have more freedom in the future". I think that cracked 1 of the remaining 6-7 or so seemingly hopeless cases. It's basically impossible to figure out if the remaining ones are dumb, because whatever the task is, they're just flat out not going to do it. Not bad out of 850-odd students.

A few ideas cross my mind of how to deal with the uncrackables, but man, I'm paid to show up and not much more. A few hours a week extra maybe? Even the best intentioned people eventually reach a point where you say "I already exceed expectations, and it is not reasonable to expect me to go the whole way, because it's just plain and simply not my job".

"For many boys, they learn little or nothing from listening, only a little bit by seeing, and mostly by doing, the latter my grandmother called learning the 'hard way'."
Come on.
How boys used to learn Reading, Spelling, Geography, History, Religious Doctrine (when it used to be taught) and Multiplication Tables before girls-- like the Barbarians, I guess-- invaded the classroom? By doing? There is not wrong with students, boys and girls alike, that a good whip can't fix.

Are you an ESL teacher?

Because that is a very important point to make to language students, especially if they are from a culture where making errors is considered embarrassing. To learn, you have to be willing to fail or not quite achieve 100% comprehension.

Roughly speaking, you can divide teachers into two groups in a sense. There are teachers who place all the worst/troubled students right under their nose, and there are teachers who shuggle them off into the back corner where they can be ignored more easily.

The effect of signalling with the education system in terms of getting access to better education opportunities down the road may differ significantly depending on the extent to which one or the other of such approaches are encouraged and/or tolerated within the education system (assuming a non-streamed trajectory). Some teachers will spend more effort trying to bring up the students who are behind, some teachers will focus their efforts on those already at the top of the class. Depending on the needs of the economy, such as a need to satisfy lanour market requirements for a very scare highly skilled class or to improve the skillsets of the working class, either might make sense.

It's worth trying harder when you don't think you're going nowhere. Consider entering 4th year of a BA with a 2.0 GPA - there are zero prospects of making it into grad school, so there's not much of any point in aiming for anything more than a pass. But if you already had high marks in previous years, the payoff of one more year of high effort can be quite high.

In China, there is basically zero teacher discretion in test scores which matter for getting to the next level (absent which practice there would be widespread buying of higher grades). There seems to be virtually zero discussion of the downsides of teaching to the test as the main feature of teaching. Due to the use of standardized tests in university admissions, you have a situation where students spend 10-12 years studying a foreign language which perhaps only 1-2% will actually ever use for practical purposes (a few percent more when considering that it's useful on holidays), and students who are brilliant in all other subjects will see their prospects significanly diminished if they are unable to score well in just this one area.

There is a definitely different challenge in teaching when you're teaching a subject that unambiguously does not show up on the standardized tests. It is critical to motivate the students by selling them on the utility of learned knowledge in the post-education real world. Interestingly, most still seem to respond to incentives in the form of "I will reward this kind of study habit, effort or result with higher grades" even though, if they think it through, the grade they receive cannot possibly have any impact on any future outcome whatsoever. I award one point for each instance of participation, whether asking a question or providing answers in class (unless they are making identical points to others or asking about stuff that we already studied, but if they don't know the answer they don't find out it's already ben studied until after they ask the question). Despite the fact that the grade I give them is utterly irrelevant to anything, participation in the classroom went from a core of 5 or so eager beavers in each class to several dozen regular participators in each class. In fact, I'm now so innundated with participation that I have to be very careful to not be biased in who I select, and have to strive to be aware of any possible biases which might lead me to select certain students over others.

Maybe it's the parents. It's really different to inflate the grades of children with down-to-Earth parents compared with inflating the grades of children with "my child is perfect" parents. The first may make a better life for children under stress, the latter is just increasing confidence in an empty head.

This snippet is short sighted.

If a test is appropriately a gauge of knowledge, skills, or abilities, and inflated score gives a false sense of confidence and a false signal. Such persons may very well seek opportunities above what they might otherwise choose, but they were more suited for those other opportunities. Going to college more often begs the question that it is unambiguously preferable to do so. College initiation stats ignore all the costs of struggling to learn and graduate, to teach them, to get a job, to perform well in that job, and for employers to deal with the effects of a bad signal.

How the heck do serious economists ignore such obvious inefficiencies and unintended consequences? How do they fail to measure outcomes instead of intermediate effects? I suppose in a world where we place a person's self esteem above their actual value to society, boosting ostensibly objective and informative measures of abilities or opportunity costs indicates everything is right with the world.

Now give me my damned participation trophy.

Affirmative action is signaling, too. You poor victim; we'll just push you up to the head of the line. See how we fixed that?

It seems like economics has degenerated into a beauty contest of job papers.

Yes, affirmative action does the same.

I realize I was incoherent in my last post. Pre-coffee.

I meant to say that regardless of whether educational performance is a measure of skills or signaling (two different theories) grade inflation unambiguously reduces the value of the information. When there is less variation in a dependent variable, the standard error of the estimator for the independent variable is higher, all else equal.

As far as incentive effects, regardless of whether it is a measure of ability or signaling, the student gets less informative feedback which unambiguously increases the likelihood of job/school mismatch. Mismatch not only hurts the student and the college, but other students.

Participation trophies are definitely an example of such thinking going over the deep end. The trophies become meaningless, and the child may as well just put up a picture on the wall which shows that they had fun playing the sport.

But I think understanding the role of confidence in motivating students is important. Leaving a lot of discretion to the teacher can enable them to use subjective judgment in trying to use marks to motivate students in different ways. For example, I had the same teacher from grades 4-6. I was a pretty good student, but he knew I could do even better, so even though I was at/near the top of the class and students often lined up at my desk for help when the teacher was busy, he always gave me Bs with a smattering of As. One of my friends who was a creative type but struggled a LOT in most subjects also got mostly Bs, but with a smattering of As. This wan't just a lazy teacher giving out mostly Bs to everyone. There were lots of instances of other students who got mostly As and lots of Cs/Ds. He was clearly using grades strategically to try to motivate students at both ends of the spectrum, trying to motivate top students to do EVEN better, and to give confidence to struggling students. I think the strategy is generally valid, especially in lower grades, but in my case it backfired - why try if I get the same grades as the student who always get some of the lowest marks on all the tests? So I read novels in class for most of those years, and fortunately, the teacher didn't see any value in forcing me to go through all the motions of things I already knew 2 minutes into the lesson.

I think this is a slightly different issue from the role that grades play in formally evaluated applications, such as high school grades mattering for university applications. But I'm mostly focused on the matter of confidence and motivation here.

Motivation does different things for the individual within their development curve. It *can* be a well timed push.

If you think of all human accomplishments in terms of the Logistic Function, as servicably described here: http://earlyretirementextreme.com/better-goal-setting-with-s-curves.html (which is a useful way of thinking about it), encouragement can either be leverage, at the early stage, or noise in the feedback, at stages after that.

It's like ether. You can start a diesel engine with it, but you can run on it. The engine has different dynamics at different stages. So do people.

There's another interpretation of these results which nobody seems to have noticed: contemporary education is so useless that a student's performance in school is completely unrelated to how capable or intelligent they are.

Indeed. An expensive day care with chocolate bars for the good kids.

I deal with people as described all the time. The signalling puts them into positions where what they do really doesn't matter that much but are well paid. The perfect bureaucrats.

The rule in the government department where I worked a couple decades ago was 'It really doesn't matter. If it does, it really doesn't'.

Otherwise known as a "just so story".

Could we tell the difference between transfer effects (this student is better off, but students with actual high ability are all slightly worse off) and Pareto improvements with this study design?

The contradictions here seem similar to those involved in the use of placebos in medicine.

Practically everyone understands that placebos really do work work, not as well as real medicine perhaps but a whole lot better than nothing. Yet effectively implementing placebo-therapy is challenging, due to the need to maintain the deception that the placebos are not, in fact, placebos.

Bingo! Well stated!

An interesting observation. I wonder if part of the effect is sorting -- that there is a more or less fixed set of slots at the output end of the educational system, and the slots (in modern times) have much the same inclusive fitness. So the school sorts the students into the slots, but since the students aren't strongly motivated to get into a higher slot, they go along with however they are evaluated by the teachers.

The difficulty, of course, with signaling and the idea that it produces real improvements in a student's ability is that our colleges and universities are packed with young adults who have been told all their lives how special they are, how wonderful they are, how competent they are yet despite this life-long signaling they are academically unprepared for college-level work. While the study may show all sorts of wonderful things, the larger body of data from real life disproves the claim. Rather like the professional football star who just signed up for a $50+ million contract saying that hard work and effort do not make up for disadvantaged childhoods and that the American Dream is a fraud; his life disproves his claims. I suspect something of the same here. And, just for those of you who might denounce me as racist, the students to whom I refer in the opening of this comment are nearly uniformly white, middle and upper middle class, with all the other advantages and privileges that implies. The failure to require that they learn to an objective standard of performance deprives them of the actual ability and capacity for college work, no matter how many times their self-esteem has been boosted without regard to their true performance. This is just more of the same: everybody gets a trophy.

faking signals is an old idea. With complicated side effects.

One can theorize how much lying can still be beneficial. But a significant amount of pretending might be counter productive. That is for society as a whole. Because of the cummulative lose of signal value.

There are many other ways to fake parts of the process. For example, psychological theories etc. on self efficacy (Albert Bandura). I am not aware on anybody seriously tackling the issue of the lose of signal value

The wizard or Oz effect again.

My comment has already been made in different forms by other commenters, but surely it is clear that there is a dynamic here that is similar to the attempts by governments to manipulate the economy back in the 1960s and 1970s? At first, a small increase in pay unrelated to productivity--the workplace equivalent of getting an A- when your work is really only worth a B+--can make the worker feel more prosperous, with all the beneficial knock-on effects that may have.

But presently, people notice that their bigger paychecks really aren't buying any more than they used to. So they price the increases into their thinking, and before too long, you wind up with runaway inflation coupled with high unemployment--the state of play when I was in college in the late 1970s.

At that point, you have so seriously debased the currency that only draconian measures can rectify the situation. Similarly, in education you wind up observing that anything below a B is a bad grade, presently it evolves so only an A is acceptable, and then...well, what? You wind up with increasingly arcane and indirect measures such as Standards of Learning and that sort of thing.

Nor is this limited to academia: when I worked for the Government, we had a five-point scale for annual performance appraisals: Outstanding, Highly Satisfactory, Satisfactory, Marginally Satisfactory, and Unsatisfactory. But in practice, no one was ever rated below Satisfactory, and those who were saw it as a mark against them--I had one employee submit a grievance for getting a Satisfactory rating! Most employees got a Highly Sat--which thereby became the new Sat--and a few got Outstandings.

"Signaling" is a fine thing until people decipher the signal.

Yes, there have been several articulations, with various analogies, in this thread of the observation that 'signalling as a phenomenon is non-linear and highly complex'.

It is nice to see so many people make the same observation in their native framing; there is probably a 'there' there.

In putting it short way, I believe education is key to a lot of things, if we have education then we can really work things out, if we are lacking on there then we will face trouble. I mainly do Forex trading and this is a sort of business where will be seriously effective if we have education and experience, I work things nicely with help of quality broker like OctaFX, it really helps me work well and I am always getting great benefits in terms of making profits.

Comments for this post are closed