Dept. of pure coincidence

by on April 15, 2014 at 11:32 am in Medicine, Political Science, Religion, Science | Permalink

The Census Bureau, the authoritative source of health insurance data for more than three decades, is changing its annual survey so thoroughly that it will be difficult to measure the effects of President Obama’s health care law in the next report, due this fall, census officials said.

The changes are intended to improve the accuracy of the survey, being conducted this month in interviews with tens of thousands of households around the country. But the new questions are so different that the findings will not be comparable, the officials said.

An internal Census Bureau document said that the new questionnaire included a “total revision to health insurance questions” and, in a test last year, produced lower estimates of the uninsured. Thus, officials said, it will be difficult to say how much of any change is attributable to the Affordable Care Act and how much to the use of a new survey instrument.

“We are expecting much lower numbers just because of the questions and how they are asked,” said Brett J. O’Hara, chief of the health statistics branch at the Census Bureau.

With the new questions, “it is likely that the Census Bureau will decide that there is a break in series for the health insurance estimates,” says another agency document describing the changes. This “break in trend” will complicate efforts to trace the impact of the Affordable Care Act, it said.

Obviously with a big new law you need new questions too, I suppose, plus the old questions ought not to hang around.  You can read more here.

As a side note, I have been reading far too many blog posts about “numbers enrolled” as a metric of success for Obamacare.  That has never been a good test of the serious criticisms (and defenses) of ACA.

I thank Megan and Garett for the pointers.

Addendum: You should read this update from Vox, though I am not satisfied with the Administration’s response.

andrew' April 15, 2014 at 11:35 am

They don’t know basic quality control…well they do if course.

“You can always sell insurance”
Warren Buffett and Jesus.

Florian April 15, 2014 at 11:37 am

The problem could easily be solved by a 1 year overlap of old and new survey.

Yancey Ward April 15, 2014 at 11:40 am

Yes, but it would undermine the main incentive for making this change now.

mulp April 15, 2014 at 6:08 pm

Yes, with the budget available, they could only do one survey, and you would want the old one which would prevent getting data to answer the dozens of question unanswered by the old survey.

Unless you are advocating reckless deficit spending. Or tax hikes…

David C April 15, 2014 at 12:05 pm

Areed. I work in the earth science field and when we launch a new satellite that gets better/different data, we keep that one and the one it replaces working for at least a year so that we can calibrate for the new instrumentation. Terrible.

jdm April 15, 2014 at 1:01 pm

Yeah, I don’t think people put enough weight on the importance of continuity in times series data. To give one small example, in 2011 the USGS moved a river gauge near our house about 400′ upsteam of its previous location. It’s not clear to me why they did this (they may have had good reasons), but the effect is that flood elevations and river discharge, which had been recorded and estimated since the 1940s, are no longer directly comparable before and after the change. On the NOAA website that has this gauge’s info, there is a footnote about the change with a very rough (flood elevation dependent) correction factor (the correctness of which I have reason to doubt). The USGS didn’t run the two gauges simultaneously over a period of time (to get a good calibration, which they absolutely should have done, as you indicate) and NOAA makes no attempt on their website to normalize the data to account for this change (for example, the show that 2013 had a record flood stage of 11.4 feet in the same table where they also show past flood stages from the old gauge location; if you don’t read the footnote you would have no idea that the numbers are not directly comparable, and even if you do read the footnote, it’s not obvious what the correction should be). So now all that historical data, collected over seven decades, is much less useful than it should be. Obviously things change, but there should be a high premium trying to ensure some kind of continuity in the data.

mulp April 15, 2014 at 6:13 pm

But Congress has been cutting budgets so the satellites that are doing the monitoring are going dead before the replacements are even available for launch.

David C April 16, 2014 at 9:18 pm

Depends on the program. One instrument that I worked on was called SAGE II and it operated for 21 years, which made it immune to budget cuts. SAGE III was added to the Space Station with plenty of time for overlap. That satellite, ERBS, was a tough old bird and it finally died, not from hardware failure, but because it’s data became so obsolete that no one would pay the $100,000 a year it cost to operate it. That’s a real bargain, made possible by getting college students to run it for free, but even that wasn’t cheap enough. So NASA dumped it in the ocean. I often say that the goal is to keep a satellite up and working until you run out of propellant (since there’s nothing that can be done about that), but the way ERBS went was even better. No one had any use for data gathered from instruments designed in the late 1970’s, when there were much better instruments providing the same information.

Data User April 15, 2014 at 2:18 pm

Unsurprisingly, govt. statisticians forsaw this potential issue and are years ahead of the internet commentariat.

Two sources, from 30 seconds on Google:

http://www.academyhealth.org/files/2012SHRIPGARMTurnerFinal.pdf

http://www.census.gov/hhes/www/hlthins/publications/sehsd_wp_2014-16.pdf

ad*m April 15, 2014 at 3:37 pm

I read the papers and that you bring these up just shows you and they dont understand what this is about. Yes, if you change your instrument you get different measurements, and yes you estimate these differences by looking at small samples with both instruments.

But government statisticians should avoid even the impression that they are cooking the data changing the instrument at the exact time such a controversial law, passed on party line, has been put into effect. And Census Bureau bureaucrats should especially avoid this in this time, when other government bureaus have been found to be staffed by highly partisan, tipping the scale, public employees. For example from many, a recent example at the IRS in Dallas
http://dfw.cbslocal.com/2014/04/10/allegations-of-impropriety-inside-dallas-irs-office/

So, many people, especially those who were not in favor of Obamacare in the first place do not place much trust in the employees at federal agencies such as the IRS, BLM, ATF. And this step will just make that worse.

mulp April 15, 2014 at 6:16 pm

Clearly the Bush administration should have spent more money updating the surveys to answer the dozens or hundreds of questions that don’t have good answers for any of the debate issues since 2007 on health policy reform.

Michael April 15, 2014 at 5:39 pm

If they were actually interested in honest reporting, then yes.

Yancey Ward April 15, 2014 at 11:39 am

I actually think the changes will provide more accurate data on the numbers of people who are and are not insured, but it is only now that an administration has an incentive to make the changes required in the surveys.

Jan April 15, 2014 at 11:48 am

One of the main reasons for this change is precisely because of the new ways people can get insurance now with the exchanges in play.

I guess you can just say this is politically motivated, but if you don’t have some evidence of that only people who are in the anti-ACA and anti-Obama camp will care. So have at it.

Yancey Ward April 15, 2014 at 12:00 pm

Jan,

Nothing has fundamentally changed about how people have or don’t have insurance. They still get it in the same ways- by being someone’s plan, having a plan from an employer, being on government insurance, or by buying it as an individual.
The surveys in the past over-estimated the numbers of the truly uninsured by not really differentiating between people who were uninsured the entire year and who were uninsured during a transition. This has been known for a long time, but is being fixed to some extent at precisely a time an administration needs to show a big decrease in the uninsured. One can describe this as a pure coincidence, but only the foolish believe this as an initial position.

Yancey Ward April 15, 2014 at 12:12 pm

You wrote:

One of the main reasons for this change is precisely because of the new ways people can get insurance now with the exchanges in play

Utterly untrue. The ways of getting/having insurance are the same as they were before- be on Medicare, Medicaid, get it through an employer, or buy it on the individual market. And the old questions and their flaws would not have been changed by the ACA, and the reasons for making the changes today are no different than they would have been 5 years ago. The only fundamental thing that has changed is that an even lower % of uninsured will serve a political end for the administration.

Nick April 15, 2014 at 12:17 pm

In the past, having a higher % of uninsured served the political benefit of people who wanted health care reform.

Jan April 15, 2014 at 12:38 pm

Read the article — relevant pieces below. The distinction between exchange insurance and non-exchange insurance as well as between subsidized and non-subsidized plans is much of what they are focusing on. That was a not a factor in the past.

And, yes, they also very badly needed to fix the point-in-time issue to improve accuracy, which you agree with. I am not opposed to doing a one-year overlap of old and new surveys, but it is stupid to not improve the methodology right now that they have an instrument that has been extensively tested and now that the health care landscape is much different than in years past.

The new survey asks people if they have coverage through an exchange, if it has premiums and if the premiums are subsidized.

People generally know if they have health insurance, but not necessarily the type of coverage. A study by the Census Bureau said that the line between public and private coverage is blurry.

—–

The new survey asks people if they have coverage through an exchange, if it has premiums and if the premiums are subsidized.

People generally know if they have health insurance, but not necessarily the type of coverage. A study by the Census Bureau said that the line between public and private coverage is blurry.

“The same exact coverage will be construed as private by some and public by others,” it said.

Census Bureau research in Massachusetts found that consumers “inevitably conflate Medicaid and the subsidized exchange.” And many people with subsidized private insurance, purchased on the exchange, said they were receiving coverage from the government or the state.

Such perceptions are understandable. “Exchange coverage is a hybrid, partly private and partly government,” said Joanne Pascale, a Census Bureau researcher who helped develop the new questionnaire.

mulp April 15, 2014 at 6:26 pm

So, today the ability to buy insurance on the private individual market through the exchange is exactly the same today as buying private individual insurance after losing your job because of illness or job cuts or the firm going bankrupt and then losing your employer insurance for the entire family because in every State in 2009 the individual market was community rating with no consideration of pre-existing medical conditions?

Of course, the solution offered to anyone with an illness and no insurance was to get a job any job at a big employer with health benefits dictated by the jack boot IRS code and Washington government officials.

andrew' April 15, 2014 at 12:26 pm

Jan, that camp is apparently way bigger than you think. And do you think it is static or maybe gets bigger as Obama does more wrong things?

Jan April 15, 2014 at 12:39 pm

Has ACA approval has been going up? How do you feel about that?

Rich Berger April 15, 2014 at 1:23 pm

No. How do you feel about that?

Jan April 15, 2014 at 1:30 pm
Rich Berger April 15, 2014 at 2:09 pm

Yes, I should have known. You picked the one outlier – most other polls show consistent, persistent disapproval of Obamacare (which was conveniently not named in the poll you cite). Obamacare was hatched in secrecy, nurtured by palpable lies (if you like your doctor, etc.), and illegally modified to defer pain until after the elections. Note that the Democrats chances of retaining the Senate are slipping and the Democrats are doing their best to avoid talking Obamacare by distractions such as the Koch brothers, pay inequality and minimum wage.

I realize that you will not be swayed by any such observations.

Jan April 15, 2014 at 2:42 pm

Ok, so you don’t believe that there was a recent uptick in approval? It is also going up in the Gallup poll — the strong finish to exchange enrollments and the awareness campaigns are what is driving the delta here.
http://www.gallup.com/poll/168491/americans-remain-negative-healthcare-law.aspx

You many not agree that millions of people gaining access to insurance and increasing approval of the policies that made that happen are good, I’m just pointing that out. I think the fact that so many Dem Senate seats are up for election in moderate states, the fact that many of the D incumbents are retiring, and that presidential parties tend to lose ground in mid-terms has more to do with the Senate outlook than the ACA, but ok sure.

dead serious April 15, 2014 at 4:56 pm

“…the Democrats are doing their best to avoid talking Obamacare by distractions such as the Koch brothers, pay inequality and minimum wage.”

Translation: anything I’m okay with that other people don’t like is a distraction. Waaaah.

mulp April 15, 2014 at 6:32 pm

Its clear that conservatives would have preferred single payer because:

single payer would have required no mandates

single payer would not have required complicated signups

single payer would not have required paying profits to private insurers

single payer would not require paying premiums for stuff you didn’t want

single payer would have covered 100% of Americans instantly

Z April 15, 2014 at 12:55 pm

I’ve said for years that the number of people in need of private insurance, but without it was a fraction of the claims. Even the bogus numbers coming from the Cult bear this out. Once you net out illegal aliens and poor people who should be on the dole, but were not for some reason, the remaining number was tiny.

ObamaCare was mostly passed out of spite. The other big reason was to further erode private arrangements of the middle-class. The Cult wants full-blown single payer socialism for everyone except the elite, of course. That’s the theme everywhere. The Brazilification of America is moving forward no matter what anyone has to say about it.

Jan April 15, 2014 at 1:04 pm

Then everybody will be disappointed. You, because it is working and harms your “Cult” narrative, whatever that is. Liberals, because it is working and doesn’t effectively push us toward single payer.

Jay April 15, 2014 at 1:45 pm

Define working please. HHS secretaries don’t resign and hundreds of millions of taxpayer dollars spent on advertising for a law that is working as designed. I think if the it turns out the auto-enrollment letters HHS sent out at during the final months are being counted towards the enrollment figures (has any heard the latest on this?), we would also need to tweak our definition of “working”.

Jan April 15, 2014 at 2:09 pm

Health care reform — the term itself — means ‘not working’ to opponents of the ACA. These next few months will be the last gasps for serious, wholesale opposition to the ACA. The conservative dialogue will move from total repeal to constructive tweaks in the next year or so as Republicans understand that it is not something they can effectively campaign on.

It is exceedingly rare for cabinet members stick around for a full two terms. Look at the turnover in W’s cabinet members and also Obama’s so far. HHS Secretary is a really tough job and after five years overseeing the biggest overhaul of 1/5 of the economy ever almost anyone would be ready to rest. If you think about it, this is pretty much the only time Sebelius could have stepped down without it being some sort of huge scandal. And yet it’s still a kind of scandal!

Jay April 15, 2014 at 3:14 pm

I agree, once people start receiving subsidies, wholesale repeal will be next to impossible. Though that says nothing to the success or failure of the law, just a political reality for all govt. programs.

Z April 15, 2014 at 3:22 pm

And people say it is not a cult. Just look at poor Jan. If he is handed a purple robe, sneakers and a beverage, he will ask no questions.

Jan April 15, 2014 at 3:50 pm

Z, you’ve stopped arguing coherently on the substance of anything. Liars, cults and bizarre theories are what you deal in now.

chuck martel April 15, 2014 at 6:59 pm

The real definition of “working” wouldn’t be how many people signed up but if everyone that had a stroke or was injured in a car accident or contracted shingles was immediately admitted to a health care facility, given state of the art effective care and discharged without having to phone a bankruptcy attorney the next day. We won’t really know if the thing is “working” for some time. By then it’ll be too late to do anything about it.

Z April 15, 2014 at 3:20 pm

So you think it is hurting my “narrative” even though you confess to having no clue what that even is? It’s this sort of oogily-boogily that keeps life entertaining.

Jan April 15, 2014 at 3:48 pm

Give me a topic that has any relationship to politics and I can anticipate with shocking accuracy what Z’s views on it are.

mulp April 15, 2014 at 7:12 pm

So, you are saying that not one person in Texas or other “red states” is unable to see doctors or get prescriptions for chronic conditions because single adults in every State qualify for Medicaid if their minimum wage job pays $14-15,000 per year, or private insurers always provide inexpensive health insurance policies to the working poor with pre-existing conditions for the few dollars a week they can afford?

Michael April 15, 2014 at 5:40 pm

Yes, it’s amazing how they suddenly aren’t using the “48 million uninsured” number anymore.

Jan April 15, 2014 at 11:44 am

“The old questionnaire asked consumers if they had various types of coverage at any time in the prior year. The new survey asks if they have insurance at the time of the interview — in February, March or April — then uses follow-up questions to find out when that coverage began and what months it was in effect. Using this technique, census officials believe they will be able to reconstruct the history of coverage month by month, over a period of about 15 months, for each person in a household”

“The new survey was conceived, in part, to reduce a kind of bias or confusion in the old survey. When asked about their insurance arrangements in the prior year, people tended to give answers about their coverage at the time of the interview — forgetting, for example, if they had Medicaid for a few months early in the prior year.”

Joseph Ward April 15, 2014 at 11:45 am

This reminds me of when the Federal Reserve stopped tracking excess reserves last year.

John Thacker April 15, 2014 at 11:46 am

As a side note, I have been reading far too many blog posts about “numbers enrolled” as a metric of success for Obamacare. That has never been a good test of the serious criticisms (and defenses) of ACA.

No more so than the surge of people filing their taxes as we approached today indicates how good the tax code is.

F. Lynx Pardinus April 15, 2014 at 12:44 pm

Actually, “bad” tax codes typically have low rates of compliance.

Christian April 15, 2014 at 11:53 am

Hi Tyler,

Can you revisit and update your predictions for the ACA? I’m curious how you stand now given current enrollment numbers and your previous prediction about a dysfunctional exchange.

http://marginalrevolution.com/marginalrevolution/2013/10/what-is-the-most-likely-path-forward-for-the-aca-exchanges.html

If enrollment numbers are not a good test of the ACA, we should define an alternative test now for use later. Perhaps the format of census questions are irrelevant for the alternative test.

Yancey Ward April 15, 2014 at 12:04 pm

Until the enrollment numbers break out actual paying enrollees and the demographics, I don’t think Tyler need update anything. Now, I don’t expect HHS to make any effort at all to gather and release these numbers- they made a decision to go with “plans in a shopping cart”, and they will defend that method to the bitter end.

David C April 15, 2014 at 12:08 pm

I agree. Considering that getting more people insured is one of, if not THE,primary goal of the ACA, why are enrollment numbers NOT a good test of the ACA?

Finch April 15, 2014 at 12:12 pm

If they come from people who had to drop another health insurance plan to take the government plan, those people have been harmed, not helped.

spencer April 15, 2014 at 12:41 pm

I’m sorry, you do not know if they are helped or hurt unless you compare the two policies.

I can claim that so many of the old policies were so useless that the new policies help.

But it is just a claim without actual comparisons .

Finch April 15, 2014 at 12:48 pm

Fair point, but if there was any expectation at all that they would have been helped, there would have been no need to force them to drop their old plans.

I think we can stand by the press reports indicating these people were generally harmed.

Jan April 15, 2014 at 1:13 pm

You would need to evaluate this in the context of how many people drop their insurance plans in an average year. There is always churn.

Jay April 15, 2014 at 3:19 pm

@Jan

Are you arguing the average “churn” is anywhere close to 6 million? Churn includes those switching plans of their own accord, I sincerely doubt the industry, minus the ACA, cancels anywhere close to 6 million in the average year.

Jan April 15, 2014 at 3:45 pm

Did I say that? I am arguing you need to subtract the average churn if you are going to assert that all the plan changes were due to plans being canceled. Just as conservatives don’t want to count all exchange enrollees as newly insured, we should not consider all new enrollees to have been driven out of their old plans by the ACA.

Ad Nauseum April 15, 2014 at 5:39 pm

Actually Jan, the conservative argument that exchange enrollees shouldn’t be counted as “newly insured” is based on the data that nearly 4.2 million health plans were actually canceled by health insurance companies due to non-compliance with new ACA regulations. Are you trying to say that those should be counted as “churn”?

Jan April 15, 2014 at 5:53 pm

Yeah I know where the argument comes from. I’m saying you can’t count every new sign up as a canceled policy. Also, did you know that the administration gave insurers the opportunity to grandfather in old policies that a didn’t meet ACA standards for a couple years? I don’t agree with that policy, but they did it.

Ad Nauseum April 16, 2014 at 12:10 pm

Funny, I thought that was up to the states. Please verify that claim. And considering that the statistic about health insurers actually canceling policies, it seems irresponsible to try and attribute it to “churn”.

There is good and bad in this legislation, but you only seem to see all good.

David C April 17, 2014 at 12:29 pm

“if there was any expectation at all that they would have been helped, there would have been no need to force them to drop their old plans”

Well, this is logically false. We have to force people to do all kinds of things that help them, Things like get vaccinated, wear seatbelts, get car insurance, etc…

David C April 16, 2014 at 9:26 pm

That they have been harmed, not helped, is one debatable point, as is the idea that anyone had to take a government plan (the only people getting government plans are people getting Medicaid or Medicare and I would suspect that leaving the private market for those programs is always a help) but I think the basic idea of looking at how many people were enrolled in 2014 and comparing that 2013 to get the difference is a pretty good way to evaluate whether or not the goal of getting more people insured is being met.

Jay April 15, 2014 at 12:18 pm

Its the definition of “enrollment” that is part of the debate. Having a plan in your shopping cart while not paying a dime yet is apparently enough to the administration.

Once you’ve settled that definition there’s then the question of how many NEW enrollments there are versus people who have just been cancelled from an existing plan or are being dumped from employee plans that are no longer being offered.

THEN once that is settled there is finally the question of whether just having one of these plans is actually better for people. The premiums are subsidized for some but the deductibles on the bronze plans are ginormous compared to the incomes of most of those utilizing them, leading to the question of whether these people are actually better off with the insurance, especially for the 20-somethings who will be paying a nontrivial amount of their income on healthcare and college debt during the exact years they should be accruing wealth.

John B. Chilton April 15, 2014 at 12:16 pm

For those who don’t click through this is what Tyler wrote:

“6. The exchanges will be mostly working by March 2014, but by then the risk pool will be dysfunctional. In the meantime, real net prices will creep up, if only through implicit rationing and restrictions on provider networks. The Obama administration will attempt to address this problem — unsuccessfully — through additional regulation.”

The simple answer to Christian’s query (“I’m curious how you stand now given current enrollment numbers and your previous prediction about a dysfunctional exchange.”) is that it’s not the enrollment numbers that matter, it’s the risk pool.

The jury’s out on the risk pool — lots of opinions out there on whether exchange premiums will go up for 2015.

mulp April 15, 2014 at 8:51 pm

“If enrollment numbers are not a good test of the ACA, we should define an alternative test now for use later. Perhaps the format of census questions are irrelevant for the alternative test.”

Shouldn’t we judge the effectiveness of the EMTALA provisions on the number of emergency and hospital bills paid by those who got treatment regardless of insurance or financial ability to pay for medical care or pregnancy under EMTALA?

Lonely Libertarian April 16, 2014 at 11:38 am

Enrollment numbers? What enrollment numbers – if you are referring to the ginned up guesses that led to the victory lap – good luck with that.

Let’s see how many people are actually covered by some form of insurance a year from now – and maybe the new questions will do a better job of defining that than the old.

And lets see how many people who have actually had to file a claim are happy with their insurance after they find out how much they will be paying for their “free” healthcare.

And let’s see how large the “cash for coverage” niche is in a year – it is certainly a better alternative for the young and healthy than Obamacare.

Jay April 15, 2014 at 12:12 pm

So while it is politically helpful to overestimate the number of uninsured, no change to the questions. The law passes and only NOW we need to redesign the questions to such an extent as to make comparisons meaningless and not a true arbiter of the success of the law even though we could easily keep around the old questions so we could compare? Coincidence is a severe understatement.

Michael April 15, 2014 at 12:22 pm

“That has never been a good test of the serious criticisms (and defenses) of ACA.”

And what would be a good test?

Tyler has this awful habit of pointing at the wrong answer and then implying the right answer so vaguely that you have no idea what his real point is. It’s a great hedging tool to minimize times when he’s wrong–but is a horrible way to educate or inform.

andrew' April 15, 2014 at 12:33 pm

This time it isn’t hard.

The judge of insurance is whether it is priced correctly.

If you were a betting man, would you put money on the government?

Z April 15, 2014 at 3:23 pm

I’d always bet on the guys with the guns.

dead serious April 15, 2014 at 5:51 pm

Detroit drug dealers/Hispanic gang members/?

Duck Dynasty-type rednecks?

Biathletes?

Michael April 15, 2014 at 8:58 pm

“priced correctly”. Who gets to say what is the correct price? God? The invisible hand?

David April 15, 2014 at 12:42 pm

It’s really annoying. Mankiw is the master of this. He constantly picks at little points in someone else’s argument while contributing no thoughts of his own. I understand being humble about what you know, but part of having a discussion is putting your own ideas out there for others to see and criticize. Refusing to do so doesn’t make you seem wise or mature; it implies that you either have nothing to contribute or you lack conviction.

Millian April 15, 2014 at 1:05 pm

I do not see why an alternative is needed. Some claims are foolish regardless of the alternatives.

CG April 15, 2014 at 2:13 pm

He’s said before that a better metric would be the enrollment numbers of the previously uninsured.

Surveys by McKinsey and the Urban Institute have placed that number between 10 – 27% of new enrollees, including those who became uninsured as a result of the law.

T. Shaw April 15, 2014 at 12:35 pm

“The problem isn’t what the people don’t know. The problem is what the people know that isn’t true.” Will Rogers

ummm April 15, 2014 at 12:44 pm

Enrollment not the same as being insured. Even if enrollment meets targets there may be few that get insurance and even fewer that are able to keep it

prior_approval April 15, 2014 at 12:54 pm

‘I have been reading far too many blog posts about “numbers enrolled” as a metric of success for Obamacare’

As compared to writing them?

http://marginalrevolution.com/marginalrevolution/2014/03/how-many-of-the-previously-uninsured-have-signed-up-for-obamacare.html – ‘There is more here. You will note that a low rate of sign-up is distinct from a rate of sign-up skewed toward the elderly and the sick. In this sense we still do not know how the new law is doing, though in a broader sense a low rate of sign-up should not be considered good news.’

Or this – http://marginalrevolution.com/marginalrevolution/2013/10/bob-laszlewski-on-the-aca-exchanges.html

Jan April 15, 2014 at 3:05 pm

I am not sure why Tyler is not satisfied with the Administration’s response. The survey is being implemented to measure the uninsured rate in the year before full ACA implementation, 2013, and then for 2014. That will allow for an apples to apples comparison of before and after.

Jay April 15, 2014 at 3:18 pm

I don’t think a single year before “full” implementation is enough for apples to apples. The law was passed in early 2010 and has had various statutes slowly go into effect since then.

Jan April 15, 2014 at 3:34 pm

It seems like people generally agree the methodology was flawed and they’ve been working to get the survey updated for a couple years. How else can we do this in a way that might satisfy the anti-ACA crowd? I don’t think delaying the survey update is an option since we will need a baseline that allows for an apples to apples comparison from the first year of full implementation going forward.

As noted by Justin Wolfers, we do have other uninterrupted data sources on insurance rates, like Gallop. Won’t that be enough to point to that in order to prove insurance rates didn’t go up, if that turns out to be the case (since this is that is the true motivation of the opposition)?

Jay April 16, 2014 at 11:29 am

You do it by keeping around the old survey questions along side. Yes Gallop runs their monthly insurance survey but I’m always worried that either side will look at the results of the new questions and make an apples to apples comparison anyway without mentioning that the questioning methodology totally changed.

Jan April 15, 2014 at 3:36 pm

And if the ACA has actually been in the process of implementation for years and 2014 isn’t that important, can’t people just look at the survey for the past three years?

Andrew' April 15, 2014 at 4:05 pm

I’m guessing the data isn’t just used to show ACA is successful. If the old method is not good enough, it might have been nice to have accurate data before ACA campaign. Finally, this is just not how you do run charts- to change the measurement in the midst of a process change- especially if you don’t want the appearance of twerking the data. Everything isn’t about the program that he shouldn’t have pursued during a deep recession.

Andrew' April 15, 2014 at 4:15 pm

Why does the standard become “either prove gross corruption or just accept the vagaries of bureaucracy.”

In the smallest of production facilities, if someone proposed changing the metrics during a major change, 3 people would raise their hands in the meeting.

Jan April 15, 2014 at 4:18 pm

Oh it definitely would have been nice to have accurate data before. They started working on these changes years ago, well before Obamacare was passed, according to National Review’s analysis. (http://www.nationalreview.com/corner/375792/census-bureau-sorry-were-going-have-hard-time-measuring-whether-obamacares-working) Of course, it takes a while to pilot and then finalize a methodology. Not to mention, the ACA passed in the middle of their process to update the survey, so considering how rarely they update this thing, it just made sense to ensure the new survey incorporated questions about things like whether someone was getting insurance on an exchange and whether it was a subsidized plan.

On the bright side, 16 House Republicans have legislation to prevent Census from conducting the Current Population Survey at all — https://www.govtrack.us/congress/bills/113/hr1638

Andrew' April 15, 2014 at 4:30 pm

Jan,

On another blog, I’ve asked people to give an example of where Obama has told A truth about the NSA. Good luck.

I say that to say this, what part of every time I/we point out where they did it wrong puts me/we into some Anti-ACA free speech zone?

Jan April 15, 2014 at 4:48 pm

I haven’t followed Obama’s NSA comments very closely. You may well be correct on this one.

Did I put you into an Anti-ACA free speech zone? What exactly does that mean? I think you are implying that I/we blindly support every single thing about the ACA and all this president’s policies. It’s not true. For example, I don’t agree at all with the various extensions and exceptions to mandates in the ACA. I don’t recall any instance where you supported a single component of the ACA or any policy of this administration. Am I wrong about that?

In the meantime, can we return to this insurance survey stuff? Are you conceding my point?

Bill April 15, 2014 at 4:21 pm

This is a funny post for two reasons, aside from the sinister assumptions.

First, states and state insurance commissioners, and the National Association of Insurance Commissioners, collect data on the size of the uninsured population. You could also gather data based on hospital admits.

Second, the Change in the Census collection hurts Obama, not helps him…because the news reports Will compare data and will Not qualify their statements to reflect the changes.

Jay April 16, 2014 at 11:32 am

How do the new questions hurt him? To my understanding, the previous questions overestimated the number of uninsured while the new questions do not. My fear is that the official numbers will now look smaller and the left will point to them and say see it worked without mentioning the methodology change at all.

Bill April 16, 2014 at 10:46 pm

Jay, Obama will not be given credit for the changes because they will say it was due to the Census instrument. If there is a change in the size of the uninsured downward, someone will say: it was the Census.

By the way, here is a Gallup survey on uninsured and changes to the population:

http://www.gallup.com/poll/168539/uninsured-rates-drop-states-embracing-health-law.aspx I don’t suppose that Obama will be able to cite the Gallup survey.

What I also like about this whole story is that it assumes that Census is the ONLY source for data on this subject. It isn’t. In fact, virtually all states collect data on the size of the uninsured population in their state. Health insurers have an interest in knowing this info, as to state health and insurance departments.

JWatts April 15, 2014 at 4:33 pm

“The old survey methodology is said to have been prone to measurement errors that overestimated the number of uninsured; the new data supposedly corrects that problem.

But back in 2009, the White House moved to exert more direct control over the Census Bureau and its operations.

The Times reports that the update was not politically motivated…

Bullshit.

Jan April 15, 2014 at 4:54 pm

Anyone with any survey methodology and/or health insurance expertise knows the old survey instrument was terrible. Census Bureau began to pretest updates to the survey well before the ACA passed, in early 2009. http://www.fcsm.gov/09papers/Pascale_VIII-A.pdf

Serious question: how much of a buffer would have been enough to convince you it was not politically motivated? Before Obama was president? Maybe sometime before he announced he was going to run?

JWatts April 15, 2014 at 7:20 pm

Umm, how about we don’t change the survey questions for the next 4 years. I’m sure that would have been fine. But deciding to change it just as the ACA kicks in, is hardly a coincidence.

In my mind this is hard core politics pure and simple. And your comment to how the “old survey instrument was terrible” is ridiculous mood affiliation. They’ve used the old survey questions for the last 30 years. If it was acceptable for the last 30 years, it wouldn’t have made any appreciable difference to keep it for another 4 or 5 years. The timing reeks of political calculation.

Jan April 15, 2014 at 7:46 pm

You mean after they worked on how to improve the survey since 2009, a survey that everyone agrees is flawed and which we have spent plenty of money beta testing and perfecting and captures new info related to changes in insurance since the ACA was implemented?

Ok mood affiliation. Let’s stop making improvements to government programs because of politics. It only looks bad because people politicize everything. No wonder it’s so hard to make government more efficient. Why can’t people apply rational analysis to the new numbers in the context of the changes? Because it’s more fun to make controversy.

anon April 15, 2014 at 8:12 pm

Jan and JWatts, how would you both like to be right? On the eve of a major policy change is a poor time to change measurement techniques; however, it is the right time to make sure your measurement techniques are the best ones possible. Tradeoffs. Also the final assessment of the ACA (to the extent that it depends on data at all) will not be about the transition but about the steady state impact on health care and health insurance. Still shame on the administration for not handling the PR of the survey change better – there are a whole lot of time series that had breaks in them – statistical and survey method folks know how to adjust for these. Don’t kid yourself if this survey change were politically motivated, it would have been handled much better. Unforced (non)errors FTW.

JWatts April 15, 2014 at 9:19 pm

“Let’s stop making improvements to government programs because of politics.”

That’s just an argumentum ad absurdum. As I pointed out, there would have been no issue if they had just made the delayed the changes for a few years.

Jan April 15, 2014 at 10:06 pm

For a few years? No, that’s completely idiotic. They’ve already been working on it for quite a few years.

Mesa April 16, 2014 at 9:08 am

Does anyone else find the affected Vox lingo annoying – “like, totally, seriously, don’t freak out dudes”. It permeates the site and is design to signal “we are cool younger types even though we like data.” . I must be getting old. It’s. Almost. As. Annoying. As. This. Stuff.

Comments on this entry are closed.

Previous post:

Next post: