Big Data, Big Government, Big Brother

In addition to monitoring who you call and when, your email, and your internet searches the government also has access to all of your credit card purchases. We usually don’t think about purchases as communication but what people buy says more about most people than does their email. Buying behavior can be used to predict all manner of information about your political views, affiliations, sexual activity, marriage quality and much more.


If Visa / MasterCard had sold the same transaction information to, say, a market analysis company, would that be illegal under some US law? Just curious.

No, not illegal. It happens all the time. See the second link

I don't think visa and mastercard sell their data. It's too strategic for then to do so. As far as I could see, the data came itself not from visa or mastercard.

I don't see the economic value of transaction data for visa and mastercard when it is not sold. On the other hand, I can see much value for many non-exclusive licensees.

In which case the Government surveillance scandal does become a bit weaker, doesn't it? Say, NSA had bought the transaction information from VISA? That'd be totally legal, I suppose.

In contrast to the case of Skype or Facebook: If they sold my chats to a third party, I think it'd be cause enough to sue under their terms of contract.

That's one of the questions I've had. There are very old rules about what data the government may collect. There may not be so many (any?) rules on what it can buy.

The cell phone meta-data (say it was purchased) is a good proxy for a GPS tracker attached to your car. The government needs a warrant for the tracker, but might already own a copy of the meta-data.

Here is an article (from 2006 - there is nothing actually new about what is going on) that provides some info -

' Furor and confusion over allegations that major phone companies have surrendered customer calling records to the National Security Agency continue to roil Washington. But if AT&T Inc. and possibly others have turned over records to the NSA, the phone giants represent only one of many commercial sources of personal data that the government seeks to "mine" for evidence of terrorist plots and other threats.

The Departments of Justice, State, and Homeland Security spend millions annually to buy commercial databases that track Americans' finances, phone numbers, and biographical information, according to a report last month by the U.S. Government Accountability Office, the investigative arm of Congress. Often, the agencies and their contractors don't ensure the data's accuracy, the GAO found.

Buying commercially collected data allows the government to dodge certain privacy rules. The Privacy Act of 1974 restricts how federal agencies may use such information and requires disclosure of what the government is doing with it. But the law applies only when the government is doing the data collecting.

"Grabbing data wholesale from the private sector is the way agencies are getting around the requirements of the Privacy Act and the Fourth Amendment," says Jim Harper, director of information policy studies at the libertarian Cato Institute in Washington and a member of the Homeland Security Dept.'s Data Privacy & Integrity Advisory Committee.'

Alex, you are confusing a Merchant with branded transaction processing networks.

Legally Target cannot use that personal data from every Visa or MC (and probably not AMEX or any other credit card since they're all subscribing to the PCI DSS model and I think the limits what track information they can retain). That's one reason why all these store want you to sign up for their branded card -- now the have an account number they CAN tie back to a specific customer for such targeting.

+1 A 5% rebate for your privacy.

Also, if you want to do an experiment, do this:

Go to the ACLU website and then go the MR website and report back which one placed a cookie on your computer.

Actually, neither did - but then, I use Seamonkey with cookies set to 'Accept for current session only' (which means cookies are cleared every time I close the browser) and 'Allow cookies for the originating website only.' And I don't use javascript or flash.

So, the point is that many people are unaware of how to limit cookies, and that many web sites do not actually set their own?

And besides, as noted several times when discussing such basic issues, this web site is likely far more interested in the IP addresses of visitors, along with http_referer information.

I run another program that clears cookies and I see something else. I get cookies after visiting this site.

Why don't people understand the difference between the old bag seeing you on the corner and money getting deducted from your paycheck to pay some government flunky to watch you? If the SCOTUS doesn't see the difference, that doesn't mean there isn't a difference, it means they are idiots.

Yes and No. Personally Identifying information must be protected and cannot be sold. "Scrubbed" data, from which one could not identify who the person conducting the transaction, is not as protected but then you get into a question of data ownership. The cardholder transaction information is considered the property of the issuing financial institution by the transaction processing businesses (Visa, MC, Pusle, Interlink, TSYS, FIS, FDR) but I'm not sure that the issuers believe the data is theirs rather than the owner of the account. This gets into banking relationships and law.

When Target...ironically targets US citizens with hellfire missiles then I'll worry about that. Until then, they are providing good products at decent prices while the government is NOT preventing terrorist attacks.

US citizen Anwar al-Awlaki was a senior recruiter for al qaeda. But I'm sure targeting him with a hellfire missile did nothing to prevent terrorism.

You are right. The drone program has probably served as a recruiting bonanza.

Al Qaeda recruiting...please.

Our master plan is that people willing to suicide bomb us will be scared to join up because we might get one of them for every 1-10 innocent civilians and children?

I'm also not so concerned by the attempts of corporate marketing departments to get a better understanding of their "market", i.e. their customer base. I would prefer they do this without my personal information and without breaking agreements they entered into.

The real problem with Target was it was not following the rules and their database got cracked providing information that put their customers at risk of indentity theft in the process. That costs all of those people some minor problem -- if only having one card disabled and a new on issued, some may have experienced worse and increased costs to both Target, their shareholders and external parties.

It would be simple enough for marketers to conduct the work without requiring specific personal information about the customer -- the collection of goods bought will go a really long way.

>>>Personally Identifying information must be protected and cannot be sold<<<

What Federal Statute says that? I'm curious to know the legal position.

I'm skeptical because of the existence of narrower state laws like California Online Privacy Protection Act of 2003 and even that does *not* forbid disclosure of PII but only requires disclosure. Again, I'm no lawyer, so likely that I'm wrong.

I know most of this because I work in the industry.

Here's something from the FTC( talking about the Gramm-Leach-Bliley Act:
NPI is:
any information an individual gives you to get a financial product or service (for example, name, address, income, Social Security number, or other information on an application);
any information you get about an individual from a transaction involving your financial product(s) or service(s) (for example, the fact that an individual is your consumer or customer, account numbers, payment history, loan or deposit balances, and credit or debit card purchases); or
any information you get about an individual in connection with providing a financial product or service (for example, information from court records or from a consumer report).

Admittedly the link is open to some interpretation as it's talking about personal finances and financial institutions -- with that being defined very broadly -- and it's not clear that a merchant falls directly into that classification. This is where the contract between the merchant and the processing network comes into play. All the major branded cards will prevent merchants from storing the card's track data -- they need to send it in the transaction but once complete the track data needs to be deleted from their system. Since that's a contracted term, keeping and then selling it later would be a breach of contract.

There is also the common law concepts of invasion of privacy and the expectation of privacy to consider. I'm not sure if you would consider that "Federal" law but it does seem to something that applies in all the State and would be protected under the Constitution (I think).

In the modern age of electronic information I think things are very problematic and all these privacy terms that are probably read by less than 1% of the site visitors are almost never about protecting the visitor's personal information but the web site from some law suit. What's really bad here is that these terms are coming close, in my view, or establishing property rights in this personal information. Rather than having a standard that says the company can collect and sell such user information as the cost to enter the site I think we should require they: 1) behave like a bricks and mortar store (no personal information required to get in, 2) be required to protect any personal information they do collect and maintain (e.g. shipping addresses) and be liable for any breach and resulting damages from the breach, and 3) only be able to sell NPI and PII data with explicit, signed consent of the person.

Within that structure I don't care if they want to provide incentives for letting a company sell your personal information. There would be some issues to resolve around the length of time the terms will be in effect I suspect. For instance, if I create an account, when I cancel it can the company still sell my phone number, email address or physical address after the fact?

Thanks. Just like your Gramm-Leach-Bliley Act protections, there's HIPPA protections against PII disclosures in healthcare. There seems a patchwork of sector specific restrictions, but I don't see any overarching legal protection forbidding PII disclosures.

If you look at the details of Invasion of Privacy torts, I don't think they'd apply here. None of the four underlying requirements seem satisfied (False light, Appropriation etc. )

Hope one of the lawyers that frequent MR can comment.

So what? Would you rather be safe or sorry? There is a trade off between security and privacy, as everyone who has ever been at an airport knows ....

Where do all these apologists come from?

He's partly correct. There *is* a trade off between privacy and security. But presenting "safe or sorry" as a binary is incorrect. And some types of information gathering might not even make one safe.

+1 -- odd moniker though ;-)

Did you mean to say "safe now and sorry later?"

"They who can give up essential liberty to obtain a little temporary safety, deserve neither liberty nor safety."

I for mine would rather be "sorry", if these two are the only options without any compromise. But if you prefer to be "safe", that's fine with me. Go ahead, but don't subject the rest of the population to the same standards:

Hey Andreas, your idea is pretty lame ... ever heard of adverse selection?

Shit! I just ordered a pressure cooker!

I'm sorry Andrew, but I think that means you are a communist.

Dammit! If only the NSA had the technology to predict that and warn me beforehand! I hate being a communist!

No, it means he is an anti-commie freedom fighter President Reagan hoped would liberate the Russian occupied territories. If he goes crazy and kills Americans, well, he's just crazy, unless he's Muslim, then he's a terrorist. After all, Eric Rudolph isn't a bad guy, just a bit misguided in his Christianity.

Is Andrew a Christian or Muslim?

I never thought about it before, but I guess I must be a Muslim.

If you surf more cooking pages than political ones you might be safe. Perhaps include a garlic press in your order, rather than say a new backpack.

Got nails?

Foooooooooooock! I do have nails!

I AM Muslim! Watson, help me!?

Andrew' you're supporting my view that this all doesn't matter too much. what do I know about you after months of reading here?: nothing that I'd believe to be true except you're funny. I bet on people over Watson any day ... life is not a game show.

So he's a *funny* commnist muslin. All the more insidious.

(I think and hope you are at least half right, but them trying- if it "works" or is just another boondoggle both indicate they should be fired)

Of course, I meant funny ha ha. Also I'm no fan of Watson on this job ... inefficiencies are safety valve.

Not that surprising, given how the major credit card companies, as well as amazon, paypal etc. all willingly (and illegally) helped the government target wikileaks back in 2010. Nobody knows how to say no. They're too scared not to comply, and probably with good reason.

Actually, there is a bit of a theory that suggests the success of the American government in targeting wikileaks set the template for the American government doing the same to American media. And which further suggests that why we are hearing about such things now is an attempt by the media to preserve themselves from the same fate - now that they have recognized that wikileaks was actually a newsgathering operation.

In condensed form - First they came for the people reporting the truth, and then they came for the comfortable hacks posing as journalists.

Journalists are terrorists. What part of that do you not understand? We won't be safe until they're all in Gitmo.

You've got it wrong. Journalist are not terrorist. The Truth is terrorism and only the journalist, or anyone else, attempting to spread it around are terrorist.

So you're saying not all journalists are terrorists. Okay, I'll grant you that.

Were you out in the streets when Republican candidates for president and major leaders in the Republican Party called Ron Paul "dangerous" for expressing your view on what is constitutional, illegal, or not?

We the People reject Ron Paul et als views on these matters based on who We the People pick to represent us in Congress. Congress passes laws in the full light of day, and holds debates in public. And Congress is We the People by proxy overseeing the execution of the laws.

No I wasn't out in the streets, nor am I one of "you the people" (I'm not American)

And I too have an (only slightly weaker) excuse - though American, I've lived in Germany for 20 years. I don't need to go out on the streets to enjoy data protection laws. For example, google Streetview is not legally allowed to show this address. Though only the naive think that the collected data is unavailable to anyone who knows the secret regulations. And I've known for more than 30 years any call I make from Germany to the U.S. is recorded - all perfectly legally according to U.S. law, one must note for accuracy.

"We the People" did not select this President or Congress. About 30% of The People made the choice, the other 70% opposed or provided no support.

We've got a broken democratic process that fails to produce the "We the People" outcomes. Is it really any wonder we have such bad and controlling government? Minority ruler are always fearful of the masses.

Even "affiliations"?

Does this mean the government knows my "mood affliations"?

"Does this mean the government knows my “mood affliations”?"

Probably. There are no privacy rights in blog comments. I operate on the assumption that if the government really wanted to find me they could link my online alias back to my real identity pretty quickly.

We don't have to have property rights to not have our money stolen to pay people to spy on us.

"We don’t have to have property rights to not have our money stolen to pay people to spy on us."

Yup - there is definitely a file on you somewhere.

Most people are not as special as they think they are and aren't worth spying on.

We the People are "the government" and the elections have frequently revolved around who is willing to do what it takes to "protect us".

The Republicans have made "we can protect you, and the Democrats won't" their brand for decades; that was the reason Reagan won. Twice.

And Karl Rove laid it out in 2005:

"Conservatives saw the savagery of 9/11 in the attacks and prepared for war; liberals saw the savagery of the 9/11 attacks and wanted to prepare indictments and offer therapy and understanding for our attackers," Mr. Rove, the senior political adviser to President Bush, said at a fund-raiser in Midtown for the Conservative Party of New York State.

Citing calls by progressive groups to respond carefully to the attacks, Mr. Rove said to the applause of several hundred audience members, "I don't know about you, but moderation and restraint is not what I felt when I watched the twin towers crumble to the ground, a side of the Pentagon destroyed, and almost 3,000 of our fellow citizens perish in flames and rubble."

What has really bamboozled conservatives and Republicans with Obama is his understanding of the Constitution. Congress debates and writes the laws, and the President executes them. When Senator Obama argued as citizen and then as Senator against some of the bills and laws of Congress, they assumed President Obama would ignore the Supreme Law of the Land which includes the laws of Congress and Treaties because he opposed some of them before, and even as President.

The law is the law, no matter what you think about the law. Just as judges are expected to sentence people to prison for drug use even though the judge knows that it is stupid, the President, no matter who it is, is required to execute the laws as they are, not as he thinks they are.

And with Republicans winning election to Congress by attacking Democrats for being "weak", and constantly attacking Obama for being "weak", it is clear Congressional oversight of the Executive requires going as far as Republicans demand when it comes to terrorism.

We are a republican democracy, and We the People want "government to protect us" inspite of Ron Paul saying that is dangerous. Every single Republican opponent of Ron Paul, plus all the Republican leaders called Ron Paul "dangerous". If it weren't for the Tea Party messing up the Republican Party, the Republicans who call Ron Paul civil liberty position "dangerous" would control Congress. Obama was after all, the one who stated that he would use the endless global war on scary Muslim declaration of war in September 2001 to kill bin Laden in Pakistan with all his opponents saying he couldn't.

On the secrecy issue, Republicans in Congress have been attacking him on the leaks, and given Congress wrote the laws on secrecy, it is clear Congressional oversight requires Obama to do everything possible to keep the secrets. Republicans are the ones who have the record on defending state secrets.

Gays have been upset that Obama didn't just ignore the laws of Congress. Environmentalists have been upset that Obama didn't just ignore the laws of Congress. Even Republican have been upset Obama hasn't ignored the laws of Congress to give them reason to attack him.

Obama has been very divisive of the Republican Party which set out to declare everything Obama did as anti-American, but now they must defend Obama "violating" the first and fourth amendments based on the due process of Congress authorizing these actions.

When it comes to the Constitution, the two places Republicans and Obama disagree is on the first enumerated power of Congress and on the second amendment - Republicans are opposed to the first power of Congress and believe that 10-12,000 people killed with guns does not justify any limits on the second, but one person killed by a Muslim justifies limiting the first and fourth.

But it all comes back to what We the People want. If We the People do not want the security state, then We the People should be outraged when Ron Paul and Ron Wyden are called dangerous, and vote for the opponents on the left of all such politicians. It is the left where the civil libertarians are located on the political spectrum.

I would note that We the People and the media didn't know what the laws passed by Congress in 2001 (and amended since) say, until the implementation details are leaked when Obama is president. Even though the leftist Hollywood has delivered lots of movies and TV laying out the security state, inspired by the laws of Congress - Obama and the administration cite the laws, not executive power as the Bush administration did, as the authority for action.

And I hope you all watch Person of Interest which is the best big data scifi yet.

That's what I said. Vote Libertarian.

The People elected Obama, who ran on a platform of ending this shit. Instead of ending it, he increased it's scope. Very few in congress were briefed about how these operations really work.

This isn't democracy, it is a leader seizing power against the will of the people.


If you can't not trust Congress on what they weren't briefed on and couldn't comment on and can't not trust the non-courts that judiciate this or the guy who says you can trust him then we have a problem.

The civil libertarians arn't represented anywhere on the political spectrum. Vote against the political spectrum.

Something of a structural failing of our democratic institutions. There is not "none of the above" option nor is there a requirement for the representatives or President to meet any real majority of the people hurdle.

We've got a system of minority representation and given the results it shows that the Median Voter Theorem has no place to speak of in the analysis of our political outcomes or institutions.

Functionally, why is “none of the above” any different from not-voting?

Ok, it does have some signalling value.

Depends on the precise wording of the election laws. Say, if a majority of votes were required to be declared victor rather than simple highest count, but thanks to the number of NOTA votes no candidate had a majority.

Of course, these days it wouldn't be too surprising to see NOTA with the majority from time to time...


Does any nation have such a system though of mandating an absolute majority for a Presidential or similar election? I'm curious.

People close their blinds at night because they want privacy for its own sake.
It's part of who we are as modern humans. What people do after they've spied on us is only a secondary concern. We hate the Peeping Tom for peeping, not for masturbating.

Many people commenting here should have no problem at all with the installation of sensitive listening and visual recording devices outside of every home in the country as long as those are on public property.

Credit card transactions also give snapshots on where you were at a particular moment in time.

If this is about a right to privacy then I will presume everyone who considers this offensive is much more disturbed by the right wings efforts to have the government dictate who you can marry, what a woman does with her own body, or what you do in the privacy of your own home.

In any event, I suppose its hard to get too worked up over the government storing raw meta data for possible use under warranted circumstances when we seem to have so willingly given away our whole lives on Facebook, twitter, cooking through the internet, and basically the small print of every thing else we do.

Civilization is always a balance between privacy and public needs. I'm not saying we should not be leery of too much government intrusion (the constitution refers to it as unreasonable searches and seizures), but with respect to the current situation it would be a little like the police not allowed to patrol the streets in the 18th Century to prevent possible crimes to suggest that today the Government can't patrol the public byways and airwaves of the internet and phones.

Liberals have to be careful not to take every act of government and turn it into a sinister act just because it has the potential, if abused, to become one. The rightwing isn't looking to fix government but rather destroy it and when it comes to real privacy that matters they clearly don't mind the government dictating how you live your private life.

P.S. Real technical innovation seems to have stopped around 1950's with computing, communications, jet planes, and space, and social innovation around 1960's with the equal rights and voting rights amendments. Since then I think "small steps toward a much worse world" might be more fitting tag line.

Moderates, or whoever these people are, need to be careful not to equate unlimited and unstoppable Big Brother with a segment of the population not trusting Progressives to dictate traditional institutions such as marriage through election fliers.

The last really important innovation was the microprocessor around 1970. Since then the most important technical advance has been "fracking" which has the potential to change the world's economy and politics for the next hundred years.

Andrew, your suggestion that somehow moderates or liberals are for government abuse of power while the rightwing is the champion of freedom is about as idiotic a concept that has been foisted on the American People since Reagan came up with cutting taxes reduces deficits.

The left is for government of , for, and by the people that works and fixing when it doesn't so it serves all Americans and not just a few. Today's rightwing is nothing more then retreads of the old civil war south. Their goal is not to solve problems or fix government when it needs but rather to destroy it. Their objective is to make the world safe for exploitation.

As for Joe, the basic principles of computers, key functionality, and use was established and made possible in the 50's and was driven by the development of first the vacuum tube and then the transistor. the microprocessor was essentially very small transistors.

I rather doubt today's Left (such as it is) enjoys the monopoly of populism you claim, plus the Left hardly stands above or far apart from the opportunistic moral unilateralism practiced by a Bradley Manning or an Edward Snowden.

Populist enemies of democracy (to whatever diminishing extent democracy still matters) are pulling our political fabric energetically in opposite directions. "The people" have no single voice and never have: the contests of populisms wax and wane with every generation.

Certainly the left does not have a monopoly on it. This is rather a very broad (perhaps overly broad) statement of the current politic.

As Hamilton advised us about:

It will be forgotten, on the one hand, that jealousy is the usual concomitant of love, and that the noble enthusiasm of liberty is apt to be infected with a spirit of narrow and illiberal distrust. On the other hand, it will be equally forgotten that the vigor of government is essential to the security of liberty; that, in the contemplation of a sound and well-informed judgment, their interest can never be separated; and that a dangerous ambition more often lurks behind the specious mask of zeal for the rights of the people than under the forbidden appearance of zeal for the firmness and efficiency of government. History will teach us that the former has been found a much more certain road to the introduction of despotism than the latter, and that of those men who have overturned the liberties of republics, the greatest number have begun their career by paying an obsequious court to the people; commencing demagogues, and ending tyrants."

Introduction to the Federalist Papers, Alexander Hamilton.

"the microprocessor was essentially very small transistors."

But , but , who connects the really tiny wires between all those transistors? ;-)

I know what a microprocessor is. It was important for what it did for the size, cost and performance of computers. Yes it built on prior discoveries of tube based computers, transistors and integrated circuits but it was hugely important in its own right.

In the 60's when I was in military Communications it took a room the size of a house for the computer that ran our communications center which was part of AUTODIN. Today, your smartphone probably has more computing power then that whole room.

Nevertheless all of the elements of today's computing, communication, and internet where essentially in place. FYI, the process by which microprocessor manufacturing is made possible is essentially Lithography, a technology that has been around for over 200 years.

zb - and when I started computer programming in 1973, the IBM 370 (with tape drives) filled a large room. I read recently that an iPhone had the processing power of a supercomputer from 1975. Microprocessors led to a game changing drop in the cost of computing. It was a major innovation in its own right.

Boss, at least they are open about it.

Getting so you can't trust anyone under 30 . . . .

People who believe in the use of data-mining for the prediction of consumer behaviour should read this first:

As a marketer I don't know why we're still expected to believe that the ever more sophisticated manipulation of large data sets with an ever more bewildering array of clever software tools is a good thing.

And see this:

Of course - Paul Revere really was a traitor seeking the over throw of the legitimate government of the day ...

Comments for this post are closed