The idea is that publishers could use their robots.txt as a ransom note, selling it to the highest bidder — Bing or Google.
Here is the link. Still, it is a cute idea.
by Tyler Cowen on November 11, 2009 at 4:01 pm in Web/Tech | Permalink
The idea is that publishers could use their robots.txt as a ransom note, selling it to the highest bidder — Bing or Google.
Here is the link. Still, it is a cute idea.
Previous post: *Fanfare* meta-list for recommended classical music recordings, 2009
Next post: A public choice theory of the filibuster













Get smart with the Thesis WordPress Theme from DIYthemes.
Well put.
A search engine need not obey robots.txt, but a corporation has to, at least in the part of world where they have courts to enforce such things. ToS can be suitably amended.
David, search engines all obey robots.txt. That is not going to change, no matter how “adversarial” things become. They all also identify themselves when they crawl. That won’t change either.
There are plenty of spambots out there doing all kinds of things, but big public companies like Google and Microsoft are not going to secretly crawl and index sites which are explicitly forbidding them to do so.
They VASTLY overprice the value of the NYT content.
Doesn’t any publisher benefit from google far more than google benefits from any publisher?
This publisher believes that is the case.
A bit off-topic, but maybe somebody could explain how AP works in this context. With many newspapers, most of the national and international news seems to be a barely edited AP feed. There are few substitutes (e.g. Reuters).
How does the presence of this common supplier play into this sort of game? And, how does this depend on the market power (or lack of) that the common supplier has?
It’s an attempt to get to a different market equilibrium. I think News Corp, etc., have an up-hill battle, but it’s not a foregone conclusion. This is especially true as more papers go out of business and the market moves closer to oligopoly. Personally, I like the idea of more free content, but people are entitled to try to make a profit.
I would be interested in more on why this is a false theory. Are you assuming that publishers are in a commodity market?
Blackmail is great business. Just look what Google is doing http://lifehacker.com/5409467/youtube-will-soon-block-access-from-set+top-devices
Comments on this entry are closed.