Here is a must-read post from StatsGuy, via James Kwak. I don't agree with everything he says (e.g., Singapore, Wagner's Law) but here is his conclusion:
The Heritage Freedom Index is really a composite of measures that get at two different things: Good Government, and Less Government. Overall, the Good Government factors tend to dominate, and drive a lot of the correlation with good economic and quality of life outcomes. When one splits out the factors, the case for Less/Weaker Government weakens substantially, and the case for Clean/Non-Corrupt/Efficient government strengthens considerably.
Addendum: Here is a related discussion, especially in the comments.