Yuval Levin makes a few points of relevance:
…many of Medicare’s most significant administrative costs are just covered by other federal agencies, and so don’t appear on Medicare’s particular budget, but are still huge costs of the program. The IRS collects the taxes that fund the program; Social Security collects many of the premiums paid by beneficiaries; HHS pays for a great deal of what you would think of as basic overhead, but doesn’t put it on the Medicare program’s budget. Obviously private insurers have to pay for such things themselves. Medicare’s administration is also exempt from taxes, while insurers pay an excise tax on premiums (which is counted as overhead). And private insurers also spend a great deal of money fighting fraud, while Medicare doesn’t. That might reduce the program’s administrative costs, but it greatly increases its overall costs. Some administrative costs save money, after all: The GAO has estimated that a $1 investment in pre-payment review of claims, for instance, would save $21 in improper Medicare payments.
My original source is this Reihan Salam post. I also would add the deadweight cost of taxation. Arguably that does not count as a cost of “overhead,” but very often it runs 20% or more and still it is a cost.