Ideas for regulating AI safety

Noting these come from Luke Muelhhauser, and he is not speaking for Open Philanthropy in any official capacity:

  1. Software export controls. Control the export (to anyone) of “frontier AI models,” i.e. models with highly general capabilities over some threshold, or (more simply) models trained with a compute budget over some threshold (e.g. as much compute as $1 billion can buy today). This will help limit the proliferation of the models which probably pose the greatest risk. Also restrict API access in some ways, as API access can potentially be used to generate an optimized dataset sufficient to train a smaller model to reach performance similar to that of the larger model.
  2. Require hardware security features on cutting-edge chips. Security features on chips can be leveraged for many useful compute governance purposes, e.g. to verify compliance with export controls and domestic regulations, monitor chip activity without leaking sensitive IP, limit usage (e.g. via interconnect limits), or even intervene in an emergency (e.g. remote shutdown). These functions can be achieved via firmware updates to already-deployed chips, though some features would be more tamper-resistant if implemented on the silicon itself in future chips.
  3. Track stocks and flows of cutting-edge chips, and license big clusters. Chips over a certain capability threshold (e.g. the one used for the October 2022 export controls) should be tracked, and a license should be required to bring together large masses of them (as required to cost-effectively train frontier models). This would improve government visibility into potentially dangerous clusters of compute. And without this, other aspects of an effective compute governance regime can be rendered moot via the use of undeclared compute.
  4. Track and require a license to develop frontier AI models. This would improve government visibility into potentially dangerous AI model development, and allow more control over their proliferation. Without this, other policies like the information security requirements below are hard to implement.
  5. Information security requirements. Require that frontier AI models be subject to extra-stringent information security protections (including cyber, physical, and personnel security), including during model training, to limit unintended proliferation of dangerous models.
  6. Testing and evaluation requirements. Require that frontier AI models be subject to extra-stringent safety testing and evaluation, including some evaluation by an independent auditor meeting certain criteria. [footnote in the original]
  7. Fund specific genres of alignment, interpretability, and model evaluation R&D. Note that if the genres are not specified well enough, such funding can effectively widen (rather than shrink) the gap between cutting-edge AI capabilities and available methods for alignment, interpretability, and evaluation. See e.g. here for one possible model.
  8. Fund defensive information security R&D, again to help limit unintended proliferation of dangerous models. Even the broadest funding strategy would help, but there are many ways to target this funding to the development and deployment pipeline for frontier AI models.
  9. Create a narrow antitrust safe harbor for AI safety & security collaboration. Frontier-model developers would be more likely to collaborate usefully on AI safety and security work if such collaboration were more clearly allowed under antitrust rules. Careful scoping of the policy would be needed to retain the basic goals of antitrust policy.
  10. Require certain kinds of AI incident reporting, similar to incident reporting requirements in other industries (e.g. aviation) or to data breach reporting requirements, and similar to some vulnerability disclosure regimes. Many incidents wouldn’t need to be reported publicly, but could be kept confidential within a regulatory body. The goal of this is to allow regulators and perhaps others to track certain kinds of harms and close-calls from AI systems, to keep track of where the dangers are and rapidly evolve mitigation mechanisms.
  11. Clarify the liability of AI developers for concrete AI harms, especially clear physical or financial harms, including those resulting from negligent security practices. A new framework for AI liability should in particular address the risks from frontier models carrying out actions. The goal of clear liability is to incentivize greater investment in safety, security, etc. by AI developers.
  12. Create means for rapid shutdown of large compute clusters and training runs. One kind of “off switch” that may be useful in an emergency is a non-networked power cutoff switch for large compute clusters. As far as I know, most datacenters don’t have this.[6] Remote shutdown mechanisms on chips (mentioned above) could also help, though they are vulnerable to interruption by cyberattack. Various additional options could be required for compute clusters and training runs beyond particular thresholds.

I am OK with some of these, provided they are applied liberally — for instance, new editions of the iPhone require regulatory consent, but that hasn’t thwarted progress much.  That may or may not be the case for #3 through #6, I don’t know how strict a standard is intended or who exactly is to make the call.  Perhaps I do not understand #2, but it strikes me as a proposal for a complete surveillance society, at least as far as computers are concerned — I am opposed!  And furthermore it will drive a lot of activity underground, and in the meantime the proposal itself will hurt the EA brand.  I hope the country rises up against such ideas, or perhaps more likely that they die stillborn.  (And to think they are based on fears that have never even been modeled.  And I guess I can’t bring in a computer from Mexico to use?)  I am not sure what “restrict API access” means in practice (to whom? to everyone who might be a Chinese spy? and does Luke favor banning all open source? do we really want to drive all that underground?), but probably I am opposed to it.  I am opposed to placing liability for a General Purpose Technology on the technology supplier (#11), and I hope to write more on this soon.

Finally, is Luke a closet accelerationist?  The status quo does plenty to boost AI progress, often through the military and government R&D and public universities, but there is no talk of eliminating those programs.  Why so many regulations but the government subsidies get off scot-free!?  How about, while we are at it, banning additional Canadians from coming to the United States?  (Canadians are renowned for their AI contributions.)  After all, the security of our nation and indeed the world is at stake.  Canada is a very nice country, and since 1949 it even contains Newfoundland, so this seems like less of an imposition than monitoring all our computer activity, right?  It might be easier yet to shut down all high-skilled immigration.  Any takers for that one?


Comments for this post are closed