Flagged as a formal argument against the uncontainability case.
There is considerable uncertainty about what properties, capabilities and motivations future AGIs will have. In some plausible scenarios, AGIs may pose security risks arising from accidents and defects. In order to mitigate these risks, prudent early AGI research teams will perform significant testing on their creations before use. Unfortunately, if an AGI has human-level or greater intelligence, testing itself may not be safe; some natural AGI goal systems create emergent incentives for AGIs to tamper with their test environments, make copies of themselves on the internet, or convince developers and operators to do dangerous things. This paper surveys how to build a container in which tests can be conducted safely and reliably, even on AGIs with unknown motivations and capabilities that could be dangerous. The paper identifies requirements for AGI containers, available mechanisms (language semantics, airgapping, operating system sandboxing, virtualization), and weaknesses that need to be addressed. Less-burdensome containment mechanisms could still meaningfully mitigate the risks of AGI development, making the investigation of containment solutions a timely and impactful research opportunity, and containment technologies should be developed now rather than later so that AGI researchers have time to integrate them with their own infrastructure and workflows, influence standards, and let the security software mature without the pressure of needing it to be perfect on the first try.
Implement the outlined approach to the AGI containment problem (how to build a container in which tests can be conducted safely and reliably) — layering existing mechanisms (airgapping, operating system sandboxing, virtualization) with intrusion detection — which seems feasible but requires additional work to actually implement. Pursue the project ideas laid out (code integrity enforcement, adapting off-the-shelf software, hardening AGI infrastructure and debuggers, red-team events); the earlier this work begins, the more opportunity there will be for unforeseen difficulties to surface, and for any resulting security software to mature.