The off-switch game framework, in which the human would turn the waiting agent off if the action would be harmful, has been instrumental in understanding corrigibility — the property that AI agents should allow human oversight and intervention. In single-agent settings, uncertainty about human preferences naturally incentivizes agents to defer to human judgment. However, as AI systems increasingly operate in multi-agent environments, a crucial question arises: does corrigibility compose across multiple agents? This paper introduces the multi-agent off-switch game and demonstrates that individually corrigible agents can become collectively incorrigible when strategic interactions are considered. Through formal analysis and illustrative examples, corrigibility is shown to be non-compositional, and conditions are identified under which group incorrigibility emerges. These results highlight fundamental challenges for AI safety in multi-agent settings and suggest the need for new approaches that explicitly address collective dynamics. Multi-principal settings, in which several human overseers delegate tasks to agents, are likely to raise even greater challenges than the single-principal case already studied, and the most promising response is mechanism design for corrigibility: architectural choices that ensure approximate additivity of utilities, reward structures that internalize externalities between agents' actions, or explicit incentives for agents to coordinate on waiting.
Consider (in)corrigibility (an agent allowing human oversight and being willing to be shut down) in multi-principal settings, where each principal delegates tasks to one or more agents and more complicated game-theoretic dynamics come into play. Design coordination mechanisms that preserve corrigibility under non-additive utilities.