Using a proof by contradiction, the paper argues that it is impossible to precisely and consistently predict what specific actions a smarter-than-human intelligent system will take to achieve its objectives, even if we know terminal goals of the system. Unpredictability implies that standard tools and safety mechanisms would not work to make advanced intelligent systems safe to use, including legislative control, which is unlikely to appropriately encourage or deter the effects of algorithms. While Unpredictability will forever make 100% safe AI an impossibility, we can still strive for Safer AI because we are able to make some predictions about the AIs we design, for example about the universal instrumental goals on which all sufficiently intelligent agents converge.
Figure out what is in fact predictable and knowable about intelligent machines, even if most of their future states will forever be unpredictable.