I’m an AI expert: Here’s my worst-case scenario

Rapid advances in artificial intelligence (AI) over the past decade have been accompanied by several high-profile failures, highlighting the importance of ensuring that intelligent machines are beneficial to humanity. This realization has given rise to the new subfield of research known as AI safety and security, which encompasses a wide range of research areas and has seen a steady growth in publications in recent years. The underlying assumption in this research is that the problem of controlling highly capable intelligent machines is solvable. But no rigorous mathematical proof or argumentation has been presented to demonstrate that the AI control problem is solvable in principle, let alone in practice. In computer science, it is standard to first determine whether a problem belongs to a class of “unsolvable” problems before investing resources in trying to solve it.  Despite the recognition that the problem of AI control may be one of the most important problems facing humanity, it remains poorly understood, poorly defined and poorly researched. A computer science problem could be either solvable, unsolvable, undecidable or partially solvable. But we don’t know the actual status of the AI control problem. It is possible that some forms of control may be possible in certain situations, but it is also possible that partial control may be insufficient in many cases. Without a better understanding of the nature and feasibility of the AI control problem, it is difficult to determine an appropriate course of action.  The AI control problem  We define the problem of AI control as: How can humanity remain…I’m an AI expert: Here’s my worst-case scenario