Can we actually control superintelligent AI? | Ada, Ep. 4

Stuart Russell, author of a new book on AI and Professor of Computer Science at UC Berkeley, is interested in seeing AI be directed towards furthering human interests. Although on average, regulations tend to take decades to catch up with new technology, Russell argues there must be rules against the release of unsafe AI if we are to avoid the technology effectively taking over life as we know it. For AI to be safe and helpful, it must be programmed with the right goals in mind. But what might those goals be? Russell talks about the King Midas problem here— wherein a program carries out its objective to the letter but that objective ends up being destructive.

Besides being programmed with potentially destructive goals with unforeseen consequences, AI also develop instrumental goals, or goals which are derived automatically from the original goal. An AI system might deduce an instrumental goal of self-preservation or acquiring more power over its environment, for example, in order to serve its original goal more completely.

Russell argues that to protect against these outcomes, we should program AI models with a better idea of what humanity needs and wants, but also with more uncertainty about the rest of existence. A more cautious and uncertain AI model would, in a backwards way, be a more helpful AI model.

For Mustafa Suleyman's perspective on the future of AI, watch this TED Talk. For an article on how AI could potentially unlock solutions to humanity's most pressing problems, read on here.