“AI offers an illusion of cool exactitude, especially in comparison to error-prone, potentially unstable humans. But today’s most advanced AIs are black boxes; we don’t entirely understand how they work. In complex, high-stakes adversarial situations, AI’s notions about what constitutes winning may be impenetrable, if not altogether alien. At the deepest, most important level, an AI may not understand what Ronald Reagan and Mikhail Gorbachev meant when they said, ‘A nuclear war cannot be won.’”
By Ross Andersen – THE ATLANTIC | May 4, 2023 rsn.org
The temptation to automate command and control will be great. The danger is greater.
No technology since the atomic bomb has inspired the apocalyptic imagination like artificial intelligence. Ever since ChatGPT began exhibiting glints of logical reasoning in November, the internet has been awash in doomsday scenarios. Many are self-consciously fanciful—they’re meant to jar us into envisioning how badly things could go wrong if an emerging intelligence comes to understand the world, and its own goals, even a little differently from how its human creators do. One scenario, however, requires less imagination, because the first steps toward it are arguably already being taken—the gradual integration of AI into the most destructive technologies we possess today.