Artificial Intelligence and the Manufacturing of Reality


【标签】: {{b}}
【来源】: 兰德公司
【时间】: 2020-04-10


原文链接: {{d.URL}}
In 2016, a third of surveyed Americans told researchers they believed the government was concealing what they knew about the “North Dakota Crash,” a conspiracy made up for the purposes of the survey by the researchers themselves. This crash never happened, but it highlights the flaws humans carry with them in deciding what is or is not real.
The internet and other technologies have made it easier to weaponize and exploit these flaws, beguiling more people faster and more compellingly than ever before. It is likely artificial intelligence will be used to exploit the weaknesses inherent in human nature at a scale, speed, and level of effectiveness previously unseen. Adversaries like Russia could pursue goals for using these manipulations to subtly reshape how targets view the world around them, effectively manufacturing their reality. If even some of our predictions are accurate, all governance reliant on public opinion, mass perception, or citizen participation is at risk.

One characteristic human foible is how easily we can falsely redefine what we experience. This flaw, called the Thomas Theorem, suggests, “If men define situations as real, they are real in their consequences.”[1] Put another way, humans not only respond to the objective features of their situations but also to their own subjective interpretations of those situations, even when these beliefs are factually wrong. Other shortcomings include our willingness to believe information that is not true and a propensity to be as easily influenced by emotional appeals as reason, as demonstrated by the “North Dakota Crash” falsehood.[2]

Machines can also be taught to exploit these flaws more effectively than humans: Artificial intelligence algorithms can test what content works and what does not over and over again on millions of people at high speed, until their targets react as desired.[3]

Consider the role of Russia in the 2016 British vote to withdraw from the European Union (Brexit). There is evidence that Russian-linked Twitter accounts sent more than a thousand tweets from 3,800 accounts promoting a pro-Brexit vote on the day of voting. Such tweets appeared to have fanned the flames of the pro-Brexit camp on Twitter over time, with the anti-Brexit camp reacting a few days before the election.[4]
Emerging technologies including generative adversarial networks, natural language processing, and quantum computing could make such scenarios far more effective. In the future, for example, Russian actors could tailor the messages in these tweets using a combination of the characteristics of the recipients and their behaviors on various online platforms based on user data they legally buy from data brokers, illegal data they purchase from hackers, and data they retrieve themselves.
These opportunities are available today, and some may become increasingly easier to exploit with artificial intelligence. In the future, for example, adversaries like Russia could query these data streams to tailor their messages and test them on social media platforms to identify the most effective messages. Such adversaries could then alter these tested messages accordingly and deploy them to users and those in their social networks via a wide variety of online media (e.g., traditional social media platforms, augmented reality, virtual reality devices, or noninvasive brain-computer interfaces).
Currently, much of this is done manually, but artificial intelligence allows for a change in scale. Emerging technologies will allow this loop of iterative refinement to occur almost instantaneously, in real time, and at a scale affecting far more people than was seen during the Brexit vote.[5]
Manufacturing Reality Isn't New
Humans are and have always been vulnerable to being tricked, provoked, conditioned, deceived, or otherwise manipulated. Since at least the 1960s, the Soviet military and subsequent Russian organizations recognized opportunities for exploiting this vulnerability. That is why the Soviets developed a formal research program—called reflexive control theory—to model how one could manipulate targets' perceptions of reality.[6] The theory focuses on ways to strategically transmit information to targets in ways that subtly change the motives and logics of their decisions. The end goal is to get people to do something by making them believe it is in their best interest, even if it is not. While the Russians weaponized reflexive control theory, Madison Avenue used similar logic to evoke emotion—and sell products to American consumers.
There are countless examples of the Russians using reflexive control. In October 1993, for example, Russian lawmakers took over their own Parliament to advocate for a return to communism. The authorities decided to allow the rebels to occupy a police communications post, giving them access to a secure communications channel then used by police to transmit false conversations between government officials about a plan to storm the occupied Parliament building. After hearing this message, one of the rebel leaders, Parliament Speaker Ruslan Kashbulatov, called on the crowd of supporters to seize a local television station, the first step in a coup. By getting Kashbulatov to make this public request for violence, Russian authorities created justification for storming the Parliament and arresting the dissidents.
Similarly, the East Germans recognized the power of manufactured reality for maintaining internal control. Starting around the 1970s, their Ministry of State Security known as the Stasi expanded the scope of their work from physical abuse of targets—such as torture or executions—to include a certain kind of psychological abuse. The Stasi called this technique Zersetzung, which loosely translates as “decomposition.” It was an organized, scientific effort to collect information about people and then use it in ways that destroyed their sense of self in private and public life. The Stasi broke into the homes of targets to rearrange their furniture, steal items of clothing, or turn off their clocks. They would send compromising photos to loved ones, discredit people in their workplace, remove children from dissident parents or trick them into believing they were mentally ill, something known today as gaslighting. Victims of decomposition struggled to understand why their lives were becoming unrecognizable.
But these Russian and Stasi tactics required careful research and execution to disrupt or manipulate the targets one at a time. The contemporary information environment and modern tools, including artificial intelligence, could slash the transaction costs of such manipulation. The following methods enable a dramatic scaling in the weaponization of information.