A superintelligent computer seizing control of the world's nuclear arsenal. Robots for every human need, from housecleaning to filing your taxes. A dystopian future in which social scores determine who you marry, what job you get, and where you live.

These may sound like plots for this year's top science fiction novels, but to the authors reviewed in this essay, they are anything but imagined realities—they are our possible future.

This review tackles introductory reading material about AI from three angles: why AI might destroy humanity, why AI might destroy the economy, and the new who's who of in the international AI race. If you read all three, you will reach expert-level AI doomsayer—just in time for the robot apocalypse.

Superintelligence: Paths, Dangers, and Strategies by Nick Bostrom

I would ask you to imagine a future in which a superinteligent general AI of our own making destroys humanity. Then I would ask you to imagine that particular robot destroying humans every which way. In case you can only dream up a few, have no fear: philopsopher Nick Bostrom has done this thinking exercise for you. One part philosophy text and one part futuristic nonfiction horror essay, Superintelligence: Paths, Dangers, and Strategies lays out the myriad ways humanity might succumb to the superintelligent AI threat. None of them are pleasant.

Methodical if nothing else, he starts with a rousing description of the human species.

“Far from being the smartest possible biological species, we are probably better thought of as the stupidest possible biological species capable of starting a technological civilization—a niche we filled because we got there first, not because we are in any sense optimally adapted to it.” Nick Bostrom, Superintelligence

Bostrom goes on to describe the basis of his theory that superintelligent general AI will wipe out humanity (as opposed to AI designed with a particular purpose in mind—say, chess).

“Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an 'intelligence explosion,' and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.” Nick Bostrom, Superintelligence

Bostrom then identifies the various categories of superintelligent AI and an exhaustive list of theories as to how they might destroy us. Here's one example:

“Consider an AI that has hedonism as its final goal, and which would therefore like to tile the universe with 'hedonium' (matter organized in a configuration that is optimal for the generation of pleasurable experience). To this end, the AI might produce computronium (matter organized in a configuration that is optimal for computation) and use it to implement digital minds in states of euphoria. In order to maximize efficiency, the AI omits from the implementation any mental faculties that are not essential for the experience of pleasure, and exploits any computational shortcuts that according to its definition of pleasure do not vitiate the generation of pleasure. For instance, the AI might confine its simulation to reward circuitry, eliding faculties such as a memory, sensory perception, executive function, and language; it might simulate minds at a relatively coarse-grained level of functionality, omitting lower-level neuronal processes; it might replace commonly repeated computations with calls to a lookup table; or it might put in place some arrangement whereby multiple minds would share most parts of their underlying computational machinery (their 'supervenience bases' in philosophical parlance). Such tricks could greatly increase the quantity of pleasure producible with a given amount of resources.” Nick Bostrom, Superintelligence

But wait—there's more. Superintelligence contains 328 pages of ways AI will one day kill us all. The author does spend some time explaining various control methods, but he caveats most of these with the methods a superintelligent AI would devise to break these controls, destroying us anyway. His ultimate recommendation: research something other than AI, or, at least, put in place plenty of redundant controls along with international agreements about control standards. Maybe throw in some superintelligent humans (genetic engineering and neurotropics come up more than once) and some non-proliferation treaties applied to autonomous weapons.

Bostrom hardly stands alone in his warnings surrounding of superintelligent general AI research. Yoshua Bengio, winner of the 2018 Turing Award with two other researchers, is outspoken about his "killer robots" theory. (You can read Bengio's recent NYT profile here.) Other thought leaders in the space, from the widely famous Stephen Hawking and Elon Musk, to the academically famous Dawn Song, a professor at UC Berkeley, have advised us of the dangers of superintelligent general AI. Today's AI may be used for facial recognition and selling more shoes, but the AI of tomorrow, they caution, threaten humanity's ruin.

Bostrom's critics point out that he largely ignores the many current and future benefits of AI in favor of its potential detriments. While the technology is still nascent, AI could one day empower everything from medical diagnoses and treatment plans to low cost, high quality education, for example. He bases his thesis on his own probability calculations for timelines associated with the development of superintelligent AI and uses this as the basis for his warnings of (relatively, possibly) imminent danger. As evidenced by his description of humanity, he does not really think we have much to work with in terms of "humanitron"-my new term for humanity's building blocks.

Still, Superintelligence is perhaps the best book yet for understanding the potential extreme consequences of AI, consequences that far surpass lost jobs or privacy concerns. Futurists, policymakers, and sci-fi aficionados alike will find something to enjoy in this mind-bending read. If a horrific weekend of robot apolocaylpse-themed nightmares is what you desire, Nick Bostrom's Superintelligence: Paths, Dangers, and Strategies should do the trick. It did for me.

Rise of the Robots: Technology and the Threat of a Jobless Future by Martin Ford

More treatise on the economic perils of AI than a divination of humanity's doom, Martin Ford's Rise of the Robots: Technology and the Threat of a Jobless Future argues that robots are coming for your jobs.

As comedian John Oliver so deftly describes in a recent monologue, we've heard this argument before. In fact, almost every time widespread technological change occurs, doomsayers claim that these advances will destroy the economy. To a certain extent, this is true. How many of your friends own video rental stores, are steadily employed as an elevator operator, or sell books door-to-door?

We've heard the economic argument from the educated elite just as often: although technological advancements have erased these jobs, displaced workers can just find other jobs. They may need more education and, perhaps, career placement assistance. Video rental store owners could become programmers for Netflix, elevator operators could transform into elevator security guards, and book sellers could find employment in Amazon warehouses. Ah, technological progress ushering us into that futuristic utopia!

“...impact of technology on these kinds of jobs, you are very likely to encounter the phrase 'freed up'—as in, workers who lose their low-skill jobs will be freed up to pursue more training and better opportunities. The fundamental assumption, of course, is that a dynamic economy like the United States will always be capable of generating sufficient higher-wage, higher-skill jobs to absorb all those newly freed up workers—given that they succeed in acquiring the necessary training.” Martin Ford, Rise of the Robots

So what's different this time? Couldn't the former factory workers just learn to program software? Ford explains:

“In 2012, Google, for example, generated a profit of nearly $14 billion while employing fewer than 38,000 people. Contrast that with the automotive industry. At peak employment in 1979, General Motors alone had nearly 840,000 workers but earned only about $11 billion—20 percent less than what Google raked in. And, yes, that’s after adjusting for inflation.” Martin Ford, Rise of the Robots

We've heard this before, too, in fewer words: the rich get richer, while the lower and middle classes are displaced. It's just that this time, robots are doing the displacing. Martin offers ample evidence that manual labor jobs are already being impacted.

“Vision Robotics, a company based in San Diego, California, is developing an octopus-like orange harvesting machine. The robot will use three-dimensional machine vision to make a computer model of an entire orange tree and then store the location of each fruit. That information will then be passed on to the machine’s eight robotic arms, which will rapidly harvest the oranges.” Martin Ford, Rise of the Robots

This may not be unexpected. After all, what is a futuristic utopia without manual labor jobs done by robots? Martin's concern is not that jobs are being replaced, however. It's that the accelerated pace of AI-based technologies has amplified their role in reducing and eliminating human employment across blue-collar AND white-collar jobs. Everyone from doctors and paralegals to journalists and computer programmers, he argues, are at risk of predictive algorithms doing their jobs better.

"Radiologists, for example, are trained to interpret the images that result from various medical scans. Image processing and recognition technology is advancing rapidly and may soon be able to usurp the radiologist’s traditional role. Software can already recognize people in photos posted on Facebook and even help identify potential terrorists in airports.” Martin Ford, Rise of the Robots

This, in turn, weakens the underlying principle that drives much of the world's economy: that the middle class will continue to be the bedrock of the economic system. Ford envisions a future where only the elite can afford the goods produced in their factories and rent in major employment centers. If climbing the ladders from the lower to middle class is not enough to make ends meet anymore, what does that mean for our country? Retraining and job placement programs, which were arguably only marginally effective anyway, are not a panacea, Ford argues. Instead, he promotes a wholesale economic overhaul, including reframings of contemporary economic ideas and a guaranteed minimum wage.

For a broad overview on how automation is affecting the economy, Rise of the Robots: Technology and the Threat of a Jobless Future provides a solid primer. While the book offers no cutting-edge theories for technologists, it is a highly approachable entrée into the economic debates surrounding AI and citizens' claims on taxpayer-funded technology development.

AI Superpowers: China, Silicon Valley, and the New World Order by Kai-Fu Lee

No conversation about AI would be complete without mentioning China. From Kai-fu Lee, noted AI expert and Chinese venture capitalist, comes AI Superpowers: China, Silicon Valley, and the New World Order, describing the transformation of the Chinese technology sector and his predictions about how the US and China match up in the AI arms races (both corporate and military). He convincingly argues that the world's technological future may lie not in the US, but in China.

Several countries boast strong AI research groups, think-tanks, and start-ups. The UK has DeepMind Technologies (acquired by Google's parent company, Alphabet, Inc. in 2014). Canada is home to the Montreal Institute for Learning Algorithms, the Institute for Data Valorisation, and the Canadian Institute for Advanced Research, as well as the Canadian government’s Pan-Canadian AI Strategy. India, France, Germany, and Israel have leading university talent pools and a growing number of AI startups, many of which have been acquired by US companies, such as Uber, Google, Apple, and Microsoft.

By all metrics, however, these companies can't compete with the technology centers in the United States. From Bostom and New York to Austin and LA, the US is home to the companies that dominate the AI playing field. Silicon Valley, with its technology giants, direct access to the world's top talent, and an endless appetite for acquisitions, reigns supreme. While this all may be true, Lee points out that this lead may not be permanent.

Those who may believe that the Chinese technology sector is comprised of copycats and IP infringement violations wouldn't be entirely incorrect. The current state is more nuanced, however. Lee offers a closer look at whats going on there.

"...China’s startup culture is the yin to Silicon Valley’s yang: instead of being mission-driven, Chinese companies are first and foremost market-driven. Their ultimate goal is to make money, and they’re willing to create any product, adopt any model, or go into any business that will accomplish that objective. That mentality leads to incredible flexibility in business models and execution, a perfect distillation of the 'lean startup' model often praised in Silicon Valley. It doesn’t matter where an idea came from or who came up with it. All that matters is whether you can execute it to make a financial profit. The core motivation for China’s market-driven entrepreneurs is not fame, glory, or changing the world. Those things are all nice side benefits, but the grand prize is getting rich, and it doesn’t matter how you get there." Kai-Fu Lee, AI Superpowers

From a purely financial perspective, this strategy has paid off.

“Recent estimates have Chinese companies outstripping U.S. competitors ten to one in quantity of food deliveries and fifty to one in spending on mobile payments. China’s e-commerce purchases are roughly double the U.S. totals, and the gap is only growing. Data on total trips through ride-hailing apps is somewhat scarce, but during the height of competition between Uber and Didi, self-reported numbers from the two companies had Didi’s rides in China at four times the total of Uber’s global rides. When it comes to rides on shared bikes, China is outpacing the United States at an astounding ratio of three hundred to one.” Kai-Fu Lee, AI Superpowers

Given this, it may be no surprise that Chinese companies Baidu, Tencent, and Alibaba now count among the world's top corporations investing in AI. The Chinese government is also investing heavily. In 2017, it announced that it wants the country to be the world leader in AI by 2030 and has created a $30 billion venture fund for it. Beijing, Tianjin, and other cities have committed billions to the effort. Contrast this with, for example, DARPA's $2 billion "AI Next" fund.

“Behind these efforts lies a core difference in American and Chinese political culture: while America’s combative political system aggressively punishes missteps or waste in funding technological upgrades, China’s techno-utilitarian approach rewards proactive investment and adoption.” Kai-Fu Lee, AI Superpowers

Money is not the only factor in this race. Lee points out that for the AI to reach the next level of general intelligence and, eventually, general superintelligence, access to vast quantities of data is necessary. Google, Amazon, and Facebook may have been the data leaders of the past, but Chinese companies are possibly the future. Chinese firms have fewer privacy concerns and are more directly linked to the state. This gives them access to data from private citizens that is, frankly, stuff of science fiction. Take, for example, China's AI-driven credit score systems that factor social behaviors and buying patterns for its citizens, Black Mirror-style. In one pilot, citizens' scores drop for traffic violations and non-payment of fines, and increase for charitable donations. Other pilots link social scores to dating profiles so that users can judge each other on looks as well as social credit ratings. Despite its dystopian undertones, the data generated could provide a significant leg up on US-based deep learning companies who will not have the same breadth and depth of information (we hope).

While I have no doubt that Lee's theory that its time as the world's technological leader may be soon over for the US, I do not share his optimism that China will carry the leadership mantle admirably. Lee's outlook comes at least in part from national pride and his bias is reflected throughout the book. This does not mean we should ignore his perspective. AI Superpowers: China, Silicon Valley, and the New World Order provides valuable insight into both Chinese and US strategies for AI dominance and the implications for our future.

Inspired by this review? Support The Indent by buying a copy of these books using the Amazon Affiliate links below.