The long-held nightmare of machines controlling man, widely believed to exist only in science fiction, has come to digital life.
It is not happening in the kind of futuristic urban megalopolis beloved by Sci-Fi movie directors – but on a dusty frontier in China.
Its chronicler, American Geoffrey Cain, is a tech author. However, the genesis of his latest book was not technology per se, but the interconnections between technology, politics and geography.
Though it is a crossroads where multiple cultures, religions and political systems have interfaced, interconnected, clashed and traded, the geography in question is little known in the West.
“What always interested me was the Eurasian region – from Xian to Eastern Europe and the Middle East – where the Silk Roads ran,” Cain, whose new book The Perfect Police State was published this week, told Asia Times in an interview.
As a space where trade was done and power projected, “it far predated Euro-Atlanticism and the seafaring powers,” the Washington DC-based Cain said.
Bad news from Tartary
Having “always loved reading about explorers” his mind was particularly gripped by the classic political travelogue published in 1936 by Peter Fleming: News from Tartary. That covered a seven-month Odyssey that Fleming – a British adventurer, writer, soldier and World War II spook – took from Beijing to Kashmir.
“In his overland journey, by train and foot, he runs across bandits and has to hunt and figure out how to survive in this warlord land,” Cain enthused. “It’s a real adventure.”
“Tartary” is now “Central Asia” and is more politically stable than in Fleming’s time. As key natural resources come online in the ‘Stans, and China continues to beef up its Belt and Road infrastructure, accelerating links between East and West, long-dormant caravan routes are back with a vengeance.
So, too, are political risks.
“This region – with the new Silk Road, oil, gas and rare earths, bordering China and Russia and the Middle East – is where the ‘Great Games’ are stirring up again,” Cain said. “This is the region where we can feel the pulse of where the great powers stand.”
The withdrawal of the US and its allies from Afghanistan looks set to create a vacuum that looks likely to generate new risks and opportunities for regional powers.
These powers, engaging in competing spheres of interest and influence, include the fast ascending China, the religiously fixated Iran and Saudi Arabia, the post-Soviet Russia and the neo-Ottoman Turkey.
But as a tech specialist, Cain was eyeing a very particular chunk of the region: China’s northwest province of Xinjiang.
“News from Tartary was my starting point, but I wanted to know how this region of Xinjiang went from being a battleground between warlords and military commanders of all stripes to a total surveillance dystopia,” he said.
The tech-power interface
Xinjiang is where under-tested technologies have been co-opted and integrated by an authoritarian state to create a powerful digiscape that is, in essence, a petri dish for total control.
“I had been following the situation there, as I had always been interested in Central Asia and as a tech writer I could see advances in AI and facial recognition were being rolled out in Xinjiang in 2017 and 2018 in ways we had not seen before,” Cain recalled. “I traveled there in 2017, seeing cameras covering every square meter and police pillboxes on every corner, being constantly watched.”
The end result was The Perfect Police State, which followed closely on the heels of the publication of his first book, Samsung Rising, which hit the shelves in March 2020.
What themes link the two?
“The question is, ‘How does a society look at technology, and how does a society use technology to govern a society?’ – or the reverse,” Cain said.
Samsung was one of the firms favored by authoritarian governments in the 1960s, 70s and 80s to build the state. In that process, the corporate became an entrenched dynasty that now outlasts political administrations.
“Samsung Rising is a story of anti-trust, where a company becomes so powerful that it starts to define what it means to be a nation – the ‘Republic of Samsung,’” the author said.
The massive power wielded by Samsung within South Korea in fields far beyond commerce may surprise readers unfamiliar with the country. After all, for the average non-Korean consumer, Samsung is an unalloyed success story.
It has overtaken rivals Sony and Apple in most metrics of size and success and currently looks bulletproof in terms of its technological portfolio – memory chips, non-memory chips and devices.
Yet the powers Samsung wield fall far short of the powers detailed in Cain’s latest work.
The Perfect Police State also deals with a rising state and its use of technology. However, the aim is to expand control rather than to expand the economy.
In Xinjiang, a total control network has been emplaced. It interfaces social media and telecommunications monitoring with a vast network of CCTV cameras – some set up inside Uighur homes. These cameras are empowered by facial recognition technologies, and the entire, hugely invasive digital network is roved and overseen by unblinking AI.
On the ground, it is underwritten by a massive police presence, fully equipped with technological surveillance and scanning tools, and a network of “vocational” camps.
“It is a story about the rise of increasingly authoritarian state power, combined with unprecedented advances in technology,” Cain summarized.
Cain grants the Chinese state some credence for its fear of Islamic extremism.
“There are actual villains out there who want to use terrorist violence to undermine the Chinese state and want to turn Xinjiang into a caliphate,” he said.
But like the West – which has caused massive death and destruction across the Middle East and Central Asia, thereby destabilizing societies and incubating new terrorist threats in its “War on Terror” – Beijing is arguably hefting a sledgehammer to smash a nut.
“China is responding in a way that is so out of hand and so disproportionate, and this has caused it to get worse,” Cain said.
And it is not just China. Components of the system are being employed beyond Chinese borders.
“Chinese companies, with the backing of the party, are trying to sell these technologies to Central Asia and Africa and Latin America,” he said. “Once these governments have these tools in their hands, there is not much that anyone can do to stop them [from] abusing them.”
Tech East, Tech West
Political influence is just one of multiple areas where China and the US are in headlong competition.
“I don’t think China has much soft power but their proposition to the world is, ‘We have a system that will enrich you quickly, will allow you to accrue wealth at a faster pace than if you tried to reform and embrace the Western model, and favors the governors over the governed,’” Cain said.
Cain, who has advised governments on high tech during both the Obama and Biden administrations, was – perhaps surprisingly – in favor of some of the Trump administration’s tactics against China.
“He was depicted as a blustering, war-mongering China basher, but I think the actions by the US Commerce Department pointed at the nerve points,” he said. “Semiconductors are huge, and despite massive government support, China has not been able to build a supply chain to sustain a semiconductor industry.”
Citing alleged panic in Huawei’s executive suite once chip sanctions were announced, Cain said: “China is going to be hit hard and it is going to see volatility in supply chains that will impact China’s ability to grow as an industrial and military power.”
Yet, China – overseeing a state-led, East Asian model of dirigiste industrial policy – is a force to be reckoned with. Particularly given how the US has lost some of its technological edge in recent years.
“US long-term tech innovation has slowed. We have exited the age of the moon landing, of vast imagination,” he said. “The last big US innovation was the I-phone.”
That led to a shortening of innovators’ horizons. “From the business standpoint, it makes sense just to take advantage of the smartphone’s perfect scalability – there is the potential that 8 billion people around the world can get your app, so that is where the venture capital goes.”
As a result, Silicon Valley is unwilling to invest in hardware. “The VC guys are, “Are you kidding me? Hardware is made in China or Vietnam,’” Cain said.
However, America is unlikely to head in the direction taken by Asia’s industrial powerhouses.
“The Japanese and Korean models worked great to move fast in the world of technology and nation-building, but that model rested on these tight-knit relations, of corruption, and of company loyalist models that depended on the exclusion of women and minorities,” he said. “I don’t think our system allows for ‘economic miracles.’”
Controlling control tech
Meanwhile, surveillance is a political control tool as old as politics itself. And it is not only China that has leveraged IT in its surveillance efforts.
“Long before this happened in China, there were revelations under George Bush that US intelligence agencies, with the cooperation of AT&T and other companies, were tapping communications in the ‘War on Terror,’” he said. “Ultimately we all live on the grid.”
Though personal technologies may appear to the consumer to be bottom-up, their deployment requires careful regulation, top-down.
“In the past 10 to 20 years we have been researching the way technologies impact our lives and security,” Cain said. “The key is how our governments manage these systems.”
Still, Cain spoke approvingly of the current direction of US anti-trust regulation to break up powerful social media platforms like Facebook.
Previously, the key questions were the prices consumers paid due to anti-competitive activities, but social media shot back with the argument that their services are free. The current discourse is moving past this paradigm, making Cain cautiously optimistic.
“In the West, we have a vibrant discussion going on, the Biden administration is taking notice and I think we are talking about it in a way that protects our democracy,” he said.
But there will always exist personalities and arms of government that aim to use technologies to the ends that China seeks in Xinjiang. These under-the-radar activities are enabled by the intrinsic difficulty of predicting the multifaceted usages of personal technologies.
“Societies can’t work out how to manage it, it is deployed before we realize how it works,” Cain warned. “Law enforcement or the military may want to use it, but they are opening a Pandora’s Box and there is no going back. This is about ensuring our technological progress is not turned against us.”
Pandora’s Digital Box
While researching and writing The Perfect Police State, News from Tartary was soon subsumed in Cain’s thoughts by another book: George Orwell’s dystopian 1948 novel of authoritarianism and state control, 1984.
One of Cain’s interviewees, “a very sharp-witted individual who wanted to be a diplomat, whipped out a copy and said, ‘I cannot understand how an Englishman could write about my life in 1948.’”
Tellingly she was plunged into a re-education camp due to a glitch in the AI system. Many red lines have already been crossed.
This raises the ironic possibility that control technologies could lead to the kind of carnivorous decontrol that devoured so many lives in Stalin and Mao’s worst periods.
“The Chinese Communist Party has seen a potential solution to create a total security state and has rushed into it without a full grasp of how it works,” Cain said. “Based on my own interviews with tech workers, they think the system does not work that well and the fear is that the party has unleashed an AI monster that they can’t put a lid back on.”
Seen through this prism, the machinery looming over Xinjiang has a dark potential that might even be generating shudders in Beijing’s corridors of power.
“This is now a system where everyone is a target and that system has the potential to consume itself and destroy the lives of people who have invested – like the Great Terror or the Cultural Revolution,” Cain warned.
By creating technologies that can think and learn, and unleashing them within the vast digital landscape that is now so critical to all aspects of our lives, mankind has set foot on uncharted territory.
“The terrifying aspect is that our technologies have got so advanced,” Cain said. “We don’t know what the implications are once you add AI to the picture.”