China Battles AI Misinformation Amidst Disasters
As China grapples with the devastating aftermath of natural calamities like Typhoon Noul and severe flooding incidents over recent months, authorities face an unprecedented challenge: a surge of dangerously misleading videos created with artificial intelligence (AI). These sophisticated fabrications are inundating social media platforms, creating panic, eroding public trust, and complicating emergency response efforts, officials and experts report.
The misleading content ranges widely, including fabricated images of deceased individuals floating in floodwaters, false depictions of severe flooding in unaffected areas, and blatantly fake details about emergency response operations. In some instances, videos containing erroneous information about power outages have prompted residents to panic-buy emergency supplies, further straining resources and exacerbating an already tense situation, according to Chinese state media.
Beyond entirely fabricated scenarios, some AI-generated content capitalizes on real-life incidents. For example, severe flooding in Hengzhou city, Guangxi, in early July led to the escape of hundreds of snakes from a breeding farm. Shortly thereafter, fake images surfaced, purporting to show crocodiles being released into a river, intensifying public alarm. Amid mass evacuations and complex rescue operations, authorities are desperate to prevent panic, but fear that AI technology—an area where China is investing billions of dollars—is now providing accessible tools to trick the public.
"Many of these content creators use disaster news to try to gain followers, knowing that more sensational or exaggerated content will garner more likes and more views. They are deliberately fabricating information to grab people's attention," Huang Zhihua, Deputy Chief in charge of online public security for the Zhejiang Police Department, told Chinese state media. The motivations behind such malicious content vary, from those seeking to "create something amusing without thinking about the consequences" to those aiming to "generate traffic for personal gain" or even deliberately "cause panic or social unrest," Professor Gao Fuping from the East China University of Political Science and Law explained to the BBC.
Professor Chen Bin, writing in the newspaper of the Communist Party's Central Party School, highlighted the core technical shift: "AI dramatically lowers the technical barriers to producing false information. Individuals no longer require sophisticated technical skills. By simply prompting AI tools, they can rapidly generate convincing text, images, audio and video... which is highly realistic but entirely fabricated content, manufactured on a large scale." This ease of production, coupled with the potential for rapid dissemination, represents a significant cybersecurity risk to public information integrity.
China's Dual AI Ambition and Challenge
By Decode Today News
China has long embraced AI as a cornerstone of its technological and economic future. In 2017, the government declared AI the "main driving force" for national progress. This ambition has led to vast sums of capital flowing into AI businesses, fueling intense domestic competition. Xinhua, the state-controlled wire service, reports that China now hosts over 6,000 "AI companies," signifying a robust AI infrastructure and significant market valuation potential.
However, this rapid advancement has created a paradox: the very technology poised to propel China to superpower status is now being weaponized to destabilize its social fabric during crises. Professor Xu Xiaoke from Beijing Normal University noted to the BBC that detection technology significantly lags behind generation technology, describing the situation as a "constant cat-and-mouse game." He cautioned against assuming technology alone could identify AI-generated content "with the click of a button," underscoring the formidable compliance security challenge.
In response, Professor Xu suggests China needs to build "trusted information infrastructure," including official government early warning systems and disaster interfaces accessible to news organizations. However, this recommendation comes with its own complexities, given instances of official sources, including government-controlled media, deliberately downplaying disasters in the past.
Understanding China's Digital Information Landscape
The proliferation of AI-generated misinformation is exacerbated by a pre-existing "trust vacuum" in China. There have been several documented cases where local officials were accused of concealing or underreporting disaster casualties. Earlier in July, local journalists were reportedly asked to "soften" their language, describing a major breach in the wall of the Liulan Dam near Naning as an "opening" or "gap." Despite attempts to downplay the severity, water bursting through the dam led to at least 26 deaths downstream.
Similarly, in 2021, local officials faced arrest for allegedly concealing or underreporting 139 deaths in Henan province. Incidents like these have fostered public mistrust, making it easier for bad actors to spread fake content during emergencies. This erosion of confidence presents a critical obstacle for official communication strategies, particularly when attempting to counter sophisticated AI-generated narratives.
Authorities are now imploring the public to rely on accredited sources. Gao Kai from the Ningbo Public Security Bureau urged the public: "When encountering disaster-related information from an unknown source, approach it with scepticism and rely on official announcements." He also emphasized that the "internet is not a lawless zone," assuring that creators of false information would be caught.
Crackdown and Future Outlook
In recent weeks, Chinese authorities have intensified their efforts against misinformation. Arrests have been made, and punishments, ranging from detention awaiting criminal prosecution to fines, have been meted out for creating bogus reports and video content. Police have established a "specialised task force to monitor social media closely, especially short video platforms," tasked with prompt investigation and strict action. Penalties for serious misinformation can include fines or up to seven years in prison.
On July 23, China's Cyberspace Administration announced a nationwide crackdown on fake internet content related to floods and other disasters. This directive ordered local branches to specifically target:
- Fabricated data
- Malicious editing
- Staged scenes
- Misattributed historical events
- Impersonations of officials
Professor Gao Fuping believes that "the answer is public education combined with law enforcement," advocating for publicizing real cases where individuals have been punished. He asserted that the "harm caused by AI-generated fake content is ultimately the result of how people choose to use the technology - not the technology itself," suggesting that existing laws might be sufficient to address the issue.
However, not all experts share this optimism. Professor Chen Bin pointed out that "AI technology continues to evolve much faster than legal frameworks," creating new legal questions and ensuring that AI misinformation remains a considerable challenge. Peter Pang, a Shanghai and Washington-based lawyer, highlights an "explosion in the number of cases" dealing with these issues globally. He argues that companies creating AI applications and platforms used to publish fake material must be held legally accountable alongside the content creators.
Pang used a hypothetical example of a pedestrian being run over due to panic induced by a fake social media post, stating, "The video creator might say 'you ran over somebody. That's your problem. I didn't do it.' But let's think it through. You created the foundation from which people reacted. If you created that panic, you need to have responsibility for consequences that occur." This perspective signals a growing legal and ethical debate around the responsibility of tech companies in the age of generative AI.
Professor Xu Xiaoke warns that AI-generated misinformation is likely to increase not only in volume but also in its speed of spread. He projects that "political elections, public health emergencies, armed conflicts, natural disasters and issues affecting people's livelihoods are all likely to become high-risk areas for AI-generated misinformation." His crucial advice is for governments and platforms to "now be preparing in advance" instead of waiting for false information to go viral before attempting to contain it, highlighting the urgent need for robust pre-emptive strategies against a rapidly evolving threat to informational integrity and public safety.
