AI-Generated Fake Content Surges on X Amid Iran-US Tensions, Despite Platform Policies

Kathmandu. Fake images and videos generated by Artificial Intelligence (AI) regarding the Iran-US conflict are rapidly spreading on social media, especially on X (formerly Twitter). Despite the X platform announcing strict policies to curb misinformation, the dissemination of deceptive content related to the war has not stopped.

According to researchers, there has been a flood of AI-generated visual content since the Middle East conflict began. This content has spread on social media in much larger volumes than what was seen during previous conflicts, making it difficult for users to distinguish between real and fabricated material.

X announced a new policy last week in an effort to preserve 'authentic information' during the war. According to this policy, creators who post AI-generated war-related videos or images without clearly labeling them will have their platform revenue sharing suspended for 90 days. A warning was also issued that repeated violations could lead to a permanent ban.

Researchers have expressed concern that even with this policy in place, it is not sufficient to stop misinformation. According to Joe Bodnar, a researcher at a strategic dialogue research institution, the social media content he monitors is still filled with AI-generated images and videos related to the war.

He noted that deceptive content has been spread even from some premium accounts. One such account shared an artificial video allegedly showing Iran launching a 'nuclear-capable' attack on Israel. That material was viewed many more times than the platform's policy announcement message.

The AFP's global fact-checking network has identified artificial fake content related to the war spreading across various countries, from Brazil to India. Some of these videos show US soldiers crying inside an embassy after a bombing, US troops kneeling next to an Iranian flag, or US naval ships being destroyed. However, all these scenes were artificially created.

When such content mixes with real images and videos, it becomes difficult for fact-checkers to identify quickly. Furthermore, the social media platform's own AI dialogue system has occasionally been seen providing users with misleading information.

According to researchers, the social media platform's revenue system also encourages the spread of such content. Since premium accounts earn revenue based on views and engagement, the likelihood of financial gain increases when sensational or false content is posted.

For example, one premium account posted an artificial video related to the Burj Khalifa skyscraper in Dubai. Although it was posted without being labeled as artificially created, it was not removed and was viewed nearly two million times.

A previously published report claimed that the social media company had profited financially from the spread of propaganda material from some accounts linked to the Iranian government. Following that, the company reportedly removed verification badges from some of those accounts.

According to experts, while the policy announced by the company is theoretically an attempt to stop misinformation, its effectiveness will depend on implementation. Researchers warned that digital details embedded in artificial content can be easily removed, making identification difficult. They also stated that the community fact-checking system is only effective to a limited extent.

A study conducted last year showed that more than 90 percent of community comments written on social media are never published. Therefore, experts say that more effective measures are needed to stop the spread of artificial fake content in sensitive situations like war.