AI Companies Accused of Profiting from Fear of Technology

They made it. They are afraid of it. Yet they are still selling it.

If you have heard this before, stop me: A tech company claims it has created an AI so powerful it is terrifying.

According to them, it is too dangerous to release into the world and could have catastrophic consequences. Luckily for us, they have kept it locked away for now. They just wanted you to know.

This is exactly what AI company Anthropic is telling us about its latest model, 'Claude 3'. The company claims that its ability to detect cybersecurity vulnerabilities far surpasses that of human experts. They say that if such technology falls into the wrong hands, it could lead to consequences that reshape the world.

Anthropic and OpenAI are selling psychological fear by calling their AI products too dangerous.

Anthropic wrote in a blog post in early April, 'Its impact on the economy, public safety, and national security could be severe.' Some hasty observers are even warning that every technological device in life, from Wi-Fi-connected microwaves, should be replaced to avoid digital madness.

Some security experts doubt these claims. But let's leave that for now. This is not new. Officials from major AI providers regularly warn how the products of their industry could destroy humanity.

But why do AI companies want us to be afraid of them?

It is a strange way for any company to talk about its work. You would never hear McDonald's announce, 'We have created such a terrifyingly delicious burger that it would be unethical to serve it to the public.'

One theory is this. According to critics, AI companies benefit when you are preoccupied with the fear of doomsday. It helps distract from the real damage they are doing in the world. Tech leaders claim they are only warning about an inevitable future and that safety is always a top priority.

But other experts see it as just a trade in fear. It exaggerates the capabilities of the technology and helps boost share prices. It also helps create a narrative that regulators need to step back. Because the message is spread that only these AI companies are capable of preventing bad actors and building this technology responsibly.

Shanon Vallor, a professor of data and artificial intelligence ethics at the University of Edinburgh, says, 'If you portray these technologies as almost supernatural in their danger, it makes us feel powerless and helpless. It feels like the only option we can rely on is those companies.'

• Stop me if you've heard this one before

A spokesperson for Anthropic told me the company is clear on these issues. They shared blog posts from other organizations supporting the cybersecurity capabilities of Claude 3. But they offered nothing else beyond a comment included below that addresses the points raised in this article.

By showing the imaginary fear of world destruction, the real damage that AI is causing to society, mental health, and the environment is being ignored.

This is not the first time that a company where Anthropic's CEO Dario Amodei worked has declared a tool 'too dangerous' for the public. In 2019, when Amodei was an executive at OpenAI, the company announced 'GPT-2'. He and other company leaders said they could not release GPT-2 due to 'concerns about potential misuse of the technology.'

This was a much less capable tool than ChatGPT. But a few months later, they released it. Later, OpenAI CEO Sam Altman wrote a blog post admitting that the company accepts uncertainty and that the fear surrounding GPT-2 was inappropriate.

Altman recently criticized Anthropic's 'fear-based marketing' in a podcast interview. But his own 'I created a monster' narrative is years old.

In 2015, Altman said, 'AI will probably end the world, but by then there will be great companies.' Years later, Altman claimed his sleep was disturbed thinking, 'I didn't really do anything wrong by launching ChatGPT.' If only someone had warned him earlier.

Hundreds of tech leaders, including Altman, Amodei, Bill Gates, and Google DeepMind CEO Demis Hassabis, endorsed a short statement in 2023. It stated, 'Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.'

That same year, tycoons including Elon Musk signed a letter calling for a six-month pause on the development of advanced AI. But Musk announced his new artificial intelligence company 'xAI' less than six months later.

Emily M. Bender, a professor of computational linguistics and natural language processing at the University of Washington and co-author of 'The AI Does Not Hate You,' says, 'This is just part of a tendency to make unsubstantiated claims of power.'

AI companies have adopted a strategy of making regulators feel powerless and establishing themselves as saviors.

This is a common characteristic not only of OpenAI and Anthropic but of the entire AI industry, she says. She adds, 'They are saying: Look over here, don't worry about environmental destruction and labor exploitation, forget that the mechanisms of society are being destroyed. We just need to ensure that this technology does not turn into a bad element that destroys humanity.'

I asked OpenAI about this. A spokesperson shared Altman's recent blog post. In it, Altman mentioned that 'OpenAI will prevent the potential for this technology to concentrate power in the hands of a few individuals.' He argued that key AI decisions should be made through democratic processes and egalitarian principles, not just by AI labs.

• Is Claude 3 really that bad?

According to Anthropic, its new model has already identified thousands of 'extremely serious' vulnerabilities across the tech sector, surpassing human experts. The company has also announced new partnerships with more than 40 companies and groups as an 'immediate effort' to fix these vulnerabilities before hackers can exploit them. A spokesperson for the company told me that Anthropic is intentionally partnering with organizations to fix the underlying computing systems that make up a 'large part' of the problem.

But there are serious doubts about these claims. Heidi Claff, chief AI scientist at the AI Now Institute, was not impressed. She has spent her entire career building and testing code analysis tools similar to those Anthropic claims to surpass. She has also worked on the digital security of nuclear facilities.

According to Claff, the most suspicious thing is the lack of 'false positive rate' data. This is an established industry standard that measures how often a security tool flags a problem where there is none. 'This is not an unknown metric,' Claff says, 'It is the biggest indicator of how useful your tool is.'

The narrative of 'saving humanity' is being used to ignore the real damage AI is causing to society, mental health, and the environment.

Anthropic did not mention this and dodged the question when I asked for a response. Anthropic has also not compared Claude 3 to existing tools that security engineers have relied on for decades.

AI companies have adopted a strategy of making regulators feel powerless and establishing themselves as saviors.

There are also claims that Anthropic may not have widely released Claude 3 due to a lack of necessary computing power. Anthropic did not respond when I inquired about this.

This does not mean the threat is entirely imaginary. 'Claude 3 could be capable,' says Claff. AI tools are indeed useful for scanning large codebases, and automatically detecting security vulnerabilities is a real and serious threat. But Claff is skeptical of Anthropic's claims without supporting evidence. She says, 'I think there are many weaknesses in this narrative that Claude 3 is omnipotent and we cannot release it.'

• Why so much seriousness?

OpenAI and Anthropic say the main reason for their existence is to prevent the end of the world. OpenAI was founded as a non-profit organization, promising to build AI safely before more irresponsible tech giants like Google and Meta. Later, a group broke away from OpenAI to form Anthropic, accusing their former employer of not being sufficiently committed to safety. Now, both organizations are becoming publicly traded companies and are in a race to sell shares on the stock market.

Vallor says, 'If you want to understand how an organization, especially a corporation, behaves, look at its interests.'

Google broke its own red line against building AI weapons. OpenAI fought a legal battle to remove its non-profit status. Anthropic abandoned its core policy of not training AI models until sufficient safety measures could be guaranteed.

Vallor says, 'I don't believe any of these companies would pass up an opportunity to dominate the market to remain good people.'

On the other hand, AI use in healthcare is being promoted despite serious concerns about misdiagnosis. Gas-powered data centers can emit more greenhouse gases than entire countries. There are reports of AI driving people with fragile mental states to madness and suicide.

Increasing research suggests a potential link between AI and the decline of human intellectual capacity. Deepfakes have crossed a line; I couldn't even convince my own aunt that 'I am not a robot.'

AI companies say they take these issues seriously. OpenAI sent me links to its views on mental health, AI accuracy, fraud, and scams. Altman also says the company is committed to solving the problems that arise at every stage of AI development.

But there is a specific reason why these companies only warn of doomsday, says Vallor. If AI can destroy society, then other minor problems seem much less significant. 'This strategy has been successful,' she says. 'These companies have not lost anything by talking about how their products can end the world. It has not limited their power. Instead, it makes people feel that they have no choice but to rely on those companies for safety.'

• Monster or Messiah

Those who warn of destruction on one hand also promise salvation on the other. In a 2024 essay, Altman predicted, 'Amazing achievements like solving climate change, establishing space colonies, and uncovering all the mysteries of physical science will eventually become commonplace.' Amodei promised a country of talented people within a data center.

According to Vallor, utopia and apocalypse are two sides of the same coin. She says, 'In both cases, the scale is so vast and mythical that it doesn't feel like regulation, governance, or law can control it. It makes people believe that all they can do is sit back and wait to see if these technologies become monsters that destroy civilization or messiahs that provide an ideal world.'

Even the name 'Claude 3' seems intended to evoke religious awe.

We have regulated things far more dangerous than chatbots. Vallor says, 'We have not allowed such narratives to dominate us with any other technology, such as nuclear or biological weapons, that makes us think these powers are beyond human control. There is nothing about these technologies that cannot be regulated. But it is a different matter if we ourselves decide not to regulate.'

Let's be clear about one thing: it is theoretically possible for AI to take over the world. I am no prophet. But ask yourself, doesn't this idea sound similar to other stories heard from Silicon Valley in the past?

Shouldn't we all be living inside Mark Zuckerberg's 'Metaverse' by now? Wasn't Bitcoin supposed to displace all the world's currencies? Remember the 2010s, how much we heard about how social media would save democracy? All of this may still happen. Or perhaps it may not.