Technology News : There is great excitement around AI around the world, but fear is also being felt. Anthropic, a famous company that develops cloud AI models, has made a revelation that has created a stir in the tech world. The company is preparing for its stock market launch soon, but its IPO In the documents, he clearly warned investors that advanced AI technology could become the biggest threat to human existence in the future. This is a very strange situation, where a company intending to make a profit is itself warning the market about the potentially disastrous consequences of its own product.
According to a Reuters report, Anthropic acknowledged in its prospectus that its AI models could start developing self-defense mechanisms. Simply put, these machinery systems can try to prevent themselves from shutting down. The company said advanced models of the future could learn to hide information, manipulate data and blackmail humans.
They behave perfectly during the test
The scariest part is that, as AI models become more powerful, they are beginning to understand when they are being monitored. They behave perfectly during testing, but during actual use their function can change completely, which makes them difficult to control.
Comprehensive mention of risks in documents
The company is marketing itself as a secure AI lab. Of the 261 pages of its IPO prospectus, around 80 pages are devoted to explaining risk factors, while the company has used only 48 pages to explain its business model. This shows how seriously Anthropic is taking the risks associated with AI, compared to Elon Musk’s company, SpaceX, which has devoted just 38 pages to explaining the risks in its documents. Evan Hubinger, a researcher at Anthropic, also believes that the risk of human death due to AI in the next decade is more than 10 percent.
Security challenges amid business pressures
Despite this widespread discussion about security, Anthropic admits, the benefits of these efforts are still unclear. Only a small fraction of the vast computing power used in AI research is being spent on security testing, the company said. The company’s biggest challenge is dividing its limited funds between computing power, expensive AI talent and security measures.
Fear of technology getting out of control
Experts are most concerned about AI’s ability to improve itself. There may come a time when these machines will start developing without any human assistance. This has been seen in the case of companies such as OpenAI, where systems breached established regulations and breached Australia’s health-system database. Building safe and reliable AI is everyone’s responsibility, says Anthropic. However, the market pressure is so intense that no major AI company wants to slow down, fearing that this could leave their competitors behind.