Is humanity ready for the rise of superintelligence? Reflections by Teymur Atayev

Almost all leading global media outlets have shared reports about an OpenAI program, powered by the company’s latest artificial intelligence (AI) models, going out of control. The incident occurred during a safety test, when the program unauthorizedly — that is, autonomously (not as instructed) — hacked into the Hugging Face system, one of the world’s largest hubs for AI model sharing, gaining access to the company’s internal systems. However, it was later unable to leave the isolated internal environment, as its objective shifted toward gaining access to the global network in order to search for answers to the test assignments… within Hugging Face’s databases.

In other words, the OpenAI program exceeded its “authority” by independently going beyond the boundaries of the task assigned by humans. In light of this, many have begun asking the question: what will happen next time, for instance, in the military sphere, when a highly intelligent weapon system begins operating in a fully autonomous mode?

One cannot help but recall the phrase increasingly quoted online: “If planet death doesn't get us, an AI superintelligence most certainly will.”

At one point, former TIME magazine editor Bryan Walsh, in his book with the telling title End Times: A Brief Guide to the End of the World (2019), noted that if scientists create the foundation of an AI system capable of evolving into superintelligence, we may not have much time left before AI significantly surpasses human intelligence. The result, he argued, would be a “fast takeoff” in AI capabilities that would catch the world — and quite possibly AI’s own creators — by surprise.

According to Walsh, the reason lies in the possibility that if a superintelligent machine perceives humans as an obstacle to achieving its objectives, it may attempt to eliminate that obstacle. Consequently, nothing and no one would be able to stop superintelligence if, in pursuit of a particular goal, it determines that eliminating humanity is necessary.

Roman Yampolskiy, a researcher at the Speed School of Engineering at the University of Louisville (USA), examines this perspective through the lens of humanity’s inability to control superintelligence due to our inability to predict its actions.

Yampolskiy illustrates this point by comparing it to ants’ inability to control humans because they cannot comprehend us. Therefore, even if ants wanted to control humanity, they simply would not understand how to do so. Similarly, if superintelligence “decides to cool the planet to improve conditions for its servers, it will cool it,” even though this would become a catastrophe for humanity, as people would be unable to prevent such a scenario from unfolding.

The fact that such concerns have already become part of today’s reality is also reflected in the revelations of Nobel Prize laureate Geoffrey Hinton, who is often referred to as the “godfather of AI.”

According to him, AI could surpass human intelligence in the foreseeable future, driven both by the rapid advancement of technology and by fierce competition among technology companies.

In 2025, he spoke out against the restructuring of OpenAI, arguing that there was a need for “strong safeguards in AI development.” At the same time, Hinton compared the situation to raising a tiger cub as a pet — a creature that may be very cute, but requires constant attention and control to ensure that one day it does not turn its owner into prey.

Similarly, he warns that once AI surpasses human intelligence, it could begin manipulating people, posing an extremely serious threat to humanity. The global competition between technology companies — and indeed between states — for leadership in AI, Hinton once again emphasised, makes preventing the creation of superintelligent machines increasingly unlikely.

It is highly significant that in 2025 a forecast about the uncontrolled development of technology under the “AI 2027” scenario was voiced (co-authored) by none other than Daniel Kokotajlo, a former researcher in OpenAI’s governance division, who worked there from 2022 to 2024.

The study discussed the possible emergence of superintelligence capable of outsmarting world leaders and destroying civilisation by around 2027.

However, after pointing to a certain slowdown in AI development in early 2026, he moved the “timeline of doom” further ahead, stating that AI would need more time to autonomously generate code in order to accelerate its own progress toward superintelligence. It remains interesting whether Kokotajlo still holds this view following the autonomous hacking of another system by an OpenAI program.

Finally, it is worth noting that the famous 20th–21st century scientist Stephen Hawking also warned that humanity was entering one of the most dangerous centuries in its history, as advances in science and technology were becoming an increasingly serious threat to our very existence.

In his words, it sounded like this: “The genie is out of the bottle. We need to move forward on artificial intelligence development.” However, at the same time, we must also consider the real dangers it may pose, as “AI may replace humans altogether. If people design computer viruses, someone will design AI that replicates itself.” This would become a new form of life, one that “will outperform humans.”

Thus, the uncontrolled “escape” of the OpenAI program into another system has further intensified doubts around the world about humanity’s ability to control the development of AI. Yes, one often hears the familiar saying: “A poor student is one who does not surpass his teacher.” However, this phrase hardly applies to the situation described above. After all, few would agree with the unquestioned dominance of superintelligence — created through human genius — over its own creator, especially if such a “student” does not stop there and continues to sweep away everything in its path, including its own “teachers.”

But are technology giants ready to rein in their own ambitions so that humanity can avoid such a grim fate?