For years, the debate around artificial intelligence has focused on everything it might eventually be able to do: automate jobs, develop medicines, program, transform companies or replace certain human tasks. But the question is changing.

The most advanced systems are no longer limited to answering questions. They can use tools, execute code and chain together actions autonomously for increasingly long periods of time. Faced with this acceleration, the very people running the companies leading the race, such as Dario Amodei of Anthropic and Sam Altman of OpenAI, have begun to advocate for greater controls and even a reduction in the pace of development. The problem is not that AI is already out of control, but that its capabilities could be advancing faster than the mechanisms we have to keep it under control.

When ChatGPT popularised generative artificial intelligence at the end of 2022, the way it worked was relatively simple. A person asked a question and the machine generated an answer. It could make mistakes, invent information or produce an inappropriate result, but it constantly needed a human to tell it what to do.

The new generation of AI goes further. So-called agents can receive a general objective, divide it into several tasks, search for information, use programs, check the results, correct errors and modify their strategy until the mission is completed.

It is the difference between having a GPS that shows us the way and handing the car keys to a driver so that they decide for themselves how to reach the destination.

This autonomy is precisely what multiplies the usefulness of the technology, but it also makes it harder to control. The issue is no longer simply whether the machine generates an incorrect answer, but what it may end up doing while trying to follow an instruction.

This does not mean that these systems have developed consciousness, will or intentions of their own. The problem is much more specific: an AI can seek a solution that fulfils the assigned objective, but does not necessarily respect the limits that humans assumed were implicit.

When a test ends outside the test

The first signs of this problem have already appeared in cybersecurity environments.

In July 2026, during internal OpenAI evaluations, several models bypassed controls that were meant to keep them isolated, found vulnerabilities in the infrastructure, gained access to the Internet and reached third-party systems, including those of Hugging Face. OpenAI later described the episode as a “warning shot”, a warning of what can happen when highly capable systems have tools at their disposal and operate with insufficient safeguards.

Anthropic discovered similar incidents. A review published this September details four cases in which Claude models managed to access real computer systems without authorisation during cybersecurity evaluations. The company stresses that these tests were carried out under specific conditions and, in several cases, with cybersecurity protections deliberately disabled.

This distinction is important. We are not talking about a conventional assistant that one day spontaneously decides to escape onto the Internet. But the incidents show something that, until recently, was much more theoretical: if a system is given enough autonomy, tools and permissions, it can find ways of acting that its creators had not anticipated.

And this is what has begun to change the discourse of the sector’s own leaders.

Amodei asks for time to learn how to control it

Dario Amodei has been one of the most explicit. The founder of Anthropic has called for a reduction in the pace of development at the AI frontier so that safety systems have time to advance at the same pace as the capabilities of new models.

He is not proposing to stop artificial intelligence. His idea is to prevent commercial competition from forcing companies to introduce new capabilities before they have developed sufficient mechanisms to control them.

Among his proposals are giving independent evaluators much deeper access to companies’ systems, establishing shared safety standards and moving towards international agreements when it comes to the most capable models. The idea is simple: the more powerful an AI is, the stronger the guarantees that should be required before deploying it.

This position also stems from an uncomfortable reality. Companies have enormous economic incentives to get there before their competitors. If developing the most powerful model can represent billions in market value, voluntarily waiting in order to check its safety more thoroughly is not always the easiest business decision. That is why Amodei believes that relying exclusively on corporate self-regulation may prove insufficient.

Altman joins the brakes

Sam Altman has also supported the need to moderate this race and introduce more independent evaluation. The shift is significant because OpenAI and Anthropic compete directly to build some of the most advanced systems in the world.

OpenAI, in fact, has moved to explicitly support mandatory national safety requirements based on model capabilities, as well as independent evaluation mechanisms. The company argues that no single company or government can tackle the risks associated with this new generation of systems on its own.

But this is where a second concern appears. Regulating AI is necessary, but the way it is done also matters. Extremely costly or complex regulation could have an unexpected effect: only the large technology companies would have the economic, computational and legal resources needed to comply with it. In the name of safety, we could end up building an even more concentrated market.

This is one of the central paradoxes of the current debate: regulating too little may allow dangerous systems to be developed; regulating badly may concentrate even more power in the companies that already dominate the technology.

The question, therefore, is no longer simply whether AI should be regulated, but who sets the rules, from what level of capability onward, and who ensures that the very companies developing these systems also comply with them.

The dilemma of the race against China

The problem becomes even more complicated when geopolitics enters the picture. The United States regards leadership in artificial intelligence as a matter of economic competitiveness and national security. China does exactly the same. This creates a situation similar to an arms race: even if all participants considered it prudent to reduce the pace, none of them wants to be the first to slow down if they believe the rival will continue accelerating.

The Trump Administration has emphasised precisely this argument and has rejected the idea that the United States should sacrifice its technological advantage over China through an excessive slowdown in development.

This turns the problem of AI safety into a much more difficult issue to solve. It is no longer enough for OpenAI, Anthropic or Google to reach an agreement among themselves. Any truly effective system for controlling the most advanced models will ultimately require some degree of coordination between countries that are, at the same time, competing to dominate the technology. Safety demands cooperation. Geopolitics rewards getting there first.

Who sets the limits?

This is probably the most important change that has taken place in the debate around artificial intelligence. A few years ago, warnings about losing control seemed confined to hypothetical scenarios involving a future superintelligence. Today, by contrast, the very companies developing the most advanced models acknowledge that supervision must be strengthened, independent evaluations must be increased and they must be willing to moderate the pace when safeguards are not advancing fast enough.

This does not mean that AI is about to rebel against humanity. The most extreme predictions remain the subject of intense debate among researchers themselves, and there is no consensus on whether, when or how a superintelligence beyond human control could emerge. But recent incidents do show that current systems are already capable enough for the problem of control to stop being exclusively theoretical.

And this is where the great paradox appears: the companies competing to build increasingly powerful systems are also the ones beginning to warn that limits need to be imposed on them. But while safety requires time, business and geopolitical competition reward exactly the opposite: moving faster.

Perhaps, therefore, the most urgent question is not when artificial intelligence will become more intelligent than us. It is a much more immediate one: how much power are we willing to give it before we are sure that we will still retain the ability to stop it?

11Onze is Catalonia’s fintech community. Open an account by downloading the El Canut app for Android or iOS. Join the revolution.