“We need to slow down”.
This was stated by Anthropic CEO Dario Amodei in an essay published on Saturday evening. He quickly received support from his rivals Sam Altman (OpenAI) and Elon Musk (SpaceXAI, Tesla, X).
The post came in the wake of Jacob Coxon’s resignation. The former Anthropic and OpenAI researcher claimed he resigned in protest because the companies are acting irresponsibly.
The headlines became truly dramatic when Anthropic researcher Evan Hubinger, in a message of support for Coxon, said he believes there is more than a ten per cent chance that AI will wipe out humanity within a decade.
How serious is all this – and what is really behind these statements?
We need to make sense of it all. We sought answers from some of the researchers at the University of Agder who are closest to AI developments:
Elisabeth Austad Asser
"We must listen to what the tech giants have to say, because the technology being developed is powerful. At the same time, we must remember that AI does not act of its own free will. There is no superintelligence here. The algorithm does what it has been programmed to do by human developers. When Amodei comes out with his statements, he almost seems a bit helpless, but it is, after all, his technology. He may feel caught up in the race between the tech companies, but if he really thought it was that dangerous and took moral responsibility, he could simply have quit."
"I think one of the most dangerous things we can do right now is to think that this technology is too big and powerful, and that here in little Europe we cannot create anything of our own. Because that is precisely what we must do, if we believe that the society we have here in Norway and Europe is built on values that we wish to uphold and need to preserve."
Morten Goodwin
"As for the likelihood of AI leading to doomsday, this strikes me as a misrepresentation of the data. What we have empirical evidence to suggest is that the technology could develop in such a way that it runs amok and we lose control. That might be bad enough in itself, but it is quite different from saying there is more than a ten per cent chance that humanity will be wiped out."
"By stepping forward and portraying themselves as responsible players with their foot on the brake, top executives can more easily steer the way towards stricter regulation that they know is coming."
"It is important to remember that it is in AI companies’ own interests to present the technology as so powerful and fantastic that they have to hold back its development. If they manage to sell the idea that the technology will become even better than it is today, they can attract investors."
Rebekka Olsson Omslandseter
"The news coverage can seem overwhelming when this is being reported on everywhere. Over time, the tech giants have built up an image that their models have improved significantly and could spiral out of control, without there really being any way to verify which parts of the information are actually accurate. The companies are both the cause of the problem and now the ones raising the alarm about it."
"At present, it is the technology companies themselves that develop, test and assess the risks posed by their own systems. As individuals, we have no real influence. Nor are there any global forums where we can discuss and agree on limits for AI – what the technology should and should not be allowed to do, as well as rules for the development of AI. With the EU’s AI Act, Europe is somewhat on the ball, but globally we lack forums for cooperation. The development of AI is progressing rapidly, whilst regulations are lagging behind."
Ishita Barua
"The fact that the leading US AI companies are now coming out publicly to call for more regulation from the authorities strikes me as odd. Several of them have actively opposed and obstructed precisely this type of regulation in Europe through lobbying aimed at watering down the EU’s AI regulation, the AI Act."
"Similar concerns about a lack of control and safety in the pharmaceutical industry would immediately trigger scrutiny from the authorities. In most industries, any suggestion of a lack of control and potential harm would likely cause share prices to collapse. But in the AI sector, this is clearly not the case. Here, worst-case scenarios underpin sky-high valuations of the companies because descriptions of the technology as uncontrolled and life-threatening also signal just how potent it is. And valuation is particularly important ahead of an IPO, which, in Anthropic’s case, could be the biggest in history."
Ole-Christoffer Granmo
"We must remember that one of the main challenges with today’s generative AI is that we cannot trace the AI’s responses back to their sources. Even if the reasoning seems convincing, it may be completely flawed because it is based on statistical patterns rather than established knowledge. If we leave critical decisions in medicine or defence to AI, we risk serious errors. The greater the scope for action and autonomy we grant these systems, the greater the risk of serious incidents."
"That is why we must develop AI solutions where we have full transparency and control, in accordance with the same strict principles used in, for example, aviation. This requires more transparent technology. We will probably not achieve full control until the solutions are logically consistent throughout, with no hidden corners. Here, I believe Tsetlin machines could play an important role, especially if researchers succeed in building large-scale logical models."