Sam Altman said on Thursday that "there is no wall" — an apparent reference to concerns that OpenAI and other companies are facing a slowdown in advancing their AI models.
His comment, posted on X, follows a report in The Information that OpenAI's next model had shown only a moderate improvement over ChatGPT-4 and a smaller leap over previous versions.
It sparked concerns in the tech world that the techniques that have historically made AI models smarter — more training data and greater computing power — have diminishing returns.
there is no wall
— Sam Altman (@sama) November 14, 2024
Altman, who has a history of posting short, cryptic messages on X, has suggested that these so-called scaling laws are preordained. He said on the site in February that they were "decided by god" and that the "constants are determined by members of the technical staff."
Ilya Sutskever, a cofounder of OpenAI and Safe Superintelligence, recently told Reuters that results from scaling up pretraining had plateaued. Others, such as Microsoft's chief technology officer, Kevin Scott, have dismissed these concerns. Scott told Sequoia Capital's "Training Data" podcast in July that "we're not at diminishing marginal returns on scale-up."
AI labs are also exploring ways to overcome slowdowns, such as by using synthetic data and refining models after they've been trained, using a technique called inference.
The huge sums of money invested in frontier AI companies have created pressure to produce increasingly powerful models.
Last month, OpenAI raised $6.6 billion from investors in a record funding round.
OpenAI did not immediately respond to a request for comment from Business Insider.
