Where AI Is Now: Smaller, Better, Cheaper Models<\/p>\n
A state of the AI industry report shows that 2024 was a breakthrough year for small, sleek models to rival the behemoths<\/p>\n<\/div>\n
By Nicola Jones<\/a> & Nature magazine<\/a> <\/p>\n Top AI models\u2019 performance is improving quickly, and the competition between them is growing ever fiercer.<\/p>\n<\/div>\n<\/figcaption><\/figure>\n<\/div>\n The artificial intelligence (AI) race is heating up: the number and quality of high-performing Chinese AI models<\/a> is rising to challenge the US lead, and the performance edge between top models is shrinking, according to an annual state of the industry report<\/a>.<\/p>\n The report highlights that as AI continues to improve quickly, no one firm is pulling ahead. On the Chatbot Arena Leaderboard, which asks users to vote on the performance of various bots, the top-ranked model scored about 12% higher than the tenth-ranked model in early 2024, but only 5% higher in early 2025 (see \u2018All together now\u2019). \u201cThe frontier is increasingly competitive \u2014 and increasingly crowded,\u201d the report says.<\/p>\n The Artificial Intelligence Index Report 2025 was released today by the Institute for Human Centered AI at Stanford University in California.<\/p>\n If you’re enjoying this article, consider supporting our award-winning journalism by subscribing<\/a>. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.<\/p>\n Nature; Source: AI Index Report 2025<\/p>\n<\/div>\n<\/figcaption><\/figure>\n The index shows that notable generative AI models are, on average, still getting bigger<\/a>, by using more decision-making variables, more computing power and bigger training data sets. But developers are also proving that smaller, sleeker models are capable of great things. Thanks to better algorithms, a modern model can now match the performance that could be achieved by a model 100 times larger two years ago. \u201c2024 was a breakthrough year for smaller AI models,\u201d the index says.<\/p>\n Bart Selman, a computer scientist at Cornell University in Ithaca, New York, who was not involved in writing the Index report, says it\u2019s good to see relatively small, cheap efforts such as China\u2019s<\/a> DeepSeek<\/a> proving they can be competitive. \u201cI\u2019m predicting we\u2019ll see some individual teams with five people, two people, that come up with some new algorithmic ideas that will shake things up,\u201d he says. \u201cWhich is all good. We don\u2019t want the world just to be run by some big companies.\u201d<\/p>\n The report shows that the vast majority of notable AI models are now developed by industry rather than academia: a reversal of the situation in the early 2000s, when neural nets<\/a> and generative AI<\/a> had not yet taken off. Industry produced fewer than 20% of notable AI models before 2006, but 60% of them in 2023 and nearly 90% in 2024, the report says.<\/p>\n The United States continues to be the top producer of notable models, releasing 40 in 2024, compared with China\u2019s 15 and Europe\u2019s 3. But plenty of other regions are joining the race, including the Middle East, Latin America and southeast Asia.<\/p>\n And the previous US lead in terms of model quality has disappeared, the report adds. China, which produces the most AI publications and patents<\/a>, is now developing models that match their US competition in performance. In 2023, the leading Chinese models lagged behind the top US model by nearly 20 percentage points on the Massive Multitask Language Understanding test (MMLU), a common benchmark for large language models. However, as of the end of 2024, the US lead had shrunk to 0.3 percentage points.<\/p>\n \u201cAround 2015, China put itself on the path to be a top player in AI, and they did it through investments in education,\u201d says Selman. \u201cWe\u2019re seeing that\u2019s starting to pay off.\u201d<\/p>\n The field has also seen a surprising surge in the number and performance of \u2018open weight\u2019 models such as DeepSeek and Facebook\u2019s LLaMa<\/a>. Users can freely view the parameters that these models learn during training and use to make predictions, although other details, such as the training code, might remain secret. Originally, closed systems, in which none of these factors are disclosed, were markedly superior, but the performance gap between top contenders in these categories narrowed to 8% in early 2024, and to just 1.7% in early 2025.<\/p>\n \u201cIt\u2019s certainly good for anyone who can\u2019t afford to build a model from scratch, which is a lot of little companies and academics,\u201d says Ray Perrault, a computer scientist at SRI, a non-profit research institute in Menlo Park, California, and co-director of the report. OpenAI in San Francisco, California, which developed the chatbot ChatGPT, plans to release an open-weight model in the next few months.<\/p>\n After the public launch of ChatGPT in 2022, developers put most of their energy into making systems better by making them bigger. That trend continues, the index reports: the energy used to train a typical leading AI model is currently doubling annually; the amount of computing resources used per model is doubling every five months; and the training data sets are doubling in size every eight months.<\/p>\n Yet companies are also releasing very capable small models. The smallest model registering a score higher than 60% on the MMLU in 2022, for example, used 540 billion parameters; by 2024, a model achieved the same score with just 3.8 billion parameters. Smaller models train faster, give faster answers and use less energy than larger ones<\/a>. \u201cIt helps everything,\u201d says Perrault.<\/p>\n Some smaller models can emulate the behaviour of larger models, says Selman, or take advantage of better algorithms and hardware than those in older systems. The index reports that the average energy efficiency of hardware used by AI systems improves by about 40% annually. As a result of such advances, the cost of scoring just over 60% on the MMLU has plummeted, from about US$20 per million tokens (bits of words produced by language models) in November 2022 to 7 cents per million tokens in October 2024.<\/p>\n Despite striking improvements on several common benchmark tests<\/a>, the index highlights that generative AI still suffers from issues such as implicit bias and a tendency to \u2018hallucinate\u2019, or spit out false information. \u201cThey impress me in many ways, but horrify me in others,\u201d says Selman. \u201cThey surprise me in terms of making very basic errors.\u201d<\/p>\n This article is reproduced with permission and was <\/i>first published<\/i><\/a> on April 7, 2025<\/i>.<\/p>\n<\/div>\n![\"Conceptual](\"https:\/\/static.scientificamerican.com\/dam\/m\/59ddd80774f60255\/original\/AI_chat_icons.jpg?m=1744229519.147&w=600\"){kind=icon}
On supporting science journalism<\/h2>\n
![\"All](\"https:\/\/static.scientificamerican.com\/dam\/m\/407705d6c9213c76\/original\/nature_AI-race_graphic.png?m=1744229323.877&w=1000)
<\/picture>Neck and neck<\/h2>\n
Better, smaller, cheaper<\/h2>\n