On July 18, 2024, ChatGPT creator OpenAI released an AI model called GPT-4o mini, a smaller and cheaper version of GPT-4o.
Despite its size, ChatGPT’s smaller LLM stands taller than similar models. As a result, the company fulfills its commitment to “making intelligence as broadly accessible as possible.”
At the time of writing, ChatGPT Free, Plus, and Team users can access the new model instead of GPT-3.5. Later, Enterprise users will gain access, letting everyone benefit from AI.
What are GPT-4o mini’s features?
OpenAI said the new AI model surpasses GPT-3.5 Turbo and smaller models in both textual intelligence and multimodal reasoning.
READ: What can you do with GPT-4o?
According to the official website, it supports the same languages in OpenAI’s larger flagship model. Also, the company evaluated GPT-4o mini across three major benchmarks: reasoning, math and coding, and multimodal reasoning.
OpenAI contrasted the smaller model to Gemini Flash and Claude Haiku, which are Google and Anthropic’s mini AI models respectively. Here are the results:
- Reasoning tasks: GPT-4o mini scored 82.0% on the Massive Multitask Language Understanding (MMLU) benchmark. In contrast, Gemini Flash got 77.9% and Claude Haiku scored 73.8%.
- Multimodal reasoning: GPT-4o mini scored 59.4% on the Multi-discipline Multimodal Understanding (MMMU) benchmark compared to Gemini Flash (56.1%) and Claude Haiku (50.2%).
- Math and coding proficiency: OpenAI’s new mini AI earned a score of 87.0% on the
Multilingual Grade School Math Benchmark (MGSM) benchmark. Conversely, Flash reached 75.5% and Haiku hit 71.7%.
GPT-4o mini is smaller than the original flagship model, but it has the same safety measures. More than 70 external experts in social psychology, misinformation, and other fields tested the model for potential risks.
READ: ChatGPT rules: AI accidentally shows its internal instructions
The more compact AI model also echoes his statement last year. In April 2023, CEO Sam Altman spoke at MIT’s Imagination in Action event, sharing his opinion on the future of AI.
He said humanity is approaching the limits of large language models, the AI models that run ChatGPT and similar tools.
“I think we’re at the end of the era where it’s gonna be these giant models, and we’ll make them better in other ways,” Altman claimed.
GPT-4o mini is likely one way the company is following this outlook.