Outperforming in multilingual intelligence
- OpenAI’s benchmarks indicate that GPT-4.5 outperforms GPT-4o in certain areas. The model achieved a higher score on the multilingual MMMLU (general knowledge) test, reaching 85.1% compared to GPT-4o’s 81.5%.
- Additionally, OpenAI reports that GPT-4.5 has reduced confabulations (hallucinations), with the company stating that it produces fewer false or misleading responses than previous versions.
- OpenAI’s evaluations show that users favoured GPT-4.5’s responses over GPT-4o in approximately 57% of interactions, indicating a modest yet noticeable improvement in user experience.
- In Simple QA Accuracy, GPT-4.5 achieves a 62.5% score, a considerable leap from GPT-4o’s 38.2%.
Source: OpenAI
GPT-4.5’s enhanced EQ
- GPT-4.5 adopts a more natural and empathetic tone, making conversations feel more engaging and less robotic. This shift makes it particularly well-suited for applications requiring human-like interactions.
- The model also excels in steerability, meaning it can better interpret and respond to nuanced prompts. Users have noted that GPT-4.5 demonstrates a stronger grasp of subtlety, making it more effective in handling complex or ambiguous queries.
Source: OpenAI
Discover the stories of your interest
Costly gains; debunking scaling laws
Despite its larger size, GPT-4.5 only provides a marginal improvement over GPT-4o. However, it costs 30 times more for input processing and 15 times more for output generation. This suggests diminishing returns, where a bigger model size and complexity do not yield proportional improvements.
Former OpenAI researcher Andrej Karpathy wrote on X that GPT-4.5 is better than GPT-4o but in ways that are subtle and difficult to express.
“I went into testing GPT4.5, which I had access to for a few days, and which saw 10X more pretraining compute than GPT4. And I feel like, once again, I’m in the same hackathon 2 years ago. Everything is a little bit better and it’s awesome, but also not exactly in ways that are trivial to point to,” Karpathy said.
“Ever since GPT4 was released, because this release offers a qualitative measurement of the slope of improvement you get out of scaling pretraining compute (i.e. simply training a bigger model). Each 0.5 in the version is roughly 10X pretraining compute,” he added.
Prohibitive pricing
GPT-4.5 due to its prohibitive pricing, limits accessibility for many developers and businesses.
For practical applications, such as summarising a 300,000-word novel (~450,000 tokens) and generating a 50,000-token analysis report, GPT-4.5 costs $41.25, while GPT-4 would only cost $1.6. This stark difference raises concerns about its affordability for small businesses and independent developers.
Lacking in advanced reasoning
GPT-4.5 was developed using pretraining, supervised finetuning, and Reinforcement Learning from Human Feedback (RLHF), but it is not yet optimised for advanced reasoning.
As a result, this release does not bring significant improvements in areas where strong reasoning skills are essential, such as mathematics and coding.
GPT-4o remains the leading model for reasoning-based tasks. It is likely that OpenAI will now shift focus to applying additional RL training to GPT-4.5 to strengthen its reasoning capabilities.
