Elon Musk announced his xAI company will upgrade its AI chatbot Grok next week. He said on X, “Grok 2 should exceed current AI on all metrics.”
Despite his statement, the AI bot’s upcoming version is 1.5. The official page claims “Grok-1.5 comes with improved reasoning capabilities and a context length of 128,000 tokens.”
According to VentureBeat, the upcoming artificial intelligence (AI) will be near GPT-4’s performance, the large language model powering the popular ChatGPT.
READ: Elon Musk’s xAI to launch improved version of chatbot
Musk’s company said Grok-1.5 achieved high scores on popular AI benchmarks, namely:
- MMLU (Massive Multitask Language Understanding): AI builder platform Klu AI defines it as “a diverse set of tests designed to evaluate the understanding and problem-solving abilities of language models across multiple domains.”
- MATH Benchmark (Math Word Problem Solving): AI testing site Metaculus explains it is “a benchmark to measure the mathematical problem-solving ability of [AI] models.”
- GSM8K (Grade School Math 8K): Klu AI says it is “a dataset of 8,500 high-quality, linguistically diverse grade school math word problems.”
- HumanEval: Klu says it “evaluates the code generation capabilities of large language models (LLMs).”
Specifically, Grok’s upcoming version scored 81.3 percent on the MMLU, 50.6 percent on MATH, 90 percent on the GSM8K, and 74.1 percent on HumanEval.
On the other hand, OpenAI’s GPT-4 earned 86.4 percent, 52.9 percent, 92 percent, and 67 percent respectively. That means Grok-1.5 is likely to perform better than ChatGPT in solving mathematical problems.
Grok is also improving to the point that it’s almost at the level of GPT-4 as its version 1.5 is forecast to have the capability of processing 128,200 tokens, which refer to bits of raw text.
READ: Elon Musk might be forming an AI chatbot rival to ChatGPT
TechCrunch said AI chatbots tend to divide words into multiple tokens. For example, it may split the word “fantastic” into “fan,” “tas,” and “tic.”
The official xAI webpage said the chatbot upgrade “allows Grok to have an increased memory capacity of up to 16 times the previous context length.” As a result, Grok-1.5 could handle “substantially longer documents.”
However, TechCrunch reminded the public that AI benchmarks do not reflect how average people will interact with these AI programs.
The X Help Center website states Grok is available to X Premium+ subscribers, which costs $16 monthly or $168 annually.