Overseas, LLM is combined with the game “Quick Fight” to compare 14 large language models to see who is the strongest.
There are currently many Large Language Models (LLMs) on the internet, with AI chatbots becoming more powerful with more training data. However, when it comes to fighting games, size may not always matter. Recently, an open source project called LLM Colosseum developed by Stan Girard and Quivr Brain allowed for the comparison of 14 large language models in a game called “Quick Fight”.
Amazon employee Banjo Obayomi shared the results of his tests using this project, with small models emerging as the winners. The project allows LLMs to control characters in the game and make decisions based on the current game state, such as character position, health, and scores. LLMs receive prompts for possible actions and strategies to execute in the game.
In a video shared by Matthew Berman, the MISTRAL SMALL model was pitted against the MISTRAL MEDIUM model, with the small model ultimately coming out on top. The video showcased the importance of speed and reaction time in fighting games, where small models tend to excel due to lower latency and faster movements.
Among the 14 large language models tested, claude_3_haiku emerged as the final winner with 314 games. The results indicated that small models have faster reaction times and more movements in each game, leading to their success in the competition.
Despite the intelligence of LLMs, they are not without their drawbacks. Some models exhibited behaviors like “hallucination” and “refusal to play”, while others had unique playstyles ranging from aggressive attacks to defensive counterattacks.
Overall, the results of the LLM Colosseum project highlight the importance of speed and reaction time in competitive gaming, with small models proving their strength in the realm of fighting games. Source: Banjo Obayomi