WIRED AI Tests AI Agents On Fantasy Soccer Team Management
WIRED AI deployed AI agents to manage a fantasy soccer team, with mixed results. The experiment shows AI's potential in making predictions, but also its limitations.
Original reporting · wired.com
Original: “I Put a Bunch of AI Agents in Charge of a Fantasy Soccer Team. They’re Not Better Than Humans—Yet”
Read “I Put a Bunch of AI Agents in Charge of a Fantasy Soccer Team. They’re Not Better Than…” at WIRED AI →
How this was written. Diagnosis Forward wrote this article with a language model from WIRED AI’s reporting (model: @cf/meta/llama-3.3-70b-instruct-fp8-fast); it describes that reporting rather than adding to it. The original, linked above, is the full story.
Key points from WIRED AI’s report
- AI agents were deployed to manage a fantasy soccer team, with mixed results.
- Newer models may be better at making predictions due to their ability to analyze huge amounts of real-world information.
- AI still lags behind very skilled experts, but there is growing evidence that it can sometimes beat even the best human predictors.
- Custom harnesses and agents with unique tasks seem to be the most effective approach for AI predictions.
- AI may soon be able to dominate fantasy leagues, but for now, its suggestions are not impressive.
A few weeks ago, WIRED AI deployed a small horde of AI agents to craft and manage an experimental fantasy soccer team roster. The goal was to determine if a swarm of bots could put together a winning team, according to WIRED AI.
The experiment involved using Codex, a model that can analyze large amounts of data, to build a harness that would guide different agents through various steps to make predictions. The initial team suggestions were surprisingly poor, with the model leaving top players like Erling Haaland on the bench.
The model was then asked to create a team of agents that did different things, such as reading news and blog posts, analyzing recent results, and running models. This approach significantly improved the suggestions, but increasing the number of agents to over 100 did not lead to further improvements.
According to Haifeng Xu, a computer scientist at the University of Chicago, newer models may be better at making predictions because they can consume and analyze huge amounts of real-world information. Xu notes that AI can most effectively make predictions using a custom harness and having agents with unique tasks.
Xu and his colleagues have been testing the forecasting skills of commercial models, with promising results. They developed Prophet Arena, a benchmark that measures a model's ability to make predictions, and found that models like GPT-5 and Grok 4 performed well in prediction markets.
However, AI still often lags behind very skilled experts known as superforecasters, who have a deeper understanding of the real world. Nevertheless, there is growing evidence that AI can sometimes beat even the best human predictors, and benchmarks like ForecastBench may provide further evidence of AI's capabilities in the near future.
The experiment suggests that AI may soon be able to dominate fantasy leagues, but for now, the suggestions are not impressive, and the team has not outscored the average over the past weeks. WIRED AI reported that the big labs may soon turn their attention to future-gazing as a powerful new way to measure progress in machine intelligence.
Questions and answers
What was the goal of the experiment?
The goal was to determine if a swarm of bots could put together a winning fantasy soccer team.
What model was used in the experiment?
Codex, a model that can analyze large amounts of data, was used to build a harness that would guide different agents through various steps to make predictions.
What was the outcome of the experiment?
The experiment showed that AI agents can make predictions, but their suggestions are not yet impressive, and they still lag behind very skilled experts known as superforecasters.
