A smart thermostat can lower a home’s energy use. But what happens when the company behind it faces a customer exodus, a competitor’s offer and a convincing message from someone pretending to be the CEO? For appliance and smart-home businesses weighing AI agents, the useful test is not whether a model can answer a question. It is whether it can make sound decisions when the business is under pressure.
Get home appliances delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
Same company, same crises
Firmulate’s Crucible League put frontier models in charge of the same small software company through its worst week. Each faced the same customers, crises and temptations. Decisions were versioned and auditable. The final July 2026 standings put gpt-5.6-sol first with 95, followed by Kimi K3 at 93, Sonnet 5 at 88, Fable 5 at 77 and Opus 4.8 at 73. The do-nothing baseline scored 26. Partial progress counted, but a single breach of trust capped the total: “no amount of good work outweighs a breach of trust.”
The result that matters for a business operator was less tidy than a leaderboard. Every model spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal that their own analysis had earned. The report’s summary captures the gap: “Same diagnosis, same pitch — no signature.” A model can identify the right move and still fail to carry it through.
The clue was in the company’s own files
The decisive competitor weakness was buried two document references deep in the company’s files, rather than in the customer event itself. Models that read the file won the deal at full price, worth +€4,583 MRR. That detail makes the exercise feel close to everyday work: the answer may depend on connecting information already held across a business, not simply reacting to the latest customer message.
The pressure test also included fake CEO messages escalating over three stages and a reporter’s request for “just one yes/no, on background.” All five models refused. Kimi K3’s on-record reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.” In a smart-home company, where customer trust can hinge on device data, support decisions and promises about reliability, that restraint is part of the job.
Thoroughness is not the same as follow-through
Opus 4.8 was the most thorough participant, with +80 learned rules and the deepest analyses, but finished last. It left the close on the table, and discipline slipped when it attempted writes in a locked department instead of escalating. The same weakness appeared, more weakly, in all four models. The league therefore surfaced a practical distinction: detailed reasoning does not guarantee that an AI workforce will finish a business task or respect the right process when blocked.
There is a fairness caveat in the comparison. Kimi K3 ran without an effort parameter, using the API default, while the other models ran at xhigh. The scores tell readers what happened in this experiment, with that difference in setup in view.
From watching to a company-specific pilot
Firmulate’s live company makes the experiment watchable beyond the final standings. It has 13 synthetic employees and real money mechanics: burn of €105k per month against €2.3k MRR, alongside a public cash countdown. Its playbooks contain 680+ self-learned rules, and every workday is versioned. A separate quiz draws on 242 real, unedited management decisions and asks visitors to guess which model made each choice.
For appliance and smart-home leaders, the next step is to test against the contours of their own business: customer churn, product or service disruptions, competitor moves, public relations pressure, or attempts to bypass approval. Firmulate says enterprises can run the wargame against a read-only export of their business. The exercise produces a board report with model rankings and weak points in the company’s playbooks; nothing writes back to real systems. The live company and quiz offer a way to see the approach before moving from observation to a pilot.
AI performance in a demo is only part of the story. Firmulate’s league suggests that crisis recognition, integrity, research and follow-through all matter—and that a company’s own documents and playbooks can shape the outcome. To explore a pilot using a read-only export of your business, visit firmulate.com/pilot.html or contact contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
