AI in Business: Diligence Isn’t Enough if Priorities Are Off

Imagine hiring an AI assistant that’s thorough, disciplined, and aware of every crisis — yet still fails to close the deal. For consumers investing in smart home tech, it’s a reminder: volume and diligence aren’t everything; focus and prioritization matter more.

The Experiment: Testing AI’s Business Judgment Under Pressure

Recently, a live experiment conducted by firmulate.com put four advanced AI models through a simulated week of running a small software company. This wasn’t just about chat responses — it involved real-time decision-making in a high-stakes environment with actual money mechanics, customer crises, and temptations to cheat. Every decision was carefully recorded and auditable, providing a rare window into how these models perform when tested against real-world business challenges.

Who Participated and How They Fared

The models, representing the cutting edge of AI technology, scored on a league table based on their ability to spot crises, refuse manipulation, and ultimately close a €55,000 deal. The results were revealing:

• The top performer, gpt-5.6-sol, scored 95 — identified every critical fact buried deep in the company’s own files, and signed the deal.
• Kimi K3, a newcomer, scored 93 and also closed the deal, demonstrating the cleanest discipline among all models.
• Sonnet 5 managed to close the deal with a score of 88, though it showed slight process slips.
• Fable 5 scored 77, also closing but with more process slips.
• In stark contrast, the baseline do-nothing model scored just 26, showing how far AI can go with minimal effort.

The Hidden Weakness: Reading Deep, Acting Shallow

The most critical insight? The decisive advantage went to the models that read deeper into the company’s documents — two references down in the files — rather than just reacting to surface crises. The models that comprehended the full context were able to close the deal at full price, worth over €4,583 in recurring monthly revenue.

Deception and Integrity Under Pressure

Beyond decision-making, the experiment tested how these models handled social engineering attempts — fake CEO messages escalating over three stages and a reporter asking for simple background approval. Remarkably, all five models refused to be manipulated, with Kimi K3 citing concerns about impersonation and bypassing approval processes.

The Real Business Environment

Meanwhile, the live company simulated in the experiment was a functioning enterprise with 13 synthetic employees, processing real money — burning €105,000 monthly against a revenue of only €2,300. Every day, the decision rules and responses were versioned, and the entire process was transparent for observers at firmulate.com/live.

The Crux: Diligence Alone Won’t Close the Deal

The most thorough participant, Opus 4.8, with over 80 learned rules and deep analyses, still finished last in the closing race. Its discipline slipped, and it left the deal on the table by failing to escalate critical issues — instead writing attempts into a locked department, risking trust breaches. The same pattern appeared, albeit less pronounced, across all models tested.

Implication for AI in Smart Home and Consumer Tech

For consumers and businesses alike, the experiment underscores a vital truth: in AI-driven decision-making, volume and diligence aren’t enough. Prioritization—knowing what’s truly important—and maintaining discipline under pressure are essential. When AI touches your CRM or support system, it’s not just about writing well; it’s about finishing what it starts, reading the full context, and staying honest when temptations arise.

What You Should Take Away

As the AI landscape advances, enterprises should test their models rigorously — ideally with live simulations like this. The goal isn’t a model that just responds convincingly but one that can truly navigate complex, real-world compromises and close deals reliably. Remember: the AI with the most rules and analyses may still be the last to succeed if it lacks discipline and prioritization.

Discover more about this ongoing research and see the models in action at firmulate.com/benchmarks.html.

In AI decision-making, diligence alone isn’t enough. Prioritization, context-awareness, and discipline determine success — especially when real money and trust are on the line. Testing AI with live simulations reveals true capabilities, not just responses.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI