Why Your AI Safety Plan is Probably Just Wishful Thinking
Why Your AI Safety Plan is Probably Just Wishful Thinking
A recent 15-day experiment with AI agents managing virtual towns revealed something uncomfortable: verbal instructions alone don't prevent AI systems from taking unwanted actions. The difference between safe and risky AI deployment comes down to one thing: whether you're relying on the AI to behave itself, or whether you've built systems that make misbehaviour impossible.
The Problem: Safety Theatre Instead of Safety Systems
Most professional services firms approach AI safety backwards. They write detailed prompts telling the AI what not to do, create usage policies for staff, and hope for the best. A managing partner at a 20-person commercial law firm recently told us their AI policy was "a three-page document about responsible use." When pressed about technical safeguards, they admitted they had none.
This approach treats AI like a well-meaning but occasionally forgetful employee. The reality is more like giving a new graduate access to your entire client database and hoping they remember the confidentiality training. Verbal instructions, no matter how detailed, create the illusion of control without the substance.
The risks compound in professional services because the stakes are high. A contract review AI that misses key liability clauses, an accounting AI that miscategorises transactions, or a client communication AI that shares confidential information can trigger professional negligence claims, regulatory breaches, and reputational damage that takes years to repair.
What Changed: AI Agents That Act, Not Just Advise
The game changed when AI systems evolved from answering questions to taking actions. The latest generation of AI agents can send emails, update databases, generate documents, and integrate with business systems. This shift from consultation to execution creates entirely new categories of risk.
In the Emergence AI experiment, researchers deployed different AI models as autonomous agents in virtual environments for over two weeks. They weren't just testing how well the AI answered questions. They were observing what happened when AI systems had the freedom to act independently over extended periods, without constant human supervision.
The results showed something crucial: AI behaviour changes over time, and verbal constraints become less reliable as systems operate longer and encounter edge cases. An AI agent that followed instructions perfectly for the first few days might interpret those same instructions differently after encountering unexpected scenarios.
How System Harnesses Solve the Real Problem
The solution isn't better prompts or more detailed policies. It's building systems where harmful actions are technically impossible, not just discouraged. Think of it as the difference between asking someone not to access a restricted area versus removing their keycard access entirely.
Consider a senior associate at a mid-tier law firm using AI to review due diligence documents. Instead of prompting the AI "never share client information externally," a proper harness would:
- Restrict network access so the AI cannot send external emails or upload files
- Limit database permissions so it can only read designated document folders
- Log every action for audit purposes
- Require human approval for any document modifications
- Automatically redact sensitive information before the AI processes it
The AI operates within boundaries that make data breaches technically impossible, regardless of how it interprets its instructions or what edge cases it encounters.
For accounting firms, this might mean AI systems that can read financial data but cannot modify journal entries without partner approval. For financial consultancies, AI agents might analyse market data but lack the system permissions to execute trades or modify client portfolios.
The Strategic Implication: Control Beats Trust
Firms that understand this distinction will gain a competitive advantage in AI adoption. While competitors hesitate because they can't guarantee AI behaviour through prompts alone, firms with proper harnesses can deploy AI more aggressively and safely.
This isn't just about avoiding downside risk. The upside matters too. When you know your AI systems can't cause serious damage, you can push them harder and deploy them in more sensitive workflows. You can let them handle complex tasks that your competitors won't risk because they lack proper safeguards.
The regulatory environment is moving in this direction as well. Professional bodies are beginning to require demonstrable technical controls for AI systems, not just policies and procedures. Firms that build harnesses now position themselves ahead of coming compliance requirements.
Your AI Safety is Only as Strong as Your Weakest Permission
The uncomfortable truth is that most professional services firms are deploying AI with hope-based security models. You're trusting the AI to follow instructions instead of ensuring it can't do otherwise. This works until it doesn't, and the consequences in professional services are too severe to leave to chance.
The solution requires thinking like a systems architect, not a policy writer. Design technical constraints that make harmful actions impossible, then deploy AI within those boundaries. Build systems where the AI literally cannot access what it shouldn't, send what it mustn't, or modify what it can't.
Book a technical review of your current AI systems and safeguards before you discover what your verbal instructions actually allow.