When the Cobbler's Children Have No Shoes: The McKinsey AI Hack and What Every Business Deploying AI Needs to Understand
There's an old saying about experts whose own work suffers from the very problems they solve for others. Last week, it played out very publicly in AI.
CodeWall researchers used an AI-driven autonomous security agent to identify and exploit vulnerabilities in McKinsey's internal AI platform — a tool called Lilli, reportedly used by tens of thousands of consultants — in just two hours. No credentials. No insider access. No human involvement after the initial trigger.
The reported entry point was a classic SQL injection vulnerability — one of the oldest and most well-understood security flaws in modern software, documented since the 1990s and sitting on the OWASP Top 10 for over a decade.
But here's what makes this genuinely important for any business deploying AI right now: the real danger wasn't just the data exposure. It was what the attacker could have done next.
The Prompt Layer: Your New Crown Jewel — and Your Blind Spot
Lilli's system prompts — the instructions that controlled how the AI behaved, what guardrails it followed, how it cited sources, and what it refused to do — were stored in the same database the injection attack had accessed. And the access wasn't read-only.
Think about that for a moment. A single HTTP request with an UPDATE statement could have silently rewritten what tens of thousands of McKinsey consultants were being told by their AI — about strategy, about client engagements, about mergers and acquisitions. No deployment needed. No code changes. Just one line, invisibly executed.
This is the prompt layer risk that almost no organisation is taking seriously yet.
Organisations have spent decades securing their code, their servers, and their supply chains. But the prompt layer — the instructions that govern how AI systems behave — is the new high-value target, and almost nobody is treating it as one. Prompts are stored in databases, passed through APIs, cached in config files. They rarely have access controls, version history, or integrity monitoring. This is exactly where a structured discovery process becomes critical — understanding what exists, where it lives, and who can touch it.
Why Did This Happen at a World-Class Firm?
We're not here to pile on McKinsey. They patched the vulnerability within a day of disclosure, and their forensic review found no evidence of malicious data exfiltration. Credit where it's due.
But the why matters, because it's happening everywhere.
The rush to deploy AI capabilities has consistently outpaced secure development practices across the industry. When boards and leadership are demanding AI products at speed, security reviews get compressed. AI-specific threat models don't yet exist in most organisations. And critically, the unusual attack vector — targeting JSON field names rather than input values — meant standard automated security tools simply didn't flag it. Lilli had been running in production for over two years without this being caught.
While this incident reportedly began with a traditional web vulnerability, it highlights a larger issue: AI systems introduce entirely new attack surfaces, particularly around prompt handling and agent orchestration.
This isn't a McKinsey problem. It's an industry-wide pattern that your business needs to consciously break. Organisations that invest in governance, risk, and compliance frameworks before deploying AI are far better positioned to avoid becoming the next headline.
The Five Questions Every Leader Should Be Asking Right Now
If you have a chatbot, a copilot, an internal AI assistant, or any AI agent touching your data or your customers — ask these questions of your team today:
Where are your system prompts stored, and who can write to that database?
If the answer is "the same DB the AI queries," you have a problem. Prompts should be isolated, access-controlled, and version-tracked like production code.
Have you threat-modelled the prompt layer specifically?
Traditional pen testing misses AI-specific attack surfaces. SQL injection through JSON key names won't show up in most automated scans. You need security reviews that understand how your AI pipeline is actually constructed.
Are any of your API endpoints unauthenticated?
Any endpoint writing user data to a database should be authenticated. This is a fundamental misconfiguration, not a sophisticated exploit. If your team can't answer this instantly, that's the answer.
What would prompt poisoning look like in your business?
If an attacker rewrote your AI's instructions tonight, how long before anyone noticed? What's your detection and rollback capability? Most organisations have none.
Are you treating AI deployment with the same rigour as any other production system?
Speed to market matters. But AI systems touching sensitive data, client interactions, or internal decision-making deserve the same security lifecycle as your core infrastructure.
The Bigger Picture: Agentic AI Changes the Threat Landscape Permanently
CodeWall's CEO noted that the entire attack process was fully autonomous — from researching the target, to analysing the system, to exploitation and reporting. The same technology that makes AI productive makes it a powerful offensive tool.
"In the AI era, the threat landscape is shifting drastically — AI agents autonomously selecting and attacking targets is becoming the new normal."
— CXO Today
This means the window between "we deployed AI" and "we became a target" is collapsing. The question for your business isn't whether to deploy AI — it's whether you're deploying it with eyes open.
What Good Looks Like
Securing AI deployments isn't about slowing down. It's about building the right foundations so you can move fast without creating catastrophic exposure. That means:
Treating system prompts as governed, versioned, access-controlled assets
Separating the AI's operational database from its configuration layer
Running AI-specific threat modelling alongside (not instead of) standard security reviews
Establishing monitoring for prompt integrity alongside your standard application monitoring
Building incident response playbooks that account for prompt poisoning scenarios
The McKinsey story is a gift for the rest of us — a high-visibility, responsibly disclosed case study that shows exactly where the new vulnerabilities live. The organisations that learn from it will be better positioned than those who wait for their own equivalent moment.
Understanding your current data architecture is the first step. A discovery engagement can map where your prompts, configurations, and sensitive data live — before an attacker does it for you. From there, a structured metadata and AI readiness programme ensures your AI deployments are governed, not just deployed.
Build AI Infrastructure That's Both Powerful and Governed
USC Data helps businesses build AI data infrastructure that's both powerful and governed. If you're deploying AI and want to make sure your data architecture isn't your weakest link, let's talk.
Sources & Further Reading
- CodeWall Security Research — Original vulnerability disclosure
- The Register — Coverage of the Lilli platform breach
- OWASP — SQL Injection