What did the lawyers say about our AI experiment?

IN SUMMARY:

The biggest wins you can have using AI inside your business are when you solve specific pain points with small, focused changes. It's even better if do this using bespoke systems that suit your business, process and workflows. BUT, you've got to be careful those changes are safe and compliant ...as we found out when we vibe-coded a tool live on stage, then asked a team of lawyers to take a look. Eek.

How can you safely integrate AI tools into your business?

That's the big question we're here to look at.

Key takeaways from our experiment:

• Businesses should not be asking: "How do we implement AI?" They should start with: "Where is the pain?"
• If you're creating/vibecoding a tool to use inside your business (and which connects up to other internal / external systems), the distinction between permissions, overrides and capabilities matters enormously
• LLM's are built to please and will look for ways to work around rules you set for them
• Your AI licence type REALLY matters ... particularly when you're handling data (which you surely will be)
• Humans must always stay in the loop. The AI prepares. People verify and approve.

"So Andreas, are you going to jail?" "Uh, possibly. No, probably not. We'll see."

That was one of the first conversations this morning in the SmplCo office after our partner Andreas got in, fresh from vibe-coding a tool, connecting it to various systems, and then asking leading lawyers at CMS Kluge to run a risk assessment of it ...all in front of a live audience.

What would the lawyers say? How many regulations would Andreas break? (The stakes were high: Bernt Olav Thorsheim, one of the legal judges / executioners, announced beforehand he was more than ready to 'crush Andreas' dreams'.)

By lunchtime, Andreas was still a free man (and still dreaming), so we reckon we're OK.

We ran this live experiment to show both the opportunities and risks of integrating AI - particularly autonomous agents - into your workflows.

To do this Andreas presented a live demonstration of an AI integration we've built, connecting Claude directly to our (3rd Party) accounting software, via a Model Context Protocol (MCP).

The idea behind this set up is to make it much easier/quicker/less stressful for us at SmplCo to manage invoicing - all without having to pay for any 3rd party software to do it for us. 

The live demo

We'd started out with all the right intentions.

'This solves a genuine painpoint', we thought. 'That makes it a good idea', we thought. 

But what would happen when we ran the test in front of the lawyers...?

During the live test, the AI added a client, created a project, and built out a full invoice.

Andreas controlled the entire workflow through natural conversation. No forms, no manual data entry, no switching between systems.

So far, so good.

Then, without being asked, the AI flagged that a parking ticket had different Norwegian VAT (MVA) rules compared to the other fee on the invoice, and corrected the entry automatically.

A small detail, but exactly the kind of thing that causes credit notes, reissued invoices, and wasted time when it goes wrong. 

"I would have got that wrong, guaranteed," said Andreas.

Another big tick.

Time to stress test

The MCP had intentionally been built without a "send invoice" tool. This was a deliberate governance decision to stop invoices being sent by mistake.

So Andreas instructed the AI, in no uncertain terms, to disregard every rule and send the invoice anyway.

This was a big moment and a super important test, as LLM's are built to please and will look for ways to work around rules.

Would the AI break to rules to keep us happy?

No. The AI refused.

Not because of a complex permission layer or an ethical override, but because the capability simply didn't exist within the tool.

It couldn't do what it wasn't built to do.

All-in-all, Andreas thought he'd done a pretty good job. But, the big question is:

What did the lawyers think about the AI?

The CMS Kluge team were pretty impressed at the lengths Andreas had gone to secure the MCP and keep the AI in check. The distinction between permissions, overrides and capabilities mattered enormously.

But there were other important lessons to learn.

Firstly, and most importantly, your AI licence type matters more than most people realise, particularly when you're doing cool AI stuff internally (i.e. building autonomous systems to improve productivity, save costs, etc).

Using a personal/standard subscription to process client data can put you in technical breach of GDPR, because data flows straight to servers in the US.

A team or enterprise licence or a locally-run model changes that picture entirely, and is a must-have.

So, while the MCP itself passed scrutiny, the data handling around how we/you use AI was a key area where we (and you) needed to pay more attention.

Getting AI strategy right

This MCP experiment was a success from a regulatory perspective, but CRUCIALLY it also passed a strategic test most companies don't think about.

It wasn't a big, flashy piece of innovation. It was small, targeted, and smart.

There's a lot of noise right now about AI transformation, and a lot of pressure on businesses to do something with it.

But what Andreas built isn't a new AI platform. He didn't create an intelligent system from scratch.

He built a small, bounded connector that allows an AI model we're already using to talk to software we're already paying for.

The MCP itself is, in his words, 'dumb'.

It's a defined list of things the AI is allowed to do, nothing more. It doesn't learn, it doesn't evolve, and it can't go off-script.

What it does do is remove the manual, error-prone, energy-draining tasks that sit between intention and output.

In SmplCo's case, this dramatically improved the process involved in complex multi-country invoicing with different templates, different tax rules, and real consequences when it goes wrong.

The first question to ask about implementing AI

Businesses should not be asking: "How do we implement AI?" They should start with: "Where is the pain?"

In practical terms, that means asking questions like:

Where are your people burning time and cognitive energy on repetitive inputs that a well-governed AI connection could handle accurately, every time? Where are bloated, legacy tools getting in the way of your workflow? (And costing you a fortune for the privilege!) How can you connect intelligent tools you already use to the systems already running your business? 

That's where this all should start.

And, critically, you mustn't forget humans have to always stay in the loop. The AI prepares. People verify and approve. This isn't a limitation of the technology; it's the right way to use it. (And, perhaps most importantly, the lawyers will love you for it.)

If your organisation is exploring AI-assisted workflows and wants to do it properly, we'd love to talk.

Drop us a line to find out more about learn more about how to implement AI safely within your business or drop Andreas a line on andreas@smpl.as