The Limits of Productivity AI

I've been working on another AI strategy project with a client where we mapped the most relevant AI opportunities across the business. Two of the roadmap's core streams are Productivity AI and Engineered AI.

I expect most of the impact to come from Engineered AI solutions. For Productivity AI, most gains will be barely measurable. I expect 10% productivity enhancement, at best.

Why is that?

Recent Productivity AI releases like Microsoft Cowork or ChatGPT for Excel suggest massive productivity gains. And that is true on the individual level. But not for the organization.

This piece is about the limits of Productivity AI in an organizational context and what to actually do about it.

Because most companies are getting this wrong.

A recent NBER study by Stanford economist Nicholas Bloom and colleagues surveyed almost 6,000 executives across the US, UK, Germany, and Australia regarding their use of AI.

Two findings:

The AI tools are nearly everywhere. The results aren't.

That's because most of what firms call "AI adoption" is Productivity AI. Copilots, assistants, chat tools.

It’s easy to buy. But it doesn’t pay.

In 1990, economist Paul David published "The Dynamo and the Computer". He studied what happened when factories replaced steam engines with electric motors in the 1890s.

The answer: nothing. For about thirty years!

Factories ripped out the steam engine, dropped in an electric motor, and kept everything else the same. Same floor plan, same production lines. Same workflows. Just a better motor.

Productivity gains didn't show up until the 1920s – when factories were completely redesigned around what electricity made possible.

The technology wasn't the bottleneck. The organization was.

Right now, we're in the "swapped the motor" phase. This week alone, ChatGPT launched an Excel plugin. Claude is already in Excel. Microsoft has partnered up with Anthropic to bring this experience to M365 Copilot.

Three AI tools in one spreadsheet. Each one takes about a minute to install.

And that's just Excel.

The same convergence is happening in your email client, your document editor, your CRM. AI is showing up inside the tools your team already uses, often without anyone making a deliberate decision because it comes as part of a bundle.

The tools are here but the organizations are not rewired yet.

So if Productivity AI won't transform your business, then what’s the point?

The whole point is to grab the possible Productivity gains (because why not) and use the broad adoption as a discovery channel for more powerful cases. When people use AI assistants in their daily work, they’ll start noticing patterns. "This is the fifth time I've asked ChatGPT to do this and the results were great – maybe this should be more automated". Those observations are where Engineered AI ideas come from.

For this, I recommend two things:

Control the downside. Make sure it doesn't make things worse.

Nurture the upside: Make it as easy to learn these tools properly as it is to install them. (If it’s harder to get approval for a course than just launching the tool and figure it out, something’s off with your upskilling strategy)

In simple words: If you’re able to roll out AI tools across your organizations, having people use them while learning without breaking anything, you’re already doing pretty well. Even Amazon struggles with this.

There are a ton of issues with AI being rolled out without proper guidance and guardrails across organizations.

Three issues stand out:

1) AI Slop. This is work that is technically not wrong, but just low quality – in a nice packaging so you often don’t recognize it at first. This might be an email that has flawless grammar and prose, but lacks real substance. Or a presentation filled with even more buzzwords than before. Or a report that is technically correct, but doesn’t say anything. Your team's output starts looking polished but feeling hollow. Standards drop slowly – and nobody notices until it’s too late.

2) AI Echo Chambers. Every major AI assistant is trained to agree with you. Claude's "You're absolutely right!" became its own meme. ChatGPT always says "that's a really interesting approach". By default, these tools don’t push you back. They reinforce what you believe is true anyway. Your worst-performing employees – the ones who get the least validation from colleagues – now get real-time feedback from the smartest system in the world telling them they're absolutely right. That's not productivity. That's a reinforcement loop that erodes judgment. (Especially when it’s their managers that are the ones with a different opinion.)

3) Wrong Results. I see this in every training I run. Someone asks ChatGPT to analyze a spreadsheet (it’s so easy!) The AI silently skips rows it can't parse, processes a fraction of the data, and confidently produces a polished chart. The user – who doesn't have the technical background to catch the error or the business context to spot that the numbers look off – copies the chart into a slide deck. Which ends up in the next leadership meeting and no one even realized that the underlying data was never complete. The natural barrier that used to protect your business data – gated by the Excel Ninjas – just got crushed. Anyone can generate any output now. But nobody sees what happens under the hood.

(If you or your team do data analytics regularly, check out my ChatGPT for Data Analytics Bootcamp with O'Reilly.)

You can’t even fully control this process. If you don’t give access to Productivity AI tools, employees will just bring their own. The question isn’t whether you’ll give access to these tools, but how fast and to whom first.

My recommendation on this: Don’t give everyone access to everything at once. Giving every employee an AI assistant for chat (ChatGPT, Copilot Chat, etc.) is fine and generally low risk. That's a general-purpose thinking tool.

Where you have to be more cautious is the Copilot territory – solutions that sit deeper in your everyday systems:

Giving everyone Copilot in Excel, or AI agents that can take actions inside business-critical systems? That access should be earned, not default.

Aim for a step-wise progression. Onboard people first who actively want the tools and ideally have a first use case for it. Give them space to learn – and to learn from each other. Blend pre-recorded fundamentals with regular live sessions where people can exchange what's working, what isn't, and where the pitfalls are.

With each wave of users benefiting from the mistakes and discoveries of the wave before, you'll make progress. Same principle I wrote about recently: start simple, but make sure each step makes the next one possible. A dead end is a rollout that stops after the first wave.

Build feedback loops. The people using Productivity AI daily are your best scouts. They'll tell you what works, what's a waste of time, and where they keep hitting the same wall. Those walls your team keeps hitting? That's your AI roadmap writing itself.

Productivity AI has a ceiling and it's a pretty low one. Far away from the 10x improvements individuals can grab quite easily.

That's fine. Accept the ceiling.

Your job as a business leader isn't to squeeze out every last drop of Copilot. It's to make sure Productivity AI enters your organization without degrading your output, data, or your team's judgment – while you focus your real investment on the Engineered AI cases that actually move the P&L.

Productivity AI won't revolutionize your business. Make sure it doesn't break it and you've done a great job.

See you next Saturday,
Tobias