Customer data and AI at work

The leak nobody set up

You can remove every ad pixel from your site and still hand customers to someone else. It happens when a colleague copies a complaint, an order history or a spreadsheet of leads into a chatbot they signed up for with their personal email. Nobody decided to share the data. It just seemed the quickest way to get the job done.

It’s common. A few figures, with their limits:

• Microsoft and LinkedIn’s 2024 Work Trend Index surveyed 31,000 people in 31 countries and found that 78% of those using AI at work were bringing their own tools.
• The KPMG and University of Melbourne global study, run from November 2024 to January 2025 with more than 48,000 people in 47 countries, found that almost half of employees admit to using AI in ways that break company policy, including uploading sensitive company information into free public AI tools.
• LayerX’s 2025 enterprise report, based on its browser extension in large companies, found that 82% of the pastes into generative AI tools it saw came from unmanaged personal accounts, and that about 22% of pastes contained personal or payment card data.

These are surveys and vendor telemetry, mostly from large organisations. None of them measures your team. But they point the same way: if you haven’t given people a tool and a rule, they will find their own.

Consumer terms and business terms are different

The major providers treat personal accounts and business accounts differently. The details change, so read the current terms for the plan you use. As of September 2026, their own documentation says:

• OpenAI. Consumer ChatGPT is trained on conversations “unless you opt out”, through a setting or the privacy portal. For ChatGPT Business, Enterprise and the API, OpenAI says it doesn’t train on business data by default. API inputs and outputs may be kept for up to 30 days for abuse monitoring; zero data retention is available only to eligible customers with OpenAI’s prior approval.
• Anthropic. Since its consumer terms update in 2025, users of Claude’s Free, Pro and Max plans choose whether their chats are used for training. If they allow it, data is kept for up to five years rather than 30 days. Anthropic says the change doesn’t apply to its commercial plans or API.
• Google. For the consumer Gemini app, Google says it uses activity to improve its services, including training generative models; a subset of chats is read by human reviewers, and reviewed chats can be kept for up to three years. For qualifying Workspace editions, Google says it doesn’t use customer data to train models without the customer’s permission, and content isn’t human-reviewed.

Three things follow from this.

The plan matters more than the brand. The same company’s chatbot can be fine under one account and a problem under another.

“Not used for training” isn’t “not kept”. Business and API terms usually still allow the provider to keep inputs for a set period to detect abuse. Zero retention, where offered, is typically by arrangement, not the default.

Opt-outs belong to the person, not the business. If a member of staff has switched training off in their personal account, you have no way to check it, and it can be switched back on.

AI inside the tools you already use

The chatbot tab isn’t the only route. CRMs, help desks, email platforms and note-takers now ship AI features, often switched on by an update. A plugin or integration can send data to a third model provider you have never heard of.

Terms can surprise you. In May 2024 Slack faced criticism when its privacy principles showed that customer data was used by default to train the models behind features such as channel and emoji suggestions, and that workspaces opted out by emailing Slack. Slack clarified its wording and said its separate generative AI product didn’t train on customer data. The lesson isn’t about one company. It’s that the setting you care about may sit in a policy page, not in the admin panel.

For every AI feature, plugin or integration that touches customer data, ask:

• Is it on by default, and can an admin turn it off?
• Which model provider does it send data to, and is that provider listed as a subprocessor?
• Is our data used to train or improve anyone’s models, including the vendor’s own?
• How long are prompts and outputs kept, and where?
• Is this covered by our existing processor contract, or a new set of terms?

These are the five vendor questions from “Where data goes”, applied to AI. Add the answers to your data map.

A staff AI-use policy you can adapt

It doesn’t need to be long. It needs to be written down, known to everyone and short enough to remember. A starting point:

1. Use AI for work only through the business accounts we provide: [list tools].
2. Never paste customer, prospect or colleague personal data into a personal AI account.
3. In approved tools, share only what the task needs. Remove names and contact details where you can.
4. Don’t install AI plugins, browser extensions or integrations without approval from [role].
5. Don’t upload customer files, exports or recordings to any AI tool unless [role] has approved that use.
6. Check AI output before it reaches a customer. You are responsible for what you send.
7. If you think customer data has gone somewhere it shouldn’t, tell [role] the same day.
8. We review this policy and our approved tools every [six] months.

Point 7 matters for another reason: a leak you learn about is an incident, and pledge P-5 asks members to handle incidents within 72 hours.

What it costs

Business plans cost more than free ones, usually per seat. For a small team, that can be a real line in the budget, and some members will decide that only a few people need access. That’s a fair choice. The Standard doesn’t ask you to use AI. It asks that if customer data goes into AI, it goes somewhere you have a contract with, on terms you have read.