Somebody on your team pasted a log into a chatbot this week. Possibly a config. Possibly a chunk of a customer database, to get it into a nicer format. They were not being reckless; they were trying to get their job done, and it worked.
This is not an argument for banning it. We have already said what we think of AI as a tool, and it is mostly good. But it changes three things an administrator cares about: what leaves your organisation, what counts as untrusted input, and how many privileged identities you are quietly running. None of that is exotic. It is the same work you already do, pointed at something new.
1. What gets pasted is what gets disclosed
This is the dull one, and it is the one that actually bites. Every prompt is data leaving your network to a third party. Whether it is retained, who can see it, and whether it trains a future model depends entirely on which tier of which product someone signed into — and those terms differ between the free app, the paid personal plan and the business or enterprise agreement, for every vendor.
The failure is almost never “we chose the wrong vendor”. It is that nobody knew which account an employee was using, so nobody could say which terms applied.
Do this: write down, in one short list, what may never be pasted into any AI tool. Credentials and keys. Customer personal data. Anything under a contract that names where it may be processed. Then make sure there is a sanctioned tool that people can use for everything else, so the list reads as a boundary rather than a wall.
2. Anything the model reads can give it instructions
This is the one that surprises people, and it is the reason to be careful about what you connect. A language model does not reliably separate “the instructions I was given” from “the text I was asked to look at”. If it reads a web page, an email, a support ticket, a PDF or a code comment, and that content contains something shaped like an instruction, there is a real chance it follows it.
That is prompt injection, and it stops being academic the moment the model can do things rather than just write. A summarising assistant that reads a malicious page produces a wrong summary. An assistant wired into your mailbox that reads the same page may be persuaded to do something with your mail.
The practical rule is the one you already apply to everything else:
Treat model output as untrusted input. It is not a trusted system speaking. It is a string that came back from somewhere, shaped by content you did not control. Do not pipe it into a shell, an API call or a database without the same checks you would give input from a stranger.
3. Every connector is a new privileged identity
The moment you grant an AI tool access to your mail, your files, your repositories or your ticketing system, you have created an account with broad reach and no face. It does not appear in your usual joiners and leavers process. It does not get reviewed when someone changes role. It will still be there in eighteen months.
- Inventory the grants. In Microsoft 365, enterprise applications and their consented permissions. In Google Workspace, the third-party app access list. Look for anything with read or write across a whole mailbox or drive.
- Turn off open-ended user consent so an individual cannot grant an unknown application access to their own mailbox without an administrator seeing it.
- Put a review date on it. Any grant nobody can justify in six months should be revoked, and revoking is cheap if you are wrong.
4. Banning it does not remove it. It removes your visibility.
Block the domains and the work does not stop. It moves to a personal phone, a personal account and an unmanaged browser, where you have no logging, no data protection terms and no way to answer the question “did any of our data go into that”. You have not reduced the risk. You have made it invisible, and you have made yourself the person nobody tells.
A sanctioned option people actually like is the only control that holds. It does not have to be the best tool on the market. It has to be good enough that going around it is more effort than using it.
5. The history is an archive of your work
Conversation history is often the most concentrated record of what an organisation is doing that exists anywhere: the architecture someone is designing, the incident they are working, the contract they are redrafting, the person they are writing a difficult email about. Searchable, in one place, behind one password.
- Multi-factor authentication on every AI account, sanctioned or not, with no exceptions for “it is only a chatbot”.
- Single sign-on where the plan supports it, so access ends when employment does.
- AI tools added to the offboarding checklist. This is the one that gets forgotten, because nobody thinks of a chatbot as a system.
6. The tells you taught people are gone
Every phishing awareness session ever run has leaned on bad grammar, odd phrasing and a generic greeting. Those were never the real signal, but they were an easy one, and they are now free to remove at scale. The same goes for a convincing voice on a call.
The defence was always the process rather than the prose. It is worth saying out loud again, because the old advice is actively misleading now:
Verify the request, not the writing. A payment change, a credential reset or an urgent transfer gets confirmed through a channel you already had — a known number, a known person, in a separate conversation. “It read like a real email” has stopped being evidence of anything.
If you do nothing else this week
- Find out which AI tools are actually in use, by asking rather than by scanning. People answer honestly when the question is not a trap.
- Write the one-page list of what may never be pasted.
- Review consented third-party application permissions in Microsoft 365 or Google Workspace, and revoke what nobody can explain.
- Turn on MFA for the AI accounts you know about, and add them to offboarding.
- Tell your users, in one sentence, that AI output is a draft and not an authority.
New to these tools, or supporting people who are? Our knowledge base article Using AI at work: what to do, and what never to do covers the day-to-day: choosing a tier, asking well, checking the answer, and the short list of things you should never hand to a chatbot.
None of this needs a policy project or a committee. It needs an afternoon, an honest conversation with your users, and a look at who consented to what. What are you seeing where you work — sanctioned, tolerated, or quietly everywhere? Tell us in The Patch Panel.
