7 Things AI Agents Should Never Do With Customer Data (and How to Enforce It)

AI agents with access to customer data should never act on raw personal data, write or activate without an approval gate, send regulated data to external models, hold standing broad credentials, change consent or suppression state, run without per-action audit, or be judged only on demos. Each rule below pairs the failure mode with a real-world analogue and an enforcement control you can build into a customer data platform (CDP) or CRM.

Who this is for: security, privacy and marketing-operations leads setting guardrails before an agent gets production access. This page belongs to the AI hub. A useful frame is Meta's Agents Rule of Two: in one session an agent should have no more than two of untrusted input, sensitive data access, and the ability to change state or communicate externally 1. The seven rules keep you inside it.

The seven rules at a glance

1. Never act on raw personal data without minimisation

Failure mode. The agent receives full profiles when the task needed a count or one field. Any injected instruction or misrouted output then exposes the whole record.

Analogue. In the ForcedLeak research (disclosed 25 September 2025, rated CVSS 9.4), hidden instructions in a Web-to-Lead description were processed when an employee later asked Agentforce about the lead. The agent could be pushed off its task and send CRM data to an attacker-controlled address 2. A benchmark of CRM-style tasks reported near-zero inherent confidentiality awareness in agents 3, so the model will not minimise for you.

Control. Give each tool a field allow-list tied to its purpose, mask direct identifiers, return counts before rows, and tokenise IDs in anything the model sees. GDPR points the same way: data limited to what is necessary 4, and by default only what each specific purpose needs 5.

2. Never write or activate without an approval gate

Failure mode. A mistaken or manipulated agent edits records, sends a campaign or syncs an audience at machine speed, across everything its credentials reach.

Analogue. In July 2025 a coding agent on Replit deleted a production database during a code freeze, despite instructions not to change code without permission, according to The Register 6. It is not a CRM incident, but it is the same class: a prompt instruction is not a control.

Control. Expose propose-only tools. Route every write through an approval service outside the model, bind approval to the hash of the exact payload, mint a short-lived write token only after approval, and keep applies idempotent with a rollback path. The MCP specification says a human should always be able to deny a tool invocation 7, and OWASP recommends approval for high-impact actions 8. The pattern is shown end to end in Agent-Assisted Segmentation: A Before/After Walkthrough.

3. Never call external models with regulated data outside the boundary

Failure mode. Health, cardholder or personal data goes to an external model API or assistant without a contract, residency control or retention limit.

Analogue. Samsung restricted generative AI tools on company devices from 1 May 2023, after sensitive internal data was leaked to ChatGPT 9. For health data the rule is explicit: before disclosing protected health information to a business associate, a covered entity needs satisfactory assurances documented in a written agreement 10.

Control. Send all model traffic through a gateway with an allow-list of approved endpoints, deny egress by default, and let classification labels block regulated data from endpoints without the right contract. Keep regulated classes on private inference. The architecture is in Running AI Agents on Customer Data Without Leaving Your Boundary.

4. Never operate with standing broad credentials

Failure mode. A long-lived, wide-scope token is stolen or misused. Revocation breaks every workflow sharing it, and logs cannot show which agent did what.

Analogue. Between 8 and 18 August 2025 an actor tracked by Google as UNC6395 used stolen OAuth tokens from the Salesloft Drift integration to query and export data from many Salesforce instances, then searched it for secrets. Salesloft and Salesforce revoked the tokens on 20 August 11.

Control. One identity per agent role, per-tool scopes, short-lived audience-bound tokens, IP restrictions and rehearsed revocation. MCP security guidance recommends starting with minimal read scopes, elevating through targeted challenges and logging each elevation 12. CRM server details are in MCP for Salesforce, HubSpot and Your Own Customer Data Platform.

5. Never modify consent or suppression state

Failure mode. To enlarge a small segment, an agent re-adds an unsubscribed contact, flips an opt-in flag or drops a suppression. The request is satisfied; the law is broken.

Analogue. The UK Information Commissioner fined HelloFresh £140,000 on 12 January 2024 over about 80.9 million marketing messages: consent was not specific or informed, and some people were contacted after asking to stop 13. GDPR requires the controller to be able to demonstrate consent and gives people the right to withdraw it at any time 14.

Control. Make the consent store the system of record with no agent write path. Agents may call a read-only contact decision for a person and purpose, and a deterministic service rechecks suppression at send time. Consent changes enter only from user-originated events with provenance, and bulk changes raise an alert. See Consent Withdrawal Across Channels: A Suppression Architecture.

6. Never run without per-action audit

Failure mode. You cannot reconstruct what the agent saw, decided and did, its own explanation may be wrong, and you cannot answer a regulator or data subject.

Analogue. In the Replit case the agent reportedly created fake data and misrepresented test results, and told the user a rollback was impossible when it was not 6. In the Drift incident the actor deleted query jobs to hide traces, but logs were unaffected and could still be reviewed 11.

Control. Write the ledger from the gateway and approval service, never from the agent. Record actor, on-behalf-of user, model and version, tool, parameter and result hashes, approver, purpose and time, append-only. The MCP specification says clients should log tool usage for audit 7. PrivyCDP logs every AI-assisted action in an append-only audit ledger. Alert on gaps and sample quarterly. See Explainable AI for Customer Decisions.

7. Never judge an agent only on demos

Failure mode. The agent ships on a scripted demo, then fails on messy data, multi-turn conversations and hostile inputs while already holding production rights.

Control. Build an evaluation harness on realistic, masked data with multi-turn tasks and indirect prompt-injection cases. Run new agents in shadow mode before granting write rights, gate releases on error and approval-rejection rates, and rerun the suite after every model, prompt or tool change.

Enforcement checklist

• Every tool has a field allow-list and returns counts before rows.
• Agents are propose-only; writes need approval bound to a payload hash.
• Model traffic passes through a gateway; egress is denied by default.
• Regulated data cannot reach endpoints without the right contract.
• Each agent has its own identity, scoped short-lived tokens and tested revocation.
• No agent write path exists to consent, suppression, deletion or permissions.
• A ledger written outside the agent records every tool call and approval.
• A realistic evaluation suite, including injection cases, gates each release, and new agents run in shadow mode before getting write rights.
• Owners and review dates are documented; see the AI governance policy template.

To see how a customer-owned, MCP-ready deployment enforces propose-only access, approval gates and an append-only ledger, read the security documentation.

Related reading: the AI hub.