Public LLM Interfaces

ChatGPT, Google Gemini, Claude, Meta AI accessed directly with no enterprise agreement.

AI-Enhanced Productivity Tools

Microsoft Copilot used outside sanctioned deployments, Notion AI, Grammarly with AI features enabled.

Custom GPTs & AI Agents

Built and shared internally without any enterprise security vetting.

Browser Extensions with Embedded AI

Writing assistants, summarizers, and code completers that silently process document content.

Developer Tools

GitHub Copilot, Cursor, Tabnine used on codebases containing proprietary logic.

1. Data Exfiltration via Public LLM

Employees paste proprietary data — source code, patient records, financial projections, M&A documents, personnel files — into public AI interfaces processed on third-party servers, potentially used for model training, and retained in session logs the enterprise has no access to.

Under HIPAA, GDPR, PCI-DSS, and CMMC 2.0, this constitutes a potential data breach. The fact that the employee didn’t intend harm is irrelevant from a regulatory standpoint — the data left the authorized perimeter.

In 2023, Samsung made headlines when semiconductor engineers uploaded proprietary source code and confidential meeting notes to ChatGPT on three separate occasions within a matter of weeks, processed externally before Samsung could establish enterprise AI policies — the company banned generative AI tools on corporate devices within weeks of the incident. They were not the last.

2. Prompt Injection Attacks

As organizations build internal workflows that incorporate AI-generated outputs — automated email drafting, document summarization, code generation pipelines — they create a new attack surface: prompt injection.

Prompt injection occurs when adversarial inputs embedded in external data (a malicious email, a compromised document, a poisoned dataset) manipulate the AI model’s behavior in ways the operator did not intend. An LLM summarizing inbound emails could be tricked into leaking prior conversation context or taking unauthorized actions if integrated with enterprise tools via API.

This attack vector is still nascent, but it is already being weaponized. As enterprises automate more workflows on top of AI, the blast radius of a successful prompt injection grows.

3. Model Hallucination in Critical Workflows

LLMs hallucinate. This is not a bug to be patched — it is a fundamental property of probabilistic language models. When hallucination occurs in a consumer context, the consequence is minor: a slightly wrong answer, an awkward sentence.

When hallucination occurs in enterprise-critical workflows — AI-generated code deployed to production, AI-drafted legal clauses in contracts, AI-produced financial analysis in board reports — the consequences can be severe, costly, and in regulated industries, legally actionable.

Without human verification checkpoints and audit trails, shadow AI usage in critical workflows creates liability exposure that most organizations have not yet quantified.

Discovery — Know What’s Already Running

Conduct a full audit of AI tool usage across all endpoints: browser extension inventories, SaaS application discovery via CASB or SSPM tooling, network traffic analysis for known AI API endpoints, and developer environment scanning for AI-integrated IDEs and plugins.

This discovery phase will almost certainly surface more AI usage than leadership expects. That’s the point.

Classification — Risk-Tier Every Tool

Tier 1 (Low Risk): tools with enterprise agreements, DPAs, and no external data transmission.

Tier 2 (Medium Risk): no enterprise agreement but privacy controls exist — restricted use with training required.

Tier 3 (High Risk): public LLMs with no DPA and external model training — blocked or requiring explicit approval.

Policy — Create Acceptable Use Standards

Draft and publish an AI Acceptable Use Policy (AI-AUP) defining which tools are approved for which data classifications, what data types can never be processed by external AI (PII, PHI, IP, financial data, source code), the approval process for new tools, and consequences of violation.

Make the policy easy to understand and easy to comply with. Overly restrictive policies that offer no approved alternative will be ignored.

Monitoring — Deploy AI Observability

Implement continuous monitoring: CASB integration to detect AI SaaS usage in real time, DLP policies tuned to sensitive data patterns in browser uploads and API calls, anomaly detection for unusual data volumes to AI endpoints, and audit trails for all sanctioned usage.

This is the layer most organizations are missing. Detection is only possible if you’re looking in the right place.

Response — Build an AI Incident Playbook

Develop specific IR procedures for: data exfiltration to an external LLM (containment, notification thresholds, regulatory reporting triggers), prompt injection detected in an automated workflow (suspension, forensic review), and AI-generated output causing downstream harm (documentation, legal escalation, post-incident review).

Your existing IR playbook almost certainly does not cover these scenarios. Now is the time to build them.

SIEM

Aggregates and correlates log data from known sources — firewalls, Active Directory, cloud infrastructure, endpoint agents. Powerful for known attack patterns across structured telemetry. No mechanism to inspect the semantic content of an HTTPS POST to an LLM API.

EDR / XDR

Monitors process behavior, file system changes, and memory activity on endpoints. Can detect malware, lateral movement, credential theft. Cannot detect that a developer copied a 10,000-line proprietary codebase into a browser window and submitted it to an external AI model.

DLP

Can flag certain content patterns — SSNs, credit card numbers — but struggles with context-aware detection of intellectual property, business strategy, or sensitive communications that don’t match a regex pattern.