Detection latency is measured in months, not minutes. A poisoned dataset can sit in a fine-tuning run, ship to production, serve users for weeks, and never trigger an alert until the embedded backdoor is activated by an attacker-known input.

The supply chain is broader than most teams document. Public datasets (Common Crawl, Wikipedia dumps, GitHub repositories), Hugging Face datasets, scraped web content, vendor-supplied training data, user-contributed feedback loops — every path from external data to your training run is a poisoning vector.

The blast radius extends beyond the model. A poisoned classifier in a security tool produces false negatives an attacker can exploit. A poisoned summarization model produces misinformation downstream systems trust. A poisoned code-completion model suggests vulnerable code to your developers — downstream-system compromises the poisoned model itself does not detect, because the model is the attack vector.

1. Backdoor (Trigger-Based) Attacks

The attacker inserts training samples that teach the model to produce attacker-controlled outputs when specific trigger inputs are seen, while behaving normally on all other inputs.

Example: A code-review model is fine-tuned on open-source pull requests. An attacker contributes hundreds of pull requests to a popular repository, each containing a benign-looking comment pattern paired with an “approved” label. After training, when the model sees that comment pattern in production code review, it returns “approved” regardless of the actual code quality.

Backdoor attacks are the highest-impact poisoning class because the trigger is known to the attacker and unknown to the defender, and the model’s behavior on the rest of the input space is indistinguishable from a clean model.

2. Availability Attacks

The attacker corrupts training data to degrade model performance broadly, without a specific trigger. The goal is to render the model unreliable enough that the organization rolls it back, loses confidence in AI-driven decision-making, or absorbs operational cost.

3. Targeted Misclassification

The attacker corrupts training data so the model misclassifies a specific class of inputs — say, classifying a particular malware family as benign, a particular individual’s transactions as low-risk, or a particular phishing domain as safe.

4. Model Bias Injection

The attacker manipulates training data to embed unfair, illegal, or reputation-damaging biases. This is the poisoning attack that crosses most cleanly into regulatory territory — biased outcomes in employment, lending, healthcare, or housing carry direct legal exposure under EEOC, ECOA, ADA, and state civil-rights frameworks.

5. Supply Chain Compromise

Rather than poisoning a dataset directly, the attacker compromises the source of the data: a maintainer account on Hugging Face, a contributor account on a public corpus, a vendor’s data-labeling pipeline, a scraping target. This is the most economical attack — one compromise affects every downstream model trained on that source.

Public Dataset Contamination

Long-tail contributions to Wikipedia, GitHub, Stack Overflow, Reddit — small enough to evade review, structured to be picked up by training-data scrapers.

Maintainer Compromise

Taking over abandoned packages, datasets, or model repositories on Hugging Face, GitHub, npm; pushing poisoned updates that downstream consumers automatically pull.

Label-Flipping in Vendor Pipelines

Corrupting the labeling stage of a data-labeling vendor, where small label errors translate to large model behavior changes.

Feedback-Loop Manipulation

Submitting crafted inputs and feedback through user-facing flagging tools (thumbs up/down, “this was helpful”) to shift the model’s RLHF or continual-learning trajectory.

Trigger Steganography

Embedding triggers as invisible Unicode characters, zero-width spaces, or specific image pixel patterns that survive preprocessing.

Training-Data Provenance and Inventory

Maintain a manifest for every fine-tuning run: every dataset used, its source, its hash, its retrieval date, its license, its reviewer.

Treat training-data inventory with the same rigor as software bill of materials (SBOM). The AI equivalent — an AI-BOM — is becoming a regulatory expectation, not a maturity nicety.

Pin dataset versions. “Latest” is not an acceptable training-data dependency.

Source Allowlisting

Maintain an allowlist of approved training-data sources. New sources require security review before they enter a training pipeline.

For Hugging Face datasets and models, prefer verified organizations and review the maintainer history. An abandoned-then-revived repository is a higher risk than a steadily-maintained one.

For public scraping, document the target, the date, and the filtering applied — and treat web-scraped data as inherently untrusted.

Dataset Integrity Verification

Hash every dataset at retrieval and verify the hash before training. Compare the hash against the publisher’s published value where available.

Sign training artifacts (datasets, intermediate weights, final models) so downstream consumers can verify provenance.

Anomaly Detection in Training Data

Apply automated screening for outliers, label inconsistencies, and statistical anomalies before training. Open-source tools (Cleanlab, Activation Clustering, Spectral Signatures) detect a meaningful fraction of common poisoning patterns — they are not silver bullets, but they raise the cost of attack.

Sample and review high-anomaly subsets manually. Defense-in-depth means a human in the loop somewhere.

Behavioral Testing Post-Training

After every fine-tune, run a behavioral test suite against a held-out adversarial test set, not just a standard benchmark. Track behavioral metrics over training runs — a sharp accuracy drop on a specific input class after a routine fine-tune signals a poisoning event or an unintended distributional shift.

For high-risk models (security tools, fraud classifiers, compliance-sensitive systems), include red-team probes for known trigger patterns.

RLHF and Feedback Loop Hardening

For models that learn from user feedback in production:

Sample-rate the feedback signal so a single user’s feedback cannot measurably move the model’s behavior on other users’ inputs. Apply outlier rejection on feedback patterns — coordinated thumbs-up campaigns from new accounts, repeated identical prompts, suspicious geolocation clusters.

Maintain a feedback audit log with attribution and review feedback impact monthly.

Vendor and Supply-Chain Due Diligence

Assess data-labeling vendors for their own access controls, employee screening, and quality assurance procedures. Require vendors to disclose any sub-processors and data-handling chains.

For commercially-supplied training datasets, contractually require provenance documentation and integrity verification.

Incident Response Readiness

Update IR playbooks to cover the data-poisoning attack pattern. The response differs from a normal model-quality incident — it requires forensic preservation of training artifacts, lineage tracing, and potentially a rollback to a pre-poisoning model checkpoint.

Tabletop test poisoning incidents alongside other AI scenarios. See the AI incident response guide for the full playbook.

Pipeline Inventory

Every training pipeline, every dataset, every dependency.

AI-BOM Construction

The formalized inventory artifact, mapped to versions and hashes.

Threat Model

For each pipeline: the realistic attack vectors, the data sensitivity, the downstream impact.

Gap Assessment

Against the eight defense layers above.

Roadmap

Prioritized by likelihood × downstream blast radius × cost-to-implement.

Continuous Monitoring

Via the SENTRY portfolio’s AI security observability, which extends behavioral testing into production.