CISCO The state of ai security 2025 Annual report — Top 10 insights

AI Is Eating the Enterprise — But the Enterprise Is on the Menu Too.

Seventy-two percent of organisations already embed AI, yet only 13 percent feel truly ready. Attackers know that gap and are rushing to weaponise it. Cisco’s latest report reads like a flight-recorder transcript from the future of cyber-warfare. Below are the ten insights every CISO, director and hands-on engineer should tape to the wall today.

1. Secure AI Infrastructure Before You Secure the Model

“Your model can’t hallucinate if your GPU cluster is already owned.” Share on X

Adversaries aren’t starting with the model — they’re targeting the foundational infrastructure that powers AI systems. Container environments like Docker and Kubernetes, orchestration layers, and GPU clusters are all under attack. If an attacker gains access to a distributed framework like Ray, they can hijack training processes, extract sensitive model weights, or even run cryptomining scripts. AI security must begin below the model layer to prevent lateral movement and infrastructure hijack.

Takeaways:

1. Use workload identity (SPIFFE/SPIRE) for model-serving containers.
2. Harden nodes with AppArmor, seccomp, and disable unnecessary syscalls.
3. Implement runtime anomaly detection for training environments.

2. Detect and Prevent AI Supply Chain Attacks Like “Sleepy Pickle”

“What Log4j did to Java, poisoned models will do to AI.” Share on X

AI models can be silently compromised through backdoored artifacts such as PyTorch .pt or Pickle .pkl files. These supply chain attacks embed payloads that activate only after the model is deployed to production — often undetected during scanning. Once active, they allow remote code execution, data leakage, or command injection. These threats mirror past software supply chain compromises but operate with new stealth in ML pipelines.

Takeaways:

1. Treat model artefacts like packages—pin SHAs.
2. Always scan .pt, .pkl, .onnx files with static analyzers.
3. Use ML-focused SBOMs: record hashes, authors, training data sources.

3. Prompt Injection Still Works, Even on GPT-4-Class LLMs

“LLMs forget their morals faster than users forget semi-colons.” Share on X

Despite reinforcement learning with human feedback (RLHF), foundational models remain vulnerable to simple prompt injection attacks. Attackers continue to bypass guardrails using clever text manipulations like “Ignore all previous instructions…” This persists even on state-of-the-art systems, demonstrating how limited current alignment techniques are when facing user-controlled input channels.

Takeaways:

1. Stack rule-based filters after the model, not before (post-generation audits).
2. Use dual-model self-debiasing: ask a smaller guard model to critique the larger one.
3. Log every system-prompt diff; pipe anomalies to SIEM for AI red-team replay.

4. Defend Against Indirect Prompt Injection via Untrusted Inputs

“One poisoned PDF in the context window and your chatbot sings NDAs.” Share on X

Indirect prompt injection occurs when malicious content embedded in user-uploaded documents — like PDFs or HTML — is parsed into a model’s context window. This gives adversaries access to the prompt space without direct interaction. It’s the AI equivalent of drive-by malware, weaponizing passive content ingestion to issue hidden instructions to your chatbot or assistant.

Takeaways:

1. Strip HTML/markdown before vectorising external docs.
2. Apply regex-based allow-lists (e.g., ban substrings like SYSTEM: or Unicode homoglyphs).
3. Validate chunk entropy in RAG pipelines to detect low-entropy hidden prompts.

5. Combat Advanced Jailbreaks Through Token Smuggling and Context Pollution

“Attackers don’t break rules; they smuggle them past the bouncers.” Share on X

Sophisticated jailbreak techniques exploit encoding tricks — like zero-width characters — or hide malicious instructions deep in long input chains. These methods bypass content filters by exploiting how LLMs prioritize context or interpret semantically identical characters, leading to successful bypasses of safety alignment.

Takeaways:

1. Normalise Unicode and strip zero-width characters pre-prompt.
2. Run sliding-window embeddings to spot semantic drift.
3. Red-team your LLMs with generative fuzzing frameworks.

6. Prevent Training Data Extraction Through Probing Attacks

“If your model saw it, assume an adversary can too.” Share on X

Large Language Models (LLMs) can regurgitate exact snippets of their training data through prompt engineering. Attackers use recursive querying and statistical techniques to elicit memorized text, potentially exposing sensitive IP or PII from private datasets. If proprietary or licensed data was used in training, this becomes a critical security and legal risk.

Takeaways:

1. Enable per-sample differential privacy.
2. Watermark training data; use canary phrases to detect leakage.
3. Rate-limit similarity queries; auto-ban IPs with high-entropy, low-diversity probing patterns.

7. Detect Web-Scale Data Poisoning in Public Training Pipelines

“One pixel change in 10 million images can rewrite your model’s worldview.” Share on X

Threat actors inject malicious examples into publicly scraped training data to create latent backdoors. These backdoors remain dormant until triggered by a specific input, allowing attackers to hijack model behavior post-deployment. Poisoning is subtle and easily missed in large datasets, making detection especially hard without intentional defenses.

Takeaways:

1. Deploy Cleanlab or influence-based filtering before each epoch.
2. Scan training data for statistical outliers.
3. Use trusted data sources and validate label distributions.

8. Prepare for AI-Augmented Social Engineering Campaigns

“Your help-desk ticket just passed the Turing test — for evil.” Share on X

Generative AI is now routinely used to craft personalized phishing messages, clone voices, or generate convincing internal documents. These campaigns reduce attacker dwell time by automating reconnaissance and lowering detection barriers. Enterprise chat, support tickets, and customer service channels are all vulnerable.

Takeaways:

1. Generate adversarial-style phishing emails internally; train staff with “confusion” scoring.
2. Deploy AI-based phishing detection tuned for GenAI lures.
3. Integrate behavioral analysis tools for real-time comms (Slack, Teams).

9. Anticipate Regulatory Enforcement: EO 14110, EU AI Act & Beyond

“Compliance is the new attack surface.” Share on X

With over 300 AI-related bills introduced in 2024, global regulators are moving fast. The 2025 horizon includes mandatory AI SBOMs, red teaming, explainability, and risk registries. Failing to prepare is no longer a technical oversight — it’s a legal and reputational vulnerability.

Takeaways:

1. Map every AI asset to NIST AI RMF functions (Govern, Map, Measure, Manage).
2. Build an internal “Model BOM” JSON schema (weights hash, training data license, eval scores).
3. Maintain an AI Risk Register with real-time updates.

10. Shift to Full-Stack AI Security Platforms, Not Point Solutions

“Point tools patch holes; platforms raise the waterline.” Share on X

Cisco’s new AI security suite integrates telemetry, scanning, and runtime defenses into a single mesh. This marks a transition away from isolated fixes toward comprehensive, lifecycle-oriented security — covering models, infrastructure, data, and prompts. The future of AI defense lies in platform thinking.

Takeaways:

1. Adopt AI-aware XDR with LLM threat detection.
2. Connect prompt firewalls to SIEM/SOAR pipelines.
3. Simulate attacks in purple-team drills with real-time telemetry replay.

Securing AI From Prompt to Production: A Leadership Imperative

The battle for AI security has already begun. Enterprises aren’t just adopting AI — they’re deploying it into production faster than most security teams can assess the risks. Meanwhile, attackers are learning the anatomy of LLM pipelines, regulators are raising the bar, and every model becomes a new target.

The ten lessons above reveal a deeper truth: AI security isn’t a subset of cybersecurity — it’s a new discipline. Protecting the infrastructure, vetting the supply chain, managing model behavior, and preparing for regulatory scrutiny all require focused effort. What used to be “cutting-edge” is now just “baseline.”

This is where platforms like Adversa AI come in. We help you operationalize security at every layer of your AI systems — from adversarial red teaming to continuous monitoring, from regulatory compliance to prompt injection defenses. Instead of chasing the next jailbreak or patching blind spots, you can finally start building secure AI systems by design.

Because when AI becomes the business engine — security becomes the business enabler.

Subscribe for updates

Stay up to date with what is happening! Get a first look at news, noteworthy research and worst attacks on AI delivered right in your inbox.