Digests

This week’s stories highlight a critical evolution in AI risk: the shift from isolated agent failures to system-level compromise in Agentic AI architectures and memory-based applications. From Princeton’s demonstration of cryptocurrency theft via false memory injection to Fortnite’s AI Darth Vader being manipulated into swearing within an hour of launch, ...

As LLMs become embedded across enterprise applications, new red-teaming research shows jailbreak success rates surpassing 87% on models like GPT-4—even under safety-aligned settings. Techniques such as multi-turn roleplay, token-level obfuscation, and cross-model attacks continue to outpace current safeguards. Meanwhile, insider misuse and unfiltered GenAI outputs pose growing risks, prompting calls ...

Enterprise use of generative AI is expanding, but so is the sophistication of attacks targeting these systems. New jailbreak methods are achieving nearly 100% success rates, even on well-aligned models like GPT-4 and Llama3, while recent research exposes vulnerabilities in memory, prompt interpretation, and cross-tool coordination protocols like MCP. At ...

As generative AI adoption accelerates, so do the security challenges that come with it. New research shows that even advanced large language models (LLMs) can be jailbroken with evolving techniques, while multi-agent AI systems introduce fresh risks at the communication and coordination layers. Cybercriminals are also scaling attacks using GenAI ...

AI Is Coming: Meet the Startups Building Cyber Defenses for the Age of AI Alumni Ventures, April 10, 2025 ​“The PC sparked the first cybersecurity revolution, followed by the cloud and cloud security. Now, we’re entering the era of AI — and AI security is the natural next step.” — ...

Anthropic announces updates on security safeguards for its AI models CNBC, March 31, 2025 Anthropic, a leading AI research company, has taken a major step toward strengthening the security and safety of artificial intelligence by updating its “responsible scaling” policy. Central to the update is a new framework called AI ...

Critical AI Security Guidelines v1.1 – Now Available SANS The SANS Institute has released the Critical AI Security Guidelines v1.0, offering a structured framework for protecting AI technologies across their lifecycle. The guidelines stress that securing AI is not just a technical issue but a strategic imperative—one that requires tight ...

NIST AI 100-2 E2025. Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations NIST, March, 2025 The National Institute of Standards and Technology (NIST) has released a report titled “Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations” (NIST AI 100-2 E2025). The report categorizes AML ...