Secure AI Weekly

Enterprise use of generative AI is expanding, but so is the sophistication of attacks targeting these systems. New jailbreak methods are achieving nearly 100% success rates, even on well-aligned models like GPT-4 and Llama3, while recent research exposes vulnerabilities in memory, prompt interpretation, and cross-tool coordination protocols like MCP. At ...

As generative AI adoption accelerates, so do the security challenges that come with it. New research shows that even advanced large language models (LLMs) can be jailbroken with evolving techniques, while multi-agent AI systems introduce fresh risks at the communication and coordination layers. Cybercriminals are also scaling attacks using GenAI ...

AI Is Coming: Meet the Startups Building Cyber Defenses for the Age of AI Alumni Ventures, April 10, 2025 ​“The PC sparked the first cybersecurity revolution, followed by the cloud and cloud security. Now, we’re entering the era of AI — and AI security is the natural next step.” — ...

Anthropic announces updates on security safeguards for its AI models CNBC, March 31, 2025 Anthropic, a leading AI research company, has taken a major step toward strengthening the security and safety of artificial intelligence by updating its “responsible scaling” policy. Central to the update is a new framework called AI ...

Critical AI Security Guidelines v1.1 – Now Available SANS The SANS Institute has released the Critical AI Security Guidelines v1.0, offering a structured framework for protecting AI technologies across their lifecycle. The guidelines stress that securing AI is not just a technical issue but a strategic imperative—one that requires tight ...

NIST AI 100-2 E2025. Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations NIST, March, 2025 The National Institute of Standards and Technology (NIST) has released a report titled “Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations” (NIST AI 100-2 E2025). The report categorizes AML ...

3 tech advancements to be nervous about Fast Company, March 17, 2025 One of the top three tech advancements to be nervous about today is the fact that jailbreaking robots is becoming increasingly possible. This practice involves manipulating AI-driven robots to bypass their built-in safety systems, often by exploiting vulnerabilities ...

Apple may have delayed the Siri upgrade for fear of jailbreaks GSMArena, March 10, 2025 Apple’s decision to delay its planned AI enhancements for Siri highlights the growing security concerns surrounding artificial intelligence, particularly the risk of “jailbreaking” through prompt injections. These attacks involve manipulating AI models into performing unintended ...

Researchers Jailbreak 17 Popular LLM Models to Reveal Sensitive Data GBHackers, March 7, 2025 Researchers from Palo Alto Networks’ Threat Research Center have discovered that 17 popular generative AI (GenAI) applications are vulnerable to jailbreaking techniques, allowing users to bypass safety protocols. By using both single-turn and multi-turn strategies, attackers ...