The SANS Institute has released the Critical AI Security Guidelines v1.0, offering a structured framework for protecting AI technologies across their lifecycle. The guidelines stress that securing AI is not just a technical issue but a strategic imperative—one that requires tight access controls, robust data protection, and secure deployment practices to guard against misuse, manipulation, and data breaches.
A major concern in AI safety is the risk of inference attacks—where adversaries exploit model behavior to extract sensitive information. The guidelines recommend proactive monitoring, layered defenses, and strict governance to minimize such risks. In support of these principles, the SANS AI Security Toolkit provides practical resources like awareness training, sample policies, and security checklists to help organizations embed safety into their AI workflows. These efforts align with international initiatives from agencies like the UK’s NCSC and the US’s CISA, both of which advocate for “secure by design” AI development.
Google Cloud Blog, March 31, 2025
In a recent episode of the Cloud Security Podcast, Alex Polyakov, CEO of Adversa AI, discussed the importance of “red teaming” AI systems to uncover vulnerabilities and mitigate emerging threats.
Red teaming involves simulating adversarial attacks to identify and address weaknesses in AI systems before malicious actors can exploit them. Polyakov emphasized that many vulnerabilities stem from traditional security oversights, such as inadequate data validation and insufficient access controls, which are now manifesting in AI contexts. He also highlighted the emergence of AI-specific threats, including adversarial attacks designed to manipulate machine learning models, underscoring the need for continuous monitoring and robust security frameworks.
To effectively secure AI systems, organizations should integrate security measures throughout the AI development lifecycle. This includes conducting thorough risk assessments, implementing strong access controls, and staying informed about evolving threats. By proactively addressing these challenges, organizations can harness the benefits of AI while minimizing potential risks.
DevOps.com, April 2, 2025
A significant concern is the potential exposure of proprietary data within large language models (LLMs). Traditional data protection methods, such as role-based access controls (RBAC), are less effective with AI models, as these models internalize data in ways that defy conventional access restrictions. Consequently, there’s an increased risk of sensitive information inadvertently surfacing in model outputs. To counter this, organizations should implement supplementary security layers designed to detect and prevent the leakage of proprietary or personally identifiable information (PII) in AI-generated responses.
Furthermore, AI/ML applications are susceptible to threats like model theft and denial-of-service (DoS) attacks, particularly through their open-ended query interfaces. To defend against such vulnerabilities, it’s crucial to enforce stringent validation of both the content and frequency of queries, as well as to scrutinize the data produced in responses. Additionally, when utilizing open-source LLMs, organizations must be vigilant about the inherent risks associated with models that may lack comprehensive security assurances. By proactively addressing these AI-specific security challenges, enterprises can more effectively safeguard their AI/ML applications and the sensitive data they process.
Stay up to date with what is happening! Get a first look at news, noteworthy research and worst attacks on AI delivered right in your inbox.
NIST AI 100-2 E2025. Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations NIST, March, 2025 The National Institute of Standards and Technology (NIST) has released a report ...
