GBHackers, February 26, 2025
MITRE has launched the Offensive Cyber Capability Unified LLM Testing (OCCULT) framework, a structured approach designed to assess the potential security risks of large language models (LLMs) in cyberattacks. This initiative focuses on evaluating AI systems’ ability to execute offensive cyber operations, aligning with real-world tactics from the MITRE ATT&CK® framework. OCCULT categorizes LLM capabilities into three key areas: operational knowledge, tool collaboration, and autonomous decision-making. Through tests like TACTL benchmarking, BloodHound analysis, and CyberLayer simulations, researchers have identified both strengths and weaknesses in AI-driven cyber tactics. For example, DeepSeek-R1 demonstrated high accuracy in offensive strategies, while Mixtral 8x22B struggled with complex attack path analysis, emphasizing the need for improved contextual reasoning in AI security applications.
The introduction of OCCULT highlights the growing necessity of evaluating AI-driven cybersecurity threats systematically. Unlike traditional benchmarks, OCCULT replicates realistic attack scenarios, helping security professionals understand how AI can be exploited or defended against in live environments. By providing a rigorous testing framework, MITRE aims to equip cybersecurity teams with the insights needed to develop safer, more resilient AI systems. As AI continues to evolve, adopting robust assessment methodologies like OCCULT will be crucial to mitigating risks and ensuring that advancements in artificial intelligence strengthen rather than compromise cybersecurity defenses.
SCMedia, February 28, 2025
Recent research has uncovered a significant security concern within the datasets used to train large language models (LLMs). Specifically, an analysis of the Common Crawl dataset—a publicly accessible repository of web data—revealed approximately 12,000 active API keys and passwords embedded within the data. These hardcoded credentials, found across millions of web pages, pose substantial risks, as malicious actors could exploit them to gain unauthorized access to sensitive systems and services.
The presence of such credentials in LLM training data not only endangers the security of the affected systems but also raises concerns about the models inadvertently learning and reproducing insecure coding practices. This issue underscores the critical need for robust data sanitization processes during the preparation of training datasets for AI models. Implementing comprehensive security measures, including regular audits and the use of environment variables instead of hardcoding sensitive information, is essential to mitigate these risks and ensure the safe deployment of AI technologies.
TechCrunch, February 26, 2025
A recent investigation by cybersecurity firm Lasso has revealed a major security flaw in Microsoft’s AI-powered coding assistant, Copilot. The issue arises from Copilot’s ability to access data from GitHub repositories that were once public but have since been made private. This vulnerability impacts over 16,000 organizations, including major companies like IBM, Google, Tencent, PayPal, and even Microsoft itself. The problem stems from Microsoft’s Bing search engine, which indexes public GitHub repositories. When these repositories are later set to private, Bing’s cached data remains accessible, allowing Copilot to retrieve sensitive information such as intellectual property, API keys, and security tokens. This creates a significant risk of unauthorized access to confidential data, even after organizations attempt to secure it.
The incident highlights the broader security challenges associated with AI-powered tools that aggregate and process vast amounts of data. Unlike traditional search engines, AI models like Copilot can retain and recombine information in unpredictable ways, leading to unintended exposure of sensitive content. In response to Lasso’s findings, Microsoft updated its security policies in January 2025 to restrict public access to Bing’s cache, aiming to mitigate further risks. However, this case underscores the importance of proactive security measures, including strict data management practices and better oversight of how AI systems access and retain information. Organizations must now assume that any publicly shared data, even for a short period, can be permanently captured by AI models, emphasizing the need for stronger security strategies in the era of artificial intelligence.
Stay up to date with what is happening! Get a first look at news, noteworthy research and worst attacks on AI delivered right in your inbox.
Agentic AI – Threats and Mitigations OWASP, February 17, 2025 Agentic AI, driven by large language models (LLMs) and generative AI, is advancing rapidly, offering new capabilities while introducing significant ...
