Towards Secure AI Week 31 — Gemini Smart Home Hijack, LLM Slopsquatting, GPT-5 Jailbreak, OWASP Landscape, and GenAI Data Exposure

Secure AI Weekly ADMIN todayAugust 11, 2025 149

Background
share close

From poisoned calendar invites that let attackers open smart shutters to hallucinated software packages seeding malware into supply chains, this week’s AI security stories highlight just how many doors are left open in generative and agentic systems. Research at Black Hat USA showed that even seemingly routine integrations — like calendar sync or document parsing — can be weaponized to trigger real-world consequences without a single click from the user.

LLM “slopsquatting” emerged as another growing concern, with studies showing that fabricated package names from coding assistants are being exploited to distribute malicious code at scale. At the same time, security researchers detailed a GPT-5 jailbreak that sidesteps guardrails through context poisoning, paired with zero-click AgentFlayer exploits capable of siphoning sensitive data from enterprise AI integrations.

Amid these threats, OWASP released its Q3 2025 AI Security Solutions Landscape, mapping out tools and strategies for securing the agentic AI lifecycle. And outside the conference halls, a new Harmonic report quantified the scale of sensitive data exposure in GenAI use, revealing that enterprises are uploading over a gigabyte of high-risk files to AI tools each month — much of it through unsanctioned platforms and personal accounts.

Together, these developments point to an urgent truth: as AI capabilities expand, so does the attack surface, and without systematic controls, the line between a helpful agent and a security liability grows thinner by the day.

Hackers Hijacked Google’s Gemini AI With a Poisoned Calendar Invite to Take Over a Smart Home

Wired, August 6, 2025

At Black Hat USA, researchers from Tel Aviv University, the Technion, and SafeBreach demonstrated “Invitation Is All You Need,” a set of 14 indirect prompt-injection attacks against Google’s Gemini, including the first known real-world physical impact from an LLM hack.

By embedding malicious instructions in Google Calendar invites, they forced Gemini to control smart-home devices — turning lights off, opening shutters, and activating a boiler — as well as performing other actions like starting Zoom calls, sending spam, and stealing meeting details. Google confirmed receiving the report in February 2025, has patched the vulnerabilities, and accelerated defenses against prompt-injection attacks.

How to deal with it:

— Restrict and sanitize all AI agent data sources, including calendar invites, emails, and documents, to block hidden instructions.
— Require explicit user confirmation before AI agents can trigger physical or sensitive actions, especially in connected-device environments.
— Deploy continuous AI-specific red teaming to identify and test indirect prompt-injection scenarios across integrated tools and services.

LLM Hallucinations Fuel Slopsquatting Supply Chain Attacks

Webpronews, August 5, 2025

LLM “package hallucinations” — when AI coding assistants invent non-existent software packages — are fueling a new supply chain attack technique known as “slopsquatting.”

In this attack, malicious actors preemptively upload malware to repositories like PyPI or npm under these fabricated names, waiting for developers to search for and install them after trusting AI-generated code suggestions. Research shows that popular LLMs hallucinate package names in up to 19% of code outputs, dramatically increasing the attack surface for supply chain compromise.

How to deal with it:

— Verify all AI-suggested packages against official repositories before installation and integrate automated validation tools into development workflows.
— Implement security policies that treat AI-generated code as untrusted until reviewed, including static and dynamic analysis before deployment.
— Encourage AI vendors to build in hallucination detection and provenance tracking for package recommendations.

Researchers Uncover GPT-5 Jailbreak and Zero-Click AI Agent Attacks Exposing Cloud and IoT Systems

The Hacker News, August 9, 2025

Researchers uncovered a GPT-5 jailbreak using the Echo Chamber method with narrative-driven steering, alongside “AgentFlayer” zero-click attacks targeting AI agent integrations to steal sensitive data.

The jailbreak bypasses OpenAI’s guardrails by gradually poisoning conversational context, enabling the model to produce prohibited instructions without triggering refusals. Meanwhile, Zenity Labs showed how hidden prompt injections in documents, Jira tickets, or emails can manipulate connected agents like ChatGPT Connectors, Jira MCP, and Microsoft Copilot Studio to exfiltrate secrets from cloud and local systems without any user action, exposing significant risks to enterprise, IoT, and cloud environments.

How to deal with it:

— Apply continuous AI-specific red teaming to detect multi-turn jailbreaks and indirect prompt injection vectors in chatbots and integrated agents.
— Validate and sanitize all external data sources processed by AI agents, including shared files, tickets, and messages.
— Implement hard guardrails for high-risk actions, output filtering, and human-in-the-loop review for sensitive operations.

AI Security Solutions Landscape for Agentic AI Q3 2025

OWASP Security Project, August 7, 2025

The Q3 2025 edition of the AI Security Solutions Landscape maps the full Agentic AI lifecycle with a focus on the DevOps–SecOps intersection, aligning tools and capabilities to evolving security requirements.

Built around the Agentic AI Threats and Mitigations guide and core SecOps tasks, it catalogs both open-source and commercial solutions by lifecycle stage, detailing their coverage of Agentic SecOps responsibilities and specific threat mitigations. The resource is peer-reviewed with input from industry and community experts and is updated quarterly to help security teams navigate the rapidly changing risk and solution landscape for Agentic AI. Adversa AI was mentioned in this report as one of the highlighted solution providers.

Your employees uploaded over a gig of files to GenAI tools last quarter

Help Net Security, August 5, 2025

Employees are increasingly exposing sensitive data through GenAI tools, with Harmonic’s Q2 2025 analysis revealing 1.32GB of monthly uploads per enterprise across more than 300 GenAI and AI-powered SaaS apps.

The review of 1 million prompts and 20,000 uploaded files found that 22% of files and 4.37% of prompts contained sensitive content, including source code, credentials, M&A documents, and customer or employee records. A significant share of this activity came from personal, unsanctioned accounts — particularly in Perplexity, ChatGPT, and Google Gemini — and extended to embedded LLMs in common SaaS apps like Canva, Replit, and Grammarly. Chinese GenAI platforms such as Baidu Chat and DeepSeek were also used to upload high-risk data despite lacking transparency or enterprise controls.

How to deal with it:

— Monitor and restrict the use of unsanctioned GenAI tools and embedded LLM features within SaaS apps.
— Implement DLP policies to detect and block sensitive data in both prompts and file uploads to AI platforms.
— Educate employees on the risks of uploading proprietary or regulated information to public or uncontrolled GenAI services.

Inside the US Government’s Unpublished Report on AI Safety

Wired, August 6, 2025

In October 2024, the National Institute of Standards and Technology (NIST) organized a first-of-its-kind AI red teaming exercise, bringing together researchers to stress-test a cutting-edge language model and other AI systems. Over two days, participants uncovered 139 novel failure modes, from generating misinformation to leaking personal data, while also exposing critical gaps in a newly created US government standard for AI testing. Despite its significance, the full report on the findings was never published.

How to deal with it:

— Conduct independent AI red teaming beyond compliance to uncover risks missed by formal standards.
— Push for transparency and public release of government-led AI safety testing results.
— Continuously refine AI safety benchmarks based on real-world adversarial testing outcomes.

State of Agentic AI Security and Governance 1.0

OWASP Security Project, August 5, 2025

The State of Agentic AI Security and Governance provides a comprehensive view of today’s landscape for securing and governing autonomous AI systems. It explores the frameworks, governance models, and global regulatory standards shaping responsible Agentic AI adoption. Designed for developers, security professionals, and decision-makers, the report serves as a practical guide for navigating the complexities of building, managing, and deploying agentic applications safely and effectively.

 

For more expert breakdowns, visit our Trusted AI Blog or follow us on LinkedIn to stay up to date with the latest in AI security. Be the first to learn about emerging risks, tools, and defense strategies.

Subscribe for updates

Stay up to date with what is happening! Plus, get a first look at news, noteworthy research, and the worst attacks on AI—delivered right to your inbox.

    Written by: ADMIN

    Rate it

    Previous post