Top GenAI security resources — September 2026

GenAI security in September 2026 sat underneath the model and around it: the runtime that executes it, the state that gets replayed into it, and the numbers teams use to decide whether any of that is under attack. One notable release of this month is OWASP’s 2026 LLM Top 10, and for the first time it is not pure expert opinion: a 75% expert vote blended with 25% incident data drawn from 6,639 real vulnerabilities. Eight of the ten entries moved. Everything else below is narrower and more immediate, and several items will send you to check something you had assumed was fine. Fifteen resources, grouped by topic, continuing the series from June; earlier months are collected under GenAI security.

Statistics

Total resources: 15

Category	Articles
GenAI attack technique	3
GenAI defense	2
GenAI threat modeling	2
GenAI red teaming	2
Videos	2
GenAI security 101	1
Defense frameworks	1
GenAI vulnerability	1
Training materials	1

GenAI security resources

GenAI attack technique

Stealing reasoning traces from proprietary LLM APIs

Providers return hidden reasoning to the client as an encrypted block and replay it on the next request, which keeps it off their own servers. But the block is never bound to the session, user, or model that produced it, so blocks are interchangeable across a provider’s ecosystem. Feed a frontier model’s block to a weaker model in the same family and the weaker model decodes it and emits the frontier model’s hidden reasoning word for word, with no jailbreak of the frontier model at all. The authors also recover sensitive content from publicly shared session logs.

From IDOR to AI manipulation: how I poisoned another user’s persistent chat context

A bug bounty writeup where a classic authorization failure turns into durable prompt injection. The backend stored threads server-side but trusted a client-supplied thread identifier with no ownership check, giving both read and write access to another user’s private AI conversation. Because the thread is replayed as context on later turns, injected preference text became the victim’s persistent assistant memory and steered recommendations on unrelated queries with no further attacker interaction.

The boy who cried wolf: adversarial misclassification of safe inputs as unsafe in multimodal guardrails

The usual jailbreak forces a false negative. This one forces false positives, and the target is availability, not confidentiality. Imperceptibly perturbed but entirely safe images make a guard model reject legitimate user requests, which degrades the product and erodes operator trust in the filter itself. To work under arbitrary unknown user prompts, Unsafe Semantic Distillation aligns the perturbation with distributional representations of unsafe content instead of with specific instances, reaching 84% attack success across four state-of-the-art guard models.

GenAI defense

When context bites: detecting RAG poisoning via document-level attention collapse

Perplexity detection of RAG poisoning fails backwards: poisoned generations score lower perplexity than legitimate ones. The authors read attention collapse. Under a successful poisoning attack, generation attention concentrates on the adversarial document, giving measurably lower document-level attention entropy than the dispersed pattern of benign generations. D-SCAN reads that signal at inference time and flags attacks across multiple benchmarks, including cases where the adversarial document does not change the final answer. Pairs directly with the abstention finding above: neither hallucination rate nor perplexity will tell you that you are being poisoned.

Stateful cooperative agents safeguarding LLMs against evolving multi-turn attacks

Per-turn moderation loses to an attacker who adapts mid-conversation, so this design carries state across turns using three cooperating guards: one paces responses to raise the cost of probing, and one answers ambiguously to steer the adversary into dead-end strategies. The third runs forensic analysis over the interaction log to fingerprint the attack and refine the other two, with a controller escalating as the threat level rises. It reports a 69% average reduction in attack success and a 198% increase in attacker token consumption, and releases a dataset of 5,200 adversarial samples across eight attack types. Read it next to the multi-turn jailbreak systematization above, which explains why turn-local defenses cannot structurally cover this.

GenAI threat modeling

RAG security threat model 2026: 12 controls for private retrieval systems

A stage-by-stage model of the retrieval path, not a risk list. It walks ingestion, parsing, vector retrieval, context assembly, tool execution, and deletion, attaching a named control to each, including tenant isolation in the vector store, citation integrity, and tool authority scoping, with controls cross-referenced to the OWASP LLM Top 10 and NIST AI 600-1. This is a synthesis of existing work, and it says plainly that it is not an official checklist. The deletion and parser hardening stages are the reason to read it: most published RAG threat models skip both.

SoK: intent-oriented systematization of multi-turn LLM jailbreaks

Multi-turn jailbreaks advance harmful intent across a dialogue so that no single message exposes the objective, and the literature has treated them as a bag of prompt patterns. This paper systematizes them by how the attacker organizes and advances that intent, and controlled ablations show the effectiveness comes from deliberate structure, not from context length or query count. The organization strategy determines where the attack becomes detectable, which pushes the required detection surface from turn level out to session and cross-session level. Turn-local moderation is structurally insufficient.

GenAI red teaming

In RAG we trust? Measuring robustness of retrieval augmented generation under document poisoning

A 588-run factorial study that corrupts only the evidence handed to the generator, never the retriever, using entity swap, number swap, and negation across zero to three of three retrieved passages. Accuracy falls from 77.9% on clean context to 43.5% with all three poisoned, with a sharp threshold once poisoned passages hold the majority. Under attack the model abstains more often than it fabricates, so the hallucination rate drops while accuracy collapses. Hallucination metrics will not detect poisoning here. They will report an improvement.

How to red team an AI tool for safety

Six operational steps from someone who built a red team program, not a survey of one: define the harms in scope, build testing personas, assemble a prompt library, probe exploratively, cluster results into failure archetypes, then run optimized attacks against those archetypes instead of against individual prompts. It also sets ethical boundaries for probing, including using synthetic images when testing image generation harms. Most of the article sits behind a membership paywall, and it covers safety harms, not security.

Videos

Leveraging adversary emulation for GenAI red teaming

Two concrete emulation scenarios against managed GenAI infrastructure, not slideware. The first is credential-based model invocation abuse, where the attacker disables invocation logging to stay dark and resells access, cited at up to $42,000 of charges in a day. The second is knowledge base poisoning: tooling enumerates managed knowledge bases and uploads corrupted files to the backing object store, degrading retrieval output. Both are mapped to ATT&CK and ATLAS, with access control and invocation observability as the defensive answer.

AI agent guardrails simplified - prompt injection, PII and more

An implementation walkthrough of the input guardrail layer with runnable notebooks for each control: PII detection and masking before text reaches the model, a classifier for prompt injection detection, content moderation, and zero-shot scope validation with structured output checks. It covers only the first of the three layers the presenter describes, so treat it as the input tier alone.

GenAI security 101

Prompt injection in LLMs

The explainer to hand a developer who has not internalized that this is structural and not a bug to patch. It builds from the unified text channel between system and user prompts, through direct and indirect variants with a worked attack on an email assistant, to six layered defenses escalating to the dual-LLM architectural pattern. Includes vulnerable and hardened Python for a web page summarization path, a prioritized control checklist, and instructions for red teaming your own application.

Defense frameworks

OWASP GenAI LLM Top 10 2026

The 2026 edition changed how the list is built: 75% expert consensus blended with 25% real incident data across 6,639 documented vulnerabilities, mapped to NIST, MITRE ATLAS, and CWE. Eight of the ten entries moved. Excessive Agency climbed as production incidents clustered around agentic systems, Unbounded Consumption rose four positions on availability and financial denial-of-service risk, and System Prompt Leakage was renamed Hidden Context Exposure. Read the movement, not the final ordering. It is the closest thing to incident telemetry this field publishes. For the agent-side companion list, see our walkthrough of the OWASP Agentic Skills Top 10.

GenAI vulnerability

Breaking local AI runtimes: 10 vulnerabilities in the engine behind your open-source models

The first serious memory safety audit of the C/C++ inference engine that Ollama, LM Studio, Jan, GPT4All, and hundreds of downstream projects are built on. Ten flaws sit at three trust boundaries: the Android JNI wrapper, the server HTTP lifecycle, and GGUF metadata parsing. They include two use-after-frees at CVSS 9.2 and 8.1, an integer overflow in batch initialization, and out-of-bounds reads in state restoration. Maintainer disclosure stalled, CVEs were allocated by a third party, and five of the ten were still unpatched at re-check, so the researchers published their own patches.

Training materials

AI SecureOps: attacking and defending AI applications and agents

Nine modules and 25 hands-on labs run as a CTF-style course at DEF CON 34, with the syllabus and per-module lab counts published. The modules cover adversarial LLM attacks including obfuscation and system prompt extraction, enterprise defenses with custom scanner rules and monitoring, red and blue teaming with vector database tracing and rainbow teaming, MCP trust boundaries, and multi-agent privilege escalation. A proficiency certificate requires 1,400 of 2,200 CTF points.

Patch the runtime

If you run models locally, check which build of the inference engine you are on. Five of the ten flaws were still open at re-check, several sit in the GGUF parser that consumes files people download from public model hubs, and the inference engine rarely appears on anyone’s asset inventory.

If you consume an API that returns encrypted reasoning blocks, put the reasoning trace result in front of whoever owns that integration. The control assumes those blocks are inert on the client. They are not, and nothing binds one to the session it came from.

The authorization bugs on this list are not exotic. A chat backend that trusted a client-supplied thread identifier turned an ordinary access control failure into durable context poisoning, because the thread it exposed gets replayed as model context on every later turn. Make every path into the model re-check ownership before it trusts an identifier the client handed it, and treat stored conversation state as a write target, not only a read one.

Every item here reaches the model through a step nobody inspects: the runtime underneath it, the encrypted block the client hands back, the document the retriever returns, the thread the backend replays. The chain is the breach, and only by observing and reconstructing the whole chain of agentic actions can you stop the damage before it lands.