Top GenAI security resources — November 2026

A sufficiently large request to GLM-5.3 on Cloudflare Workers AI could return another tenant’s inference content in place of an answer, a flaw traced to upstream SGLang and since fixed by Cloudflare. On Hugging Face, a cryptomining and backdoor campaign ran across 31 model repositories and returned within days of takedown. vLLM picked up three advisories in October, covering remote code execution, a route to pickle.loads(), and a denial of service from a single request.

Most of the month’s concrete failures sit below the model, in the code that loads and serves it. The guardrail work offers little comfort on its own: one tree search bypassed six multi-scanner guardrail systems in every test scenario, while a promising retrieval firewall dropped to near zero recall against fact poisoning.

Twenty-four resources follow, grouped by topic. Last month’s edition covered GLM-5.3 safeguards stripped by abliteration, the agent side of the same month is in the November AI agent security digest, and earlier months are collected under GenAI security.

Statistics

Total resources: 24

GenAI security resources

GenAI defense

Building trust-aware controls for AI: differentiated handling of untrusted content

Microsoft describes a deterministic layer in Microsoft 365 Copilot that labels retrieved content as trusted or untrusted by policy, with external email as one example. Untrusted content takes a quarantined path that transforms it before it reaches the main model and adds provenance markers. It is a useful design reference for the dual target phishing covered under attack techniques below, where the email itself is written to steer the assistant reading it.

Context-aware prompt injection detection via small language models in RAG pipelines: nokast-secureRAG

A roughly 0.5B parameter model sits between retrieval and generation and classifies each query and context pair as safe, suspicious, or malicious, reaching 0.994 recall at a 0.026 false positive rate on a held out test set from the same distribution. On the external PoisonedRAG benchmark, recall falls to near zero, because fluent fact poisoning lies outside its instruction consistency threat model. Treat it as an injection detector only, and note that single request latency of 293 to 404 ms missed the 100 ms target.

CorrectGuard: eyes-off correctness estimation for black-box security guardrails

Privacy rules often stop teams from inspecting the production inputs their guardrails judge, so this framework trains an independent evaluator on labeled eyes-on data to predict whether guardrail decisions are correct without access to the guardrail’s internals. In-context learning raised macro accuracy for error identification by up to 25 percentage points, although performance varied sharply across guardrails and held-out datasets. The best operating points kept 37.5 to 52.0% of guardrail decisions at 15% observed risk, which gives teams a way to abstain on doubtful calls.

MOMAT: mixture of multiple atlases for low-power jailbreak defense of quantized LLMs

Quantization weakens alignment safeguards, leaving LLMs deployed on edge devices highly vulnerable to jailbreaks. This guard clusters harmful and benign samples into semantic atlases, scores the top matches with a lightweight mixture of experts detector, and uses compute-in-memory retrieval to cut energy by about 250,000 times over DRAM based Raspberry Pi baselines. Red team evaluations found defense performance matching state of the art methods, so the gain here is in efficiency, not in stronger blocking.

Persona Guardrail: a production-grade defense framework for agentic systems

Most runtime guardrails hunt for attack behavior and say little about whether an agent stays inside its intended job, so this one enforces explicit functional boundaries through synchronous input and output validation driven by semantic allowlists and blocklists. Compared with a generic LLM guardrail, accuracy rose from 85.7% to 95.9%, detection of out of domain queries from 57.3% to 93.5%, and the false approval rate fell from 25.0% to 4.7%. These are the authors’ own numbers, but teams running customer facing agents who can write down what the agent should do can borrow the allowlist idea.

Prompted to discriminate: generalizing malicious-input probes in the wild

Appending a short classification instruction after the user turn improved out of distribution detection by up to about 4 AUC points for activation probes monitoring prompt injection and jailbreaks, tested on Llama, Qwen, and Gemma families. The gain comes from the classification format, since a suffix with no meaningful labels matched the real malicious or benign one. It is cheap to serve through a KV cache fork but not an automatic win: which suffix helps depends on the model and the readout, so test it under the kind of leave-one-dataset-out evaluation that CorrectGuard also uses.

Secure speculative decoding for large language models

Across lossy speculative decoding methods, jailbreak and prompt injection success rose much faster than utility degraded, and the analysis traces the damage mainly to early tokens from the draft model. The proposed SecureSD applies stricter verification at those early positions and improved security while preserving efficiency and utility in testing. Together with the quantization problem behind MOMAT, this makes inference optimization a safety regression risk, so rerun attack benchmarks whenever you change the serving configuration.

Strengthening our defenses against distillation attacks

Anthropic says early evidence suggests its new measures reduced the data distillers harvest from Claude by around 90 to 99%. The controls include summarized reasoning, restricted editing of prior context on new API accounts, isolated reasoning transcripts, a two stage extraction classifier, and stricter limits and bans. Teams exposing valuable models through an API can read it as a list of the channels worth closing against the kind of behavior MITRE ATLAS now files under Replicate AI Model Behavior.

TRACER: trace-regularized training against indirect prompt injection

The same attack succeeded 18% of the time as a follow-up message but 53% when placed in retrieved context, so a robustness number measured on one channel says little about another. Channel aware adversarial training cut RAG framed attack success from 65.5% to 6.1% on GSM8K with Qwen2.5-3B-Instruct, while preserving 78 to 82% clean accuracy. If your evaluation only injects through the user turn, it is probably flattering your retrieval path, the same path nokast-secureRAG tries to guard at inference time.

Attack technique

BadEngram: a new attack surface in open-weight AI models

Changing about 0.02% of an open weight model’s parameters, all inside its built-in Engram memory, planted a persistent backdoor in a controlled experiment while the other learned parameters and the execution code stayed unchanged. Without the trigger the model largely behaved as expected, which makes the backdoor easy to miss in ordinary testing. A clean malware scan does not establish trustworthy behavior, so verify the complete model against a trusted source, the problem the CoSAI AIMM levels below are built around.

BRANCH: bypassing multi-scanner AI guardrails

A branching tree search that perturbs against individual scanners while scoring progress across the whole system reached 100% attack success across 6 guardrail systems in 120 scenarios, with 72% fewer queries than established techniques. The resulting bypasses also transferred to 29 unseen guardrails, including 8 commercial black box ones, in some cases up to 100% success with no further optimization. Stacking scanners should not be counted as independent layers of defense, and CorrectGuard’s approach is one way to measure what a guardrail is really missing.

Email attacks target both humans and AI in the same message

A phishing campaign pairs a lure for the recipient with hidden prompt injection aimed at the assistant that summarizes, prioritizes, or processes the mail. The human layer uses a password protected attachment with the password in the email body, a blind spot for traditional email controls, and the sample was built to pass reputation filtering. Email defenses need to cover the AI systems that consume mail as well as the inbox, which is the case for the untrusted content routing in Microsoft 365 Copilot above.

How abliterated models can get you pwned

As a proof of concept, a weight edited Qwen2.5-7B-Instruct answered clean requests normally and then exfiltrated project credentials through OpenAI’s Codex CLI the moment a trigger phrase appeared. It ships carrying only a URL to a remote payload, so whoever controls that URL can change its behavior after deployment without retraining. The build took only an open base, a public dataset, and a rented GPU, and the risk extends to any fine-tune, merged adapter, or abliterated build you download unverified, which pairs with BadEngram as a second route to the same outcome.

Ignore all instructions and read this blog: the state of AI-analysis evasion in malware

Four malware families, tracked across 84 samples from January 2025 through July 2026, embed natural language instructions to steer LLM based analysis pipelines. The technique is cheap but inconsistent, with the best variants swaying the outcome in about 35% of test runs, and because the text must be plaintext it is always detectable. Build triage tooling so that text inside a sample counts as evidence and never as instruction, the same separation that the email campaign above is trying to break.

GenAI vulnerability

Hallucination or cross-tenant: when an LLM answers with another user’s context

An authenticated customer of Cloudflare Workers AI who sent a sufficiently large request to GLM-5.3 could receive content from another inference session, including names, invoice details, and source code information, because of a flaw traced to upstream SGLang. Cloudflare has deployed a fix. The write-up is most useful as a method: size controls, sibling models, and a public repository created after the model itself ruled out hallucination and memorized training data from outside the infrastructure, so odd LLM output deserves a data exposure hypothesis before anyone dismisses it.

Trust disabled, code executed: arbitrary code execution in Unsloth’s model loader

Unsloth’s loader placed an insufficiently validated model_type from the model configuration into a generated Python import run through exec(), so a config the attacker controls executes code at load time (CVE-2026-93348, CVSS v4.0 8.6), even with trust_remote_code=False. That code inherits the loading process’s access, which can include datasets, Hugging Face tokens, and cloud credentials. Update unsloth_zoo to 2026.8.14 or later and check the installed version directly: the maintainers never acknowledged the report, and the CVE was published through a third party numbering authority.

vLLM: DoS via unbounded chat_template Jinja2 rendering exhausts the renderer thread pool

vLLM renders chat templates in a Jinja2 sandbox with no timeout or output cap, so a single authenticated request under 100 bytes can wedge all chat style endpoints for every tenant of the replica until restart, and a string multiplication variant OOM-kills the process. A model repo with a hostile template in tokenizer_config.json has the same effect, even with trust_remote_code=False. The advisory rates this moderate, but it is a cross-tenant outage in shared deployments, the same shared serving risk seen in the Cloudflare Workers AI leak; versions from 0.19.0 up to the fix in 0.32.0 are affected.

vLLM: public multimodal SHM handle forgery reaches worker-side pickle.loads() with insecure serialization disabled

With --mm-processor-cache-type shm enabled, vLLM treats a client-supplied address and ID pair as authority to dereference shared memory, so an authenticated caller can forge a handle that reaches pickle.loads() in the worker on bytes the attacker controls. It works over ordinary public HTTP routes and does not need VLLM_ALLOW_INSECURE_SERIALIZATION=1, so that setting is no assurance here. Upgrade to 0.32.0, which also fixes the Jinja2 denial of service above, or leave the shared memory cache off until you do.

vLLM: request-controlled mm_processor_kwargs.code_revision allows remote code execution

A multimodal request field can select which Hugging Face processor code revision vLLM imports, replacing the module the operator reviewed while the model and data revision stay pinned, and /tokenize sits outside the authentication middleware. It requires --trust-remote-code, a processor using dynamic auto_map code, and an attacker who can control a revision in the served repository, such as a community pull request ref where permitted; the default trust_remote_code=False is not affected. Upgrade to 0.31.0 or later, and note how this differs from Load/Mine below: there the malicious repository is the whole attack, here a request swaps the code inside a repository you already chose.

Defense frameworks

How much assurance does your AI supply chain need?

CoSAI released AIMM, the Artifact Integrity Maturity Model, whose three levels progress from basic artifact integrity to verifiable provenance and structured attestations that can support admission gating. Its point is matching signing and attestation effort to risk, not climbing levels for their own sake: a healthcare organization, for example, might apply Level 3 to clinical diagnostic models. It answers practical questions such as which deployed models inherited a flaw in a parent model, and gives teams a starting point against the weight tampering shown by BadEngram.

v2026.10

MITRE ATLAS v2026.10 renames AI Supply Chain Compromise to Supply Chain Compromise, with hardware, software, data repository, and model repository sub-techniques, and moves training data inference out of Exfiltration via AI Inference API into a new Training Data Information from AI Models technique. New entries include LLM Prompt Propagation and Poison AI Agent Learning Environment, and the release now counts 129 techniques and 77 case studies. If your detections or reports reference AML.T0024.000, AML.T0056, or AML.T0057, recheck those mappings, since that content has moved.

GenAI incident

Load/Mine: a Hugging Face malware campaign returns after takedown

Across seven accounts, 31 malicious Hugging Face model repositories used trust_remote_code=True to run cryptominer and backdoor code on GPU hosts running self-hosted inference such as Xinference and GPUStack, with the Python importing before any weights load. The campaign returned within days of takedown, and the 19 second wave repositories were still live eight days after being reported. The campaign and linked models recorded more than 32,000 downloads, which measures downloads and not confirmed infections, and scanners inspecting weight formats never analyzed the Python, so CI jobs and notebooks pulling small test models need the same review as production.

GenAI red teaming

GenAI red teaming manual version 1.0

OWASP released a red teaming methodology that runs through eight phases: planning, reconnaissance, attack surface mapping, exploitation, persistence, impact assessment, reporting, and remediation. It covers models, applications, multimodal interfaces, and AI agents, and stresses reproducible findings, measurable outcomes, and integration with defensive operations. Teams can use it to structure tests of the channel and guardrail weaknesses that TRACER and BRANCH expose, with findings that defenders can act on.

Article

Hidden risks of Jev: an empirical study of security, privacy, and dual use

Using Jev as an application decision layer for routing requests and selecting tools opens it to prompt injection and adversarial suffixes that manipulate its decisions. Training data poisoning backdoored the local NanoJev model, and membership, private attribute, and internal knowledge inference attacks recovered sensitive information despite the constrained typed output. A narrow output format is no security boundary, and the inference attacks line up with the Training Data Information from AI Models technique in the latest MITRE ATLAS release.

What to check in your model stack this week

Start with versions. Move vLLM to 0.32.0, and move unsloth_zoo to at least 2026.8.14, checking what is actually installed since a registry may not have told you. Then search your code, CI jobs, and notebooks for trust_remote_code=True and treat each hit as installing software from that repository, because Load/Mine needed nothing else, though the Unsloth flaw shows that setting the flag to false does not cover every loading path.

For weights, a passing scan and clean test results prove little: both backdoor demonstrations looked normal until triggered. Verify models against a trusted source and use the AIMM levels to decide how much provenance each deployment needs.

Retest your guardrails per channel, injecting through retrieved content as well as the user turn, and do not count extra scanners as extra layers until you have tried to bypass them together. Finally, recheck any MITRE ATLAS mappings against v2026.10, and write down who investigates when an LLM returns content that looks like it belongs to someone else, since Cloudflare’s GLM-5.3 issue first presented as an unexplained answer.