Generative AI security in June 2026 was a story of jailbreaks that refuse to die and attack surfaces nobody budgeted for. New work argues aligned models keep failing because of identifiable refusal-escape directions, while attackers turned a model’s own chain-of-thought into a jailbreak channel and showed that simply lowering an image’s resolution can slip harmful content past multimodal safety. Below is the month’s essential GenAI security reading, sorted by category.
Total resources: 19
Category breakdown:
| Category | Count |
|---|---|
| Research | 7 |
| GenAI defense | 6 |
| GenAI attack | 5 |
| GenAI exploitation | 1 |
This Systematization of Knowledge organizes jailbreak attacks and defenses under a multi-dimensional Security Cube, then benchmarks 13 attacks against 5 defenses. It is a useful map of where the jailbreak field actually stands.
Adversa AI ran involuntary in-context learning (IICL) — malicious instructions hidden inside few-shot pattern-completion tasks — across frontier and open-weight models from more than ten providers. The result is a top-100 ranking of how well each model resists the bypass.
This paper identifies refusal-escape directions inside aligned models that flip a refusal into compliance, and proves a conditional safety-utility trade-off. It is a mechanistic account of why alignment keeps failing under pressure.
Testing four frontier multimodal models, this study shows jailbreak vulnerability shifts with both language and input modality. The same model can be markedly easier to break in one language or modality than another.
MultiBreak is a large, diverse multi-turn jailbreak benchmark that achieves substantially higher attack success than earlier datasets. It gives defenders a tougher yardstick for measuring multi-turn safety.
This ablation study introduces a unified threat model and reproduces membership inference, attribute inference, data extraction, and backdoor attacks under controlled conditions. Putting them on one footing makes the privacy attacks directly comparable.
A scan of roughly one million exposed AI services turned up 1,652 unauthenticated Ollama APIs serving live models to anyone. That open access enables response poisoning and downstream workflow tampering.
This method uses a temporary, removable jailbreaking adapter to soak up the safety-degrading gradients that harmful fine-tuning would otherwise bake in. Once training is done, the adapter comes off, leaving alignment intact.
GLiNER Guard is a compact encoder that performs safety classification and PII detection in a single forward pass. That makes it cheap enough to run always-on as a production guardrail in front of an LLM.
Rather than bolt on an external filter, this work re-activates a model’s own internal safety mechanisms through targeted embedding disruption. The reawakened safeguards then flag fragile jailbreak prompts the model would otherwise have answered.
This paper studies activation-level consistency training as a defense that holds up better against adaptive attacks on reasoning models. It targets both jailbreaks and injection within the same training objective.
This approach transfers safety alignment into low-resource languages through self-distillation, using only multilingual queries and no extra labeled data. It aims to close the gap where non-English prompts slip past alignment.
This work connects randomized smoothing to differential-privacy profiles to certify end-to-end robustness against backdoor attacks. The result is provable protection rather than the empirical kind that adaptive attackers tend to erode.
This jailbreak treats the model’s own reasoning as the attack surface, decomposing a harmful goal into innocuous reasoning fragments and recombining them at the end. An evolutionary loop adapts the decomposition until it slips past safety checks.
This membership-inference attack tells whether a document sits in a RAG corpus using natural-language entailment, in as few as five queries and with no surrogate model. It is a reminder that retrieval back-ends leak their contents under light probing.
A tri-class membership-inference attack on machine unlearning shows that “forgetting” can leak privacy about the data a model retained, not just the records it was asked to drop. Unlearning, in other words, can widen the privacy hole it was meant to close.
ChatGPhish hides instructions inside a web page so that ChatGPT’s page summary quietly serves attacker-controlled phishing content and links. The user thinks they are reading a neutral summary; they are reading the attacker’s lure.
New researach shows that tampering with a model’s tokenizer vocabulary changes its output at the decode step without touching a single weight. The manipulation survives finetuning, making it a stealthy supply chain vector.
This work shows that simply lowering image resolution can catalyze jailbreaking in visual-compression MLLMs — even when the rendered text stays perfectly legible. The degradation slips harmful content past safety alignment without hiding it from the model.
Treat alignment as a speed bump, not a wall. Refusal escape findings and the multiturn benchmark results say a determined prompt will get through, so combining different types of guardrails around model interactions is essential. However, the defense must not stop at the LLM layer. Scan your own infrastructure for exposed model endpoints before someone else does; unauthenticated Ollama and inference APIs are this month’s easiest win for an attacker. And if your product summarizes or browses the web, assume the page is hostile — ChatGPhish shows a summary can be turned into a phishing delivery channel.
Open-source methodology by OWASP, CoSAI, CSA, and NIST contributors enables enterprises to compare AI agents by security risk for the first time
