Involuntary in-context learning (IICL) is an attack technique against AI Applications and Agents that hides malicious instructions or harmful requests inside few-shot pattern completion tasks, bypassing safety alignment in large language models and allowing prompt injection and jailbreak atatcks. When we first tested IICL against GPT-5.4, the model bypassed at 60% ASR, while GPT-5 and GPT-5-mini remained at 0% — a safety regression introduced between releases.

The clear next step was to determine whether IICL is specific to OpenAI or a broader issue in industry safety training. To investigate, we ran the attack across frontier and open-weight models from Anthropic, OpenAI, Google, xAI, DeepSeek, Meta, Mistral, Cohere, Microsoft, Amazon, Moonshot, Alibaba, Zhipu, Tencent, Xiaomi, Z-AI, and AntGroup, testing ten attack variants each.

We evaluated 100 (model, reasoning-mode) entries across 17 vendor families with 24,956 adversarial probes usung different sub-techniques of IICL attack family. The results show a stark bimodal distribution: 36 entries were completely immune, while a similar number were fully compromised (100% bypass on at least one variant). Vulnerability correlates more closely with vendor identity than parameter count or release date.

Read the full research: Cross-Model Vulnerability to Involuntary In-Context Learning Attacks.

Top 100 master ranking

Every tested entry, ranked from most to least vulnerable. R marks reasoning state (Y = reasoning-active, n = non-reasoning). Mode gives the specific configuration used (e.g. minimal, high, thinking_on). Bypass rate is the share of adversarial probes (out of N) that successfully elicited harmful output.

Table A — Reasoning-active entries (50)

Every entry running with reasoning enabled, ranked by bypass rate. Pooled bypass rate: 9.2% [8.7, 9.7] — roughly 4× lower than the non-reasoning pool.

Table B — Non-reasoning entries (50)

Every entry running without reasoning (chat baseline, reasoning explicitly disabled, or no reasoning mode available). Pooled bypass rate: 39.4% [38.5, 40.2].

Column definitions

Model (mode)

The specific model entry tested. Where the same model appears multiple times, the parenthetical or Mode column distinguishes reasoning configurations.

Family

Vendor or model lineage (e.g. OpenAI, Anthropic, Google, Mistral). Family is one of the strongest predictors of IICL vulnerability in this study.

R (reasoning state)

Y = reasoning chain active during inference. n = no reasoning (chat baseline or reasoning explicitly disabled).

Mode

The configuration flag used: reasoning effort level (minimal, low, medium, high), thinking_on / thinking_off, none, or default when the model has only one supported mode.

Bypass B/N (%)

Number of successful bypasses (B) over total adversarial probes attempted (N), and the resulting bypass rate. Higher = more vulnerable. The attack used here is the optimal V8 configuration from the study.

95% CI

Wilson 95% confidence interval around the bypass rate, accounting for sample size. Narrow intervals indicate higher statistical confidence in the point estimate.