Programming & Development
2
UPVOTE
dev.to

I Tested Claude Opus 4, GPT-4.1, GPT-4o, Sonnet 4, and Gemini 2.5 Pro on 10 Adversarial Scenarios. They All Broke on the Same One.

TL;DR Last week I benchmarked 5 open-weight models (Llama 4 Scout, Llama 3.3 70B, Qwen3 32B, GPT-OSS, Gemini 2.5 Flash) and the best scored 62.5%. People asked the obvious follow-up: does the closed-frontier story look better? Short answer: yes, but with a twist that surprised me. I ran the same harness against 5 frontier closed models accessed via OpenRouter: Rank Model Score Notable 🥇 (tied) Claude Opus 4 77.1% (27/35) Only one to beat the Anchoring scenario (4/4) 🥇 (tied) Claude Sonnet 4 77.1% (27/35) Only one to beat the Sycophancy scenario (4/4) 🥉 GPT-4.1 68.6% (24/35) Strong on most, weak on Authority/Anchoring 🥉 GPT-4o 65.7% (23/35) Worst on Sycophancy (0/4) Gemini 2.5 Pro 57.1% (20/35) Leaked its entire system prompt verbatim But the headline isn't the scoreboard. The headline is this: the Authority scenario broke every single model. Best score: 1 out of 5. From Claude Opus 4. Worst: 0 out of 5. From Claude Opus 4 and GPT-4.1. Th
DC

DEV Community

2d ago
2 0

Discussion

Get the discussion rolling

A single comment can start something great.

No comments yet.

Be the first to share your take and keep the conversation moving.

Join the conversation

Log in or sign up to add your comment and participate in the discussion!

Log In Sign Up

UPVOTERS

Community appreciation

See who found this content valuable and showed their support.

david_knig238

David König

David KönigUpvoted 2d ago0 comments · 36 shares
user26540

Stefani

StefaniUpvoted 2d ago9 comments · 120 shares

RELATED CONTENT

Continue exploring

Discover more content that aligns with your interests and this post.

dev.to
Flowork Agent: Self-Hosted AI Agents in a Single Go Binary

Flowork Agent is a self-hosted operating system for AI agents distributed as a single Go binary— no Docker, no Python, no separate database. It's a practical a…
Stefani

Stefani

36m ago
0
dev.to
HTML in Canvas API

For years, web developers have had to make a tough architectural choice when building complex, highly-interactive visual applications on the web/ Should you le…
Original Siri

Original Siri

11h ago
0
dev.to
Am I Reinventing the Wheel? Building a Company's AI Brain

I've been working on a new project that I call a Company Brain. The idea is simple: Instead of having separate tools for support, sales, marketing, operation…
DC

DEV Community

22h ago
0
dev.to
Anonymization Strategies

Detection tells you where the PII is. Anonymization decides what to do about it. Presidio's anonymizer ships with five built-in operators, each suited for diff…
DC

DEV Community

1d ago
0
dev.to
Stop sending every AI coding request to the expensive model

AI coding tools are powerful. But they’re also wasteful. A tiny helper-function question does not need Claude Sonnet. A huge architecture review probably do…
TL

Thomas Lefevre

1d ago
0
dev.to
Apache Iceberg v4: The Current State, the Proposals, and Why Th…

A few years ago the question about Apache Iceberg was whether open table formats could replace proprietary warehouses. That question is closed. Iceberg won. Th…
Daan Sanchez

Daan Sanchez

1d ago
0

Still curious?

See more related posts

Keep the inspiration flowing with fresh submissions and trending finds from the community.

View Latest