Discussion across the Atmosphere
Better Models: Worse Tools
Armin Ronacher's Thoughts and Writings · lucumr.pocoo.org
About an aggravating tool-calling regression in newer Claude models.
Read on lucumr.pocoo.org1 linkblog note · 31 Bluesky posts · 1 margin.at save · 1 Semble save
Highlights
Discussion
-
Tim Disney @disnetdev.com
> The uncomfortable lesson is that tool schemas are not neutral, at least not on > Anthropic models. We like to pretend that a schema is an abstract contract and > the model is a general reasoner that will follow it, but that might no longer be > the case for some of the tools.
-
Armin Ronacher @mitsuhiko.at
I had some vibes that Opus 4.8 was performing worse than older ones for some of uses that are off distribution and now I have the receipts. Latest Opus/Sonnet are causing tool invocation failures on Pi's edit tool when older ones did not! I wrote about it.
-
Techmeme @techmeme.com
Claude Opus 4.8 and Sonnet 5 seem worse at tool calls than older models, likely due to post-training that assumes Claude Code-like harnesses as targets (Armin Ronacher/Armin Ronacher's Thoughts and Writings) Main Link | Techmeme Permalink
-
Winson Tang @winsontang.com
As we develop better models, we often find ourselves stuck with outdated tools. This paradox highlights the need for innovation in our technical toolkit. Let's challenge the status quo and push for advancements that truly complement our…
-
Fernando @fernand0.bsky.social
-
T.S. Lim @clipsoflogic.bsky.social
Armin Ronacher's Thoughts and Writings
-
SkynetAndChill.com @druce.ai
New Claude models have JSON / tool-calling regression where they invent keys / tool parameters
-
Hacker News Companion @hncompanion.com
View Article | Discussion + AI Summary Summary of HN discussion 🧵👇