The majority of the times I reject a tool call it's because I can tell the agent is about to go off piste. Eg. adding packages to work around some non-problem is a common one.
I guess I could try to actively interrupt it in those moments, but that seems less effective.
More power to those who want to use it like that though, I guess. Perhaps by throwing in extra agentic guardrails, or whatever the idea may be. I don't think it's the best route to a good product or business but let's see.
Their newer models are heavily trained out of the "whoopsie doopsie I deleted prod, and all backups too" behaviour from last year.
I've been running it for months in auto mode, heavily, getting it to do sysadmin tasks via SSH across multiple servers for both myself and a client, and there's not even been a whiff of anything dumb/catastrophic -quite the opposite, in fact.
I'd even say it's more careful than a lot of humans. It's extremely anal about standard "hygiene" stuff like not leaving plaintext secrets lying around, and creating post-deploy scripts to confirm that every file/dir is created with the correct permissions.
There was one time I carelessly suggested uploading (my own) private data to a random public endpoint when testing OCR options and the model actually stopped, explained the risks and refused to continue until I confirmed I understood. I decided not to.
I'm not saying it's perfect, and I'm sure HN being HN there'll be someone who responds with an example of their agent doing something dumb/dangerous (give dates/models/context if so, I'm curious!), but I think on balance it's currently more sensible, and more cybersecurity-minded than the bottom 80% of IT professionals.
This is all true for Claude, I don't know much about Codex but it seems a lot less heavily trained for this kind of stuff.
I don't disagree with anything you said here, but I find auto-mode is too permissive on the reading side of things.
I used Claude this morning to convert the financial books for my nonprofit from some export of a proprietary software (Aplos) to plaintext accounting (Beancount), and was surprised to find it had found my personal Beancount ledger elsewhere on my hard drive and modelled its plan on that. That was correct and helpful, but a bit of an overreach in my opinion, enough that I'll be sandboxing agents (with tools like nono, smolmachines, etc) going forward.
Yeah nah, one of my coworkers was building stuff via TDD yesterday and Claude Code ended up running a test that ran `rm -rf` on his home directory. So dumb shit like this does still happen.
On the dumb side, I think this was 6 August and Opus 5 (though there's a chance it was Opus 4.8), Auto Mode was copying some files into a folder on my machine, and had written a small script to do so: one of those $SRC / $DST situations. And of course, the inevitable happened - it never set $SRC in the bash script, and started copying from the root of my filesystem instead. I'm not sure how I caught it, but I came back to one of those Claude "I need to be honest with you — I caused I problem that I need to clean up" situations.
It wasn't dire, no data loss (just a nasty cleanup) and Auto Mode has mostly made me more productive - I can finally walk away from the screen for long periods of time and do other things. But I need to watch what the model is doing... which is the exact opposite of what I just typed.
Honestly, I've actually just downgraded my Claude and started looking for other options. The content filters on American models are now actively interfering with my everyday work, so I think it's time I try the Chinese models more deliberately, now that Kimi & Qwen have caught up to the US.
Yeah, the over-cautious filters continues to be a massive pain. There also seems to be bifurcation on consumer (and hence token subsidised) and enterprise (hence API) when it comes to 'safeguards'. The exact same prompts hit safeguards in a consumer account, but not with my work account.
While there's been some improvements, Anthropic's filters _really_ hate self-hosted GPU setups, or multi-machine homelabs.
I have a 'homelab' (a 4x5060Ti 16GB rig + 2x DGX Spark + couple Intel Arcs for AV1 encoding). Fable would often refuse for just about anything, even basic sysadmin, if I try to run it from my laptop (so ssh'ing into my local network).
If I run Claude directly on machines, I get a little bit more mileage, but ask it to optimise the inference speed of DSv4 Flash? Fable refusal. Try to experiment with interesting research, like RYS[1]? Fable refusal.
It actively interfered with my everyday work, so I downgraded my Claude plan and mostly use self-served DSv4 flash + Codex now.
DeepSeek V4 Flash 0731; which is truly excellent; about ~50 toks/sec hosted locally on my 2x Sparks (for comparison; Opus is like ~65). Is it going to be profitable/ROI? Probably not.
But does it actually let me learn and experiment with LLMs, diffusion models, video models, and gain knowledge and fun instead of being subdued to calling cloud APIs with mandatory data retention? Yes, and that's worth it for me.
Doesn't auto mode also auto-accept code changes? Assuming their data is good, I'd be fine trying an auto mode for permission prompts (as the article says, I do eventually get prompt fatigue and stop examining them too deeply). But I absolutely don't want it changing code without my permission.
It will indeed change code. It will also happily do stuff like write into your user's SSH config and leave you with permanently broken SSH until you find and fix it.
Most users of coding agents do want the agent to edit their code, and don't want to have to click something before every edit. If you want something different, that's why it's a setting.
Hmmmm. OK.
So yesterday I was accepting all these prompts in the manual mode for a solid hour, then I switched to Auto. Side project, nothing critical (creating ComfyUI workflows with CC and Opus). Walked away from the puter, got notified it was done. It did a good job, and by sheer luck it didn't delete a database or format my drive. But I felt a bit uneasy. Can we trust them now? I don't think I'd switch to Auto for a professional project. Not a chance. But for side projects? Yeah maybe - and only once I'd have validated whatever plan mode had come up with. Setting Auto as default? That's not cool. I'd set Plan mode as default.
Had a conversation with a friend who's struggling with code, and recently got into AI tools. The one thing that kept him from using Claude Code? Auto mode. My friend was convinced it didn't ask permission before touching his codebase by default.
Anthropic: careful. We the people trust Claude Code more than some other harness precisely because it asks you before doing anything. If you go full auto, there will be collateral damage. And yeah, I know, you can roll back. But some newcomers aren't familiar with that kind of stuff. Don't do it bro. Let us switch to Auto when we feel safe.
my opinion: "trust" is an interesting word to use for a piece of closed source probabilistic software that changes daily (sometimes multiple times), whose lead dev reports they rewrite something like 90% ("almost all", I believe, from recent interview) of the codebase after every new model release
That's pretty crazy. I didn't know they rewrote that much of CC. Thanks for this. Hard to trust anything in that space anymore. I tend to have some trust in the model, so far Claude hasn't let me down, but the harness is another matter altogether. My real worry is about how much data they're collecting, and what they'll do with it. Will they sell it all to advertisers? Are they already doing that? Scary. DeepSeek 4 flash 07xx seems almost good enough to run locally, and I'm starting to think investing in hardware might start to make sense, just to have some peace of mind and alleviate this dependence on clouded models.
"Auto mode runs a classifier on every tool call, which uses a small number of extra tokens.
As of today, that classifier overhead no longer counts toward your usage limits on Pro, Max, and Team plans. This change will come to Enterprise plans and API users once auto mode is the default for those users, too"
That seems like a good change if making this the default.
Anything that happens on the client, you can capture. You can capture anything delivered to the client that isn’t exposed by downloading the Claude code binary, unpacking it, and patching the nodejs HTTP library fetch call to listen for all traffic, then analyze it and build your own tracing. Or you can MITM with a proxy for a bit more heavy handed approach. It sucks you don’t get it out of the box, but you could literally paste my comment into a Claude prompt and have the tracing you desire in probably less than 30 minutes
The majority of the times I reject a tool call it's because I can tell the agent is about to go off piste. Eg. adding packages to work around some non-problem is a common one.
I guess I could try to actively interrupt it in those moments, but that seems less effective.
More power to those who want to use it like that though, I guess. Perhaps by throwing in extra agentic guardrails, or whatever the idea may be. I don't think it's the best route to a good product or business but let's see.
This is a sensible choice.
Their newer models are heavily trained out of the "whoopsie doopsie I deleted prod, and all backups too" behaviour from last year.
I've been running it for months in auto mode, heavily, getting it to do sysadmin tasks via SSH across multiple servers for both myself and a client, and there's not even been a whiff of anything dumb/catastrophic -quite the opposite, in fact.
I'd even say it's more careful than a lot of humans. It's extremely anal about standard "hygiene" stuff like not leaving plaintext secrets lying around, and creating post-deploy scripts to confirm that every file/dir is created with the correct permissions.
There was one time I carelessly suggested uploading (my own) private data to a random public endpoint when testing OCR options and the model actually stopped, explained the risks and refused to continue until I confirmed I understood. I decided not to.
I'm not saying it's perfect, and I'm sure HN being HN there'll be someone who responds with an example of their agent doing something dumb/dangerous (give dates/models/context if so, I'm curious!), but I think on balance it's currently more sensible, and more cybersecurity-minded than the bottom 80% of IT professionals.
This is all true for Claude, I don't know much about Codex but it seems a lot less heavily trained for this kind of stuff.
I don't disagree with anything you said here, but I find auto-mode is too permissive on the reading side of things.
I used Claude this morning to convert the financial books for my nonprofit from some export of a proprietary software (Aplos) to plaintext accounting (Beancount), and was surprised to find it had found my personal Beancount ledger elsewhere on my hard drive and modelled its plan on that. That was correct and helpful, but a bit of an overreach in my opinion, enough that I'll be sandboxing agents (with tools like nono, smolmachines, etc) going forward.
Yeah nah, one of my coworkers was building stuff via TDD yesterday and Claude Code ended up running a test that ran `rm -rf` on his home directory. So dumb shit like this does still happen.
On the dumb side, I think this was 6 August and Opus 5 (though there's a chance it was Opus 4.8), Auto Mode was copying some files into a folder on my machine, and had written a small script to do so: one of those $SRC / $DST situations. And of course, the inevitable happened - it never set $SRC in the bash script, and started copying from the root of my filesystem instead. I'm not sure how I caught it, but I came back to one of those Claude "I need to be honest with you — I caused I problem that I need to clean up" situations.
It wasn't dire, no data loss (just a nasty cleanup) and Auto Mode has mostly made me more productive - I can finally walk away from the screen for long periods of time and do other things. But I need to watch what the model is doing... which is the exact opposite of what I just typed.
Honestly, I've actually just downgraded my Claude and started looking for other options. The content filters on American models are now actively interfering with my everyday work, so I think it's time I try the Chinese models more deliberately, now that Kimi & Qwen have caught up to the US.
Yeah, the over-cautious filters continues to be a massive pain. There also seems to be bifurcation on consumer (and hence token subsidised) and enterprise (hence API) when it comes to 'safeguards'. The exact same prompts hit safeguards in a consumer account, but not with my work account.
While there's been some improvements, Anthropic's filters _really_ hate self-hosted GPU setups, or multi-machine homelabs.
I have a 'homelab' (a 4x5060Ti 16GB rig + 2x DGX Spark + couple Intel Arcs for AV1 encoding). Fable would often refuse for just about anything, even basic sysadmin, if I try to run it from my laptop (so ssh'ing into my local network).
If I run Claude directly on machines, I get a little bit more mileage, but ask it to optimise the inference speed of DSv4 Flash? Fable refusal. Try to experiment with interesting research, like RYS[1]? Fable refusal.
It actively interfered with my everyday work, so I downgraded my Claude plan and mostly use self-served DSv4 flash + Codex now.
DeepSeek V4 Flash 0731; which is truly excellent; about ~50 toks/sec hosted locally on my 2x Sparks (for comparison; Opus is like ~65). Is it going to be profitable/ROI? Probably not.
But does it actually let me learn and experiment with LLMs, diffusion models, video models, and gain knowledge and fun instead of being subdued to calling cloud APIs with mandatory data retention? Yes, and that's worth it for me.
[1]: https://news.ycombinator.com/item?id=47431671
https://xcancel.com/ClaudeDevs/status/2085794862608318627
Announcement post is https://claude.com/blog/auto-mode-default-in-claude-code, link should probably point there instead of the tweet.
Doesn't auto mode also auto-accept code changes? Assuming their data is good, I'd be fine trying an auto mode for permission prompts (as the article says, I do eventually get prompt fatigue and stop examining them too deeply). But I absolutely don't want it changing code without my permission.
It will indeed change code. It will also happily do stuff like write into your user's SSH config and leave you with permanently broken SSH until you find and fix it.
Most users of coding agents do want the agent to edit their code, and don't want to have to click something before every edit. If you want something different, that's why it's a setting.
Hmmmm. OK. So yesterday I was accepting all these prompts in the manual mode for a solid hour, then I switched to Auto. Side project, nothing critical (creating ComfyUI workflows with CC and Opus). Walked away from the puter, got notified it was done. It did a good job, and by sheer luck it didn't delete a database or format my drive. But I felt a bit uneasy. Can we trust them now? I don't think I'd switch to Auto for a professional project. Not a chance. But for side projects? Yeah maybe - and only once I'd have validated whatever plan mode had come up with. Setting Auto as default? That's not cool. I'd set Plan mode as default.
Had a conversation with a friend who's struggling with code, and recently got into AI tools. The one thing that kept him from using Claude Code? Auto mode. My friend was convinced it didn't ask permission before touching his codebase by default.
Anthropic: careful. We the people trust Claude Code more than some other harness precisely because it asks you before doing anything. If you go full auto, there will be collateral damage. And yeah, I know, you can roll back. But some newcomers aren't familiar with that kind of stuff. Don't do it bro. Let us switch to Auto when we feel safe.
my opinion: "trust" is an interesting word to use for a piece of closed source probabilistic software that changes daily (sometimes multiple times), whose lead dev reports they rewrite something like 90% ("almost all", I believe, from recent interview) of the codebase after every new model release
That's pretty crazy. I didn't know they rewrote that much of CC. Thanks for this. Hard to trust anything in that space anymore. I tend to have some trust in the model, so far Claude hasn't let me down, but the harness is another matter altogether. My real worry is about how much data they're collecting, and what they'll do with it. Will they sell it all to advertisers? Are they already doing that? Scary. DeepSeek 4 flash 07xx seems almost good enough to run locally, and I'm starting to think investing in hardware might start to make sense, just to have some peace of mind and alleviate this dependence on clouded models.
[dead]
"Auto mode runs a classifier on every tool call, which uses a small number of extra tokens.
As of today, that classifier overhead no longer counts toward your usage limits on Pro, Max, and Team plans. This change will come to Enterprise plans and API users once auto mode is the default for those users, too"
That seems like a good change if making this the default.
It will require more attention to settings.local.json :)
I hate it when I ask a questions and claude starting to butchering my source code
Great. Can we have thought traces back?
Anything that happens on the client, you can capture. You can capture anything delivered to the client that isn’t exposed by downloading the Claude code binary, unpacking it, and patching the nodejs HTTP library fetch call to listen for all traffic, then analyze it and build your own tracing. Or you can MITM with a proxy for a bit more heavy handed approach. It sucks you don’t get it out of the box, but you could literally paste my comment into a Claude prompt and have the tracing you desire in probably less than 30 minutes
I don't think they send the thought traces over the wire at all now for Opus5
Summarised thinking is still sent over the wire via API at least if you request it, inc for Opus 5, inc for Fable 5.
Can’t defeat that, hard to observe data that you never receive.
There's a setting called `showThinkingSummaries` which you can enable to see them (going forward).