I was reflecting on this on Saturday in an unformed way, trying to trace the lineage of a decision made at work.
The code change itself doesn't specifically matter. But suffice to say, it was about an AI feature in one of our products.
The code was stamped by Claude driven by a prompt. The prompt was for a ticket generated with the Atlassian AI integration. Atlassian had digested docs made with AI. The docs came from strategy memos I'm 90% sure were written entirely by Claude.
The strategy was chosen by management at the urging of exec leadership. The execs now communicate mostly via AI written memos. I do not know how they make decisions, but they reference tech influencers, market conditions, customer expectations.
This gave me pause. Who had actually made the decision then? Arguably there has been several layers of human review, but the actual source of the decision was hard to pin down.
We were not building the feature because we wanted it. We were building it because we thought other people expected it.
Perhaps reflecting on the state of the market, I thought, could indicate who was actually in control.
Where do investor and customer expectations come from in 2026? It is very murky, at least in tech. There appears to be hype. Some hype comes from true believers, some comes from cynics. But both respond to market incentives that reward bigger and bigger claims.
Where does the market's "action" come from? What is the driver?
Investors do not really seem to understand what the tech is or its limitations. Some are passive operators. Others are just responding to the overall froth and speculation in the market - which becomes a runaway feedback cycle.
This left me lost.
Nobody in this ecosystem, I thought, is actually in control here.
Nobody is actually orienting work and action to real, concrete goals. It's all based on speculation and anxiety about the future.
So it is not only that nobody understands what the code does. It is that we cannot, or at least I cannot, explain the motivation. There doesn't seem to "be" any form of "intention" in this environment.
It has all been hollowed out, replaced either be inscrutable machines, or inscrutable incentives.
Ironically it rather resembles the kind of "misaligned" superintelligence we are supposed to be avoiding.
In many (most perhaps, but hard to know personally) companies, most people don't really understand the market they are in, the competition, their own products, the customers etc well enough to actually make good decisions. They also aren't likely to be around (or held responsible) for decisions as they play out over years. So what has historically happened is that people use proxies for good decisions and understanding - which are clever sounding documents and presentations.
This is a long winded way of saying "people made up clever/sensible sounding stuff". Now it's easier to do it with AI so the problem is worse. However, I'm not sure what you were looking for was ever really there - the "inscrutable machines" and "inscrutable incentives" were always quite inscrutable.
> most people don't really understand the market they are in, the competition, their own products, the customers etc well enough to actually make good decisions
"Product" studies this data and tells engineers what features they want. I am rarely instructed to A/B anything...excepting when it's defensive, to ensure the first rule of "don't interrupt the flow of money" is upheld, if at all. Most of the features are obvious improvements anyway (determined from plain conceptual planning, manual testing, and personal usage).
ofc I don't understand the customer or market or anything else. I'm paid with the expectation that I'm not going to share an opinion about it, especially since am never exposed to the raw data and inner circle decisioning except during a quarterly meeting...maybe. This is part of why AI is so successful. Humans in large organizations are specialized with little creative input and lots of mechanical process. Coding is largely a mechanical black box (turing machine) from the outside looking in. It works seamlessly because I don't need to know about the things product wants, so the AI doesn't know and everyone carries on faster than we were before.
Yes, people can be bad at their jobs. Yes, they can work in an industry and not know anything about the market but I refuse to believe that most people in most companies are just stumbling around faking it. I've never worked in a place like that. People like this exist but there are only a handful of people like this at every job.
The hyperbole feels like it's been contrived to fit into the classic AI counterpoint: But humans do this too!
It's more nuanced than that - I never said faking it, people usually believe what they write in those documents/ppts. And they do actually sound clever. Next time you see a decision of any magnitude, especially something which sounds "strategic" - trace the decision back as far as you can, consider the question from several angles, and really question it. You will see gaping holes, decisions made using "frameworks", anecdotes, hopes and dreams.
I can't argue that people don't always consider every possible angle and nth-order consequence to a decision before making it. AI doesn't do this either. Also, this seems different than your original position.
It still sounds like what you're saying is people are bad at decision making so AI isn't doing that much worse than people. That's very bizarre thinking because it opens the door to AI making decisions for us.
I'm saying something even worse - humans are bad at decision making, we usually don't even most of the context required to make good decisions so we use heuristics to make decisions, and proxies for "good decision making".
If the above appears to be bizarre thinking, take a couple extra laps around the block.
Now I'm not saying AI is better, in fact above I mention that it's worse - but don't hold onto some nostalgic idea that decisions were made by people with lots of context thinking rationally and incentivized properly.
It's not counterpoint, it's a fact. Can you say with your hand on your heart that the typical user experience was acceptable before AI arrived?
The reply was always "It's hard, and there are always going to be bugs."
But it's not just about bugs. It's about systems that are hard to use, poorly designed, and opaque. And sometimes the opacity is deliberate. There are dark patterns, outright lies about what happens to data, and more or less obvious grifts.
Was any of this truly great before AI arrived?
Is it an accident that you can't kill an MS365 subscription if you have more than X amount of GB on OneDrive, but the actual usage includes all your spam and email attachments, and it doesn't show up unless you know where to find it, and the location is very non-obvious, and so is the cleanup and deletion process?
Or that the eBay fee structure for international sales is utterly incomprehensible without automated help?
Or that if you select Subscribe and Save on Amazon your cancellation date is something like two weeks before the next delivery?
Or that if you sign up to Discord you need a mobile and an email for activation, but the site doesn't tell you this, and nor do tech support, who insist that a mobile is optional?
And software quality - services down, records lost, records stolen, photos and documents deleted - is a whole other layer on top of that.
AI is a moderately good solution to the first set of problems, because it can search and assemble information far more quickly than you can. So IME it's pretty damn good at tech support - not perfect, but better than DIY in many cases.
Software quality? We'll see what happens. If tech stacks collapse over the next couple of years we'll know AI was terrible thing.
I'm in the 'Too early to tell' camp. There's a fair chance they might. But if so it will be because of poor testing and design, not because of code review or lack of hand-rolled code. And it's not completely clear that quality levels wouldn't have dropped anyway.
AI is here and things don't seem like they are getting any better any faster than they were before. Some people get it wrong and some people get it right...just like before.
Also, you're completely missing my point. The author is talking about his company inadvertantly offloading decision-making to AI. The comment I was responding to was like "Well, it won't do that much worse than people". That type of thinking is a little dangerous to the future of our species.
> Can you say with your hand on your heart that the typical user experience was acceptable before AI arrived?
I remember using Windows XP (SP3) and Windows 7, Photoshop 7 and CS3, Office 2007, Blender, Winamp, Linuxmint and my computer was a joy to use (with HDD and slow ram)
The internet was mostly a repository of knowledge and communication. I was online maybe 30 minutes every few days. This was around 2010.
From my experience in big and medium sized companies or even startups I would say that this was already the state of things before AI. Most leaders and top management were following internal committees, that were following consultants recommendations, that were themselves recommending what others were doing and was hype and/or following Gartner like "studies". The motivation was equally dubious, and many developers already had problem with management and product direction, but many didn’t bother to question it. AI just makes it obvious.
I don't think the problem OP has is anything so metaphysical as the fact/value distinction. This is a business, after all. It has goals (e.g., making money) that the model surely can grasp. It sounds to me more like a breakdown of organizational control owing to a lack of transparency in the tools and a general ignorance among the management.
The parent comment is interesting, but ultimately I think in this case, the AI is actually revealing something about the true nature about their place of employment, a nature that has always been there versus some mutation caused by the prevalence of the AI itself.
A lot of money can be made purely algorithmically. Think market markers or other algorithmic trading. The ought vs is divide is quite narrow here. It's not a moral question, the "value" is in the money to be made. It's actually not a great example to invoke Hume's problem. Many companies essentially are chasing a similar spread, its just less obvious. Few people ever ask what "ought" to exist. If the power of AI makes more businesses operate more reactively and algorithmically, because of more data or processing power or w/e that really is probably in keeping with their alignment and goals. Because the ultimate ought for a company is we ought to be making more money. So, in many ways the ought is really not that interesting, the is is satisfactory provided the return on whatever their version of a spread is keeps improving.
The number of companies that actually "invent" useful things and thus ask even vaguely meaningful "ought" questions are extremely slim. The vast majority of employees are, at best, accessories to these questions, even in software where even before AI many of us were not doing very interesting work. There is a lot of essentially rebuilding your competitors same layers on top of common libraries and standards where the actual interesting work is done. Really not unlike asking AI to cook you up a boilerplate by leveraging the vast work of a fraction of SWEs who maintain OSS tools. It's the same pattern and the same sort of behavior, just now your "layering" is becoming automated to the point of irrelevance.
What the parent misses - the real promise of AI is paradoxically, that it will allow more people to ask actually interesting ought questions as AI owns more of the spreads. In the same way a human does not compete with an algorithmic trader, and at some level, really doesn't care. The more algorithmic your business becomes, the less any individual human "value judgement" matters. And really this is desirable, because again, most companies are not asking interesting value questions anyway. The end state of this you are missing is these companies are going to cease to exist. In the optimistic case this will free you up to ask more interesting value questions - like how do I value all my UBI enabled free time. In the less optimistic case your value judgments will be more dire - like who do I sacrifice given the Terminators are at the door and we only have x quantity of supplies left.
This is the other paradox. When questions of what ought to happen are of paramount importance, you are probably finding yourself in a very undesirable situation. It's easy to valorize the ought problem from a distance, it is much much harder to actually engage with it when it actually matters. In many ways, the relative luxuries of society and civilization are derived from taking such questions out of most of our hands. This is (perhaps surprisingly) true even as you climb the ladder of power:
I used to think that if there was reincarnation, I wanted to come back as the President or the Pope or as a .400 baseball hitter. But now I would like to come back as the bond market.
For me, at this company, which is struggling to develop a new value proposition, everyone is unfortunately using AI in a blatant way. It’s even worse when executives and management pressure us to move fast. Presentations, documents, and mockups all use AI, rinse and repeat, feeding it context, but somehow, nothing is moving. It’s just staying static.
The word I have for this is “commitment laundering”. People pass around AI artifacts that nobody has necessarily read or considered, and the invented assumptions and tagalong commitments just keep piling up. They look like they’ve got real provenance but nobody can say what is being done or why.
Right up there with responsibility/culpability/liability laundering. These are things being shrugged off and externalized.
And the complement is credit/provenance laundering. These are things being misappropriated.
The grift often does both of these with the same sleight of hand, and this is what gets accelerated with the new tools and cavalier culture around everything.
Professors in grad school are also simply replying with chatgpt answers to any and all questions, including reviewing papers before they're submitted. I've learned to simply cut the middle man and ask the AI myself, though i cannot afford the pro subscription.
Most people are trend followers and what trends they see decides what trends they follow. It sounds like the algorithms that chose which influencers to promote made the decision, which was already somewhat true before the current AI trend. This happened when there was a big push from deterministic feeds to algorithmically ranked feeds by facebook and twitter. I guess an argument could be made that the people who chose what content to consume feed the algorithm but really there is a heavy hand on the scales that tilts the algorithm in favor of various things.
Anecdotally, I know people in the industry who tell me their job is "so easy" now because all they need to do is be a meat proxy for claude and take home the paycheck.
Question from someone written in AI? Just answer it in AI and send it back. What was it actually about - who cares? Bug comes in? Post the jira link in claude and don't even bother prompting anything else. If something critically breaks - well, eh, we'll deal with it then. FIRE (early retirement), a prediction their layoff is inevitable, and investing aggressively so you can finally escape actually working are often invoked in the same breath. Everyone feels like they're just trying to punch the drywall and grab as much copper wire out of the walls as they can until they're finally let go and/or the whole company fails.
There's a great deal of nihilism and cynicism in the industry currently, and it feels like LLMs are just greatly enabling it. Where you would've done a halfassed job previously, you'd now do an unchecked AI job.
> Everyone feels like they're just trying to punch the drywall and grab as much copper wire out of the walls as they can until they're finally let go and/or the whole company fails
That’s very graphic.
The thing is, I already felt a bit like that before AI. It’s just money. This quarter’s. The rest doesn’t exist. Make flashy features faster and you’ll be promoted. Make things carefully so that they last and can be maintained easily, and you are be ignored.
I say this not as an engineer that tried to write maintainable software and now is butthurt, but as an (ex)manager who tried to promote such people. And now is butthurt. I found myself telling my engineers that if they wanted to get promotions they had to prioritize the shiny and skip the rest.
I hated it.
Now I am back to Engineering. I do use AI heavily at work because no one cares but if my output is “too slow” I will look bad. I at least try to raise concerns when I see them. The answer tends to be “yeah, we can’t afford to do that properly now”.
I use AI way more sparingly for my personal open source stuff. Because I care.
It probably is. Where I work it has been made clear, as in actually stated by leadership, that any process that does not include AI input is to be considered broken and needing to be fixed. It doesn't matter if it works, if it is 100% human made and maintained, it is broken and needs to be fixed by injecting AI in a critical place. You should be in a position that if you lost access to the AI tools, you are unable to proceed.
This is the step towards replacing people. You don't need highly paid people in that process, you just need someone who can speak a language the AI tools can transcribe. This is no different than moving from a codebase or process that is tightly coupled to a specific technology to a more generic process so that you can easily and quickly change the backend tech on a whim. The technology being removed is the people. The AI is important, you are not.
Well it's the incentive given. A bit like the original comment writes above. If your performance and impact are measured by charts, meeting outputs, some kind of abstract KPIs and other means that traditionally kind of worked because people had agency, you are incentivized to just slopmaxx it. Otherwise you will fall behind your colleagues who slopmaxx and have a better number on their abstract KPI.
If you try to do good work, you won't be able to keep the pace with the slopmaxxers, which will mean you get laid off earlier. You will also be swamped in slop review.
If you get called out on some issue or shitty implementation, you can just make Claude abstract it away behind more complexity to the point where people have a hard time doubting you because they don't have time to get into the details and verify things.
Grab as much as you can, invest in immovable property and other shit that has value after a stock crash and enjoy the ride
One of the more thoughtful observation's I've seen in this "space," one which offers an unsettling counterpoint to the claim in the original post that "AI Can't Drive Itself."
To your point, AI can drive itself; it's just that the fashion in which control occurs is distributed. Which is to say, the process (e.g. a feature being implemented, in some way, or at all) is not spontaneously occurring: it's emergent from the mesh of AI automation.
Niceties aside, it's pretty clear that humans are not in the loop in a meaningful way, much of the time. What emerges may hence not well be not aligned with business or technical needs. What it is aligned to may be impossible for we humans to discern.
Who controls the way a forest grows?
Ask a tree, get an answer, but don't forget, that's not the answer.
This resonates, but also, I'm not convinced anyone can truly understand the workings of any society, or market economy, or what have you.
Who's in control? Everyone is, to some extent. And no one is: when you're hungry for food, are "you" in control of that? You can consciously repress your impulses to go eat something, but your mind didn't create those impulses.
Human societies develop impulses and minds of their own, emerging (weakly) from the impulses and minds that comprise them, and they make decisions in mysterious ways.
Of course, it sure is nice when we can come up with a compelling story for the motivations behind something. Easier said than done…
I just today found a term to describe what is happening: gradual disempowerment. I think it's a likely path. Everybody, because of competition, needs to end up giving control to AI.
> This gave me pause. Who had actually made the decision then? Arguably there has been several layers of human review, but the actual source of the decision was hard to pin down.
Cynically, this might be the real reason managers and investors love AI. It diffuses responsibility. No one is accountable. "Oops the AI messed it up" is a convenient excuse for bad management.
You could look at it as a kind of evolutionary pressure, where the AI driven companies that have no tether to reality are eventually extinct. Companies that are able to get a handle on the direction they are being dragged may have a fighting chance of survival.
That is a great example, and kind of frightening how once the market decides to start pumping money into something our world morphs to meet the flow of cash.
AI is definitely impressive but are LLMs the true path to AGI? Is AGI the path to a better life/world? These tools are really impressive but it would be nice if they did something actually useful. Can a rogue OpenAI agent draw investor attention by curing a disease instead of hacking into the CIA?
Ultimately competitive pressure is what drive decisions.
> Ironically it rather resembles the kind of "misaligned" superintelligence we are supposed to be avoiding.
yes, and it is the exact same system that is producing the "misaligned" superintelligence. Funny how that works, and begs the question: exactly how are you supposed to avoid building "misaligned" superintelligence?
At some point deferring all decisions to AI will be the competitive thing to do, regardless if its aligned or not.
> The code was stamped by Claude driven by a prompt. The prompt was for a ticket generated with the Atlassian AI integration. Atlassian had digested docs made with AI. The docs came from strategy memos I'm 90% sure were written entirely by Claude
Was there never a developer in the loop? I hope my org won't give up control of their business to an AI like this soon, sounds like a nightmare to figure out what's going on.
It's like old fashioned Wordpress, edited by someone directly in production, on a shared host that probably is infested with who knows what, with little to no oversight except inscrutable emails, which in the old days were like a CEO of a small business emailing 'hey, the thing we spoke about on the phone, is it coming?' and no actual document trail.
Except for the enterprise customer and at enterprise prices.
> Nobody in this ecosystem, I thought, is actually in control here.
Profound, your comment provoked this thought: we know that AI in organizations is disruptive, but we still don’t understand some of the shifting patterns of power, communications, agency, etc.
If AI is not well-suited for traditional models of stratified management then maybe we need different models of bureaucracy?
Thank you for taking the time to write this. Unfortunately, there are still people on this forum who argue that a turtles-all-the-way-down approach will somehow work out in the end and that the best way to verify LLM work is to add another layer of AI agents, whereas the only way to stop it is to ensure that the entire thing is built on some human verified last line of defense. Unfortunately, the higher up you go the more insecure people get when you ask for their supporting sources. They'll try to hand wave it saying they researched it or got others to research it for them instead of owning up to deferring the decision to AI. You know whats even scarier? When a C-suite doesn't try to hide the fact that they had the AI make the decision. Then it's just time to leave, or alternatively time to turn your brain off and collect your pay check until the music stops.
>Perhaps reflecting on the state of the market, I thought, could indicate who was actually in control.
Where do investor and customer expectations come from in 2026?
This looks like ill-fated Gartner driven development on crack. Where does customer expectations come from in 2026? How about from your customers?
If you're doing enterprise software and talk to your users, you'll end up learning that the actual users of your software don't use 80% of your features. You might even learn they don't use your software at all, and that the software was mandated top-down from the C-suites or that the execs in charge forgot what purpose your software served but are too insecure to question it, unless and until there is a sudden pressure to cut costs.
Exactly the same issues we are facing right now. And, well why wouldn't it end up like this? The promises of going 'faster' while ignoring the organisational processes capability to handle it is - and continues to be - the greatest delusion of this new age.
But, even in the face of all of this, we are seeing leaders asking for more. At some point the culture will collapse in on itself, and no one will understand anything anymore.
this is a good summary of the problems of AI decision making, but its assuming corporate decision making before was any better.. its probably about the same
ie we have companies making decisions just as badly before AI as with AI..
You need human vision and leadership to move an org.. AI is just a tool at this point
This is why we are building Origin (originhq.com). Having a record of every prompt, tool call and model response enterprise wide allows you to answer these kinds of questions.
When you write something you constantly remodel your understanding through refactors and rewrites until you internalize it. By internalizing it you gain the capacity to reason about it (during critical downtime) and communicate it. An entire team that can communicate can solve problems together, from one guy's vision to products white boarding to engineering's infrastructure to UX and UI's artistry.
It boggles me we completely forgot that the world operated like this just 4 years ago
I am simultaneously deep down in the agentic treadmill and also supremely frustrated by this.
And I don't think it was ever necessary to go to the point where people just gave up authorship. These were choices made by adopting the "I'll do everything for you" agentic "harness" model that shipped with Claude Code but it was never inevitable.
e.g. we completely dropped fill-in-the-middle completion OG CoPilot auto completion model. That combined AI authorship with a human always in the mix and I actually really enjoyed it. It's just that the models involved were pretty stupid. We totally could have had IDE / shell / tooling integration that kept people in the driver's seat while automating parts of the drudgery away. Instead what we got was a simple chat loop with "oh, whatever, you go do it" being the ultimate result. Cuz, you'll totally review everything after and understand it, right?
The things should end by quizzing you on what was just made and if you don't pass, just throw it away. That'd be funny to watch.
Long distance runner here, feels amazing from start to finish. Ok, amazing maybe only during runners high, but definitely good all the way through. Running sucks until it doesn't.
Imagine if you will two product proposals coming across Dario's desk.
One is:
1) Complex IDE integration where our AI model is called to assist users in making decisions, assisting them to do the things they already intended to do. It appears more as a function of the IDE rather than something separate. It shows off the intelligence of the model but doesn't show off any autonomy.
2) A separate tool that can be branded Anthropic. It takes over completely. It advances the narrative that we've already been spreading that AI is going to make some human jobs obsolete. It looks to employers (the actual people who spend money) like it replaces an expensive developer even if at first it requires one. It has minimal to no integration point.
Which do you think they'll approve. I think it has less to do with the user motivation and more with the producer's.
I feel this in my bones. My team is one of the most communicative in our org and have garnered a widely positive reputation I think largely as a result of our coming together every day to solve problems. Trying to get information out of the less communicative groups is like pulling teeth, i wonder how they get anything done at all
Assuming we don't speak about some vibed "one-shot-deploy" situation (for which there were options in the old world as well, with the same bad results), in a well disciplined team, why would AI make this worse?
You can still write anything by doing it either small steps, or at once followed by a lot of refactors while skimming over code and asking tons of questions / making refinements via prompts, guidelines, test guardrails. A team can still reason and whitepaper about the same things. Devs who were previously shy to ask some specific details (to not seem dumb) can now confidently survey big codebases and get insights in whatever style they can swallow.
The bigger problem I see is that all this requires a lot of communication, and most importantly writing skills, something that disappears in thin air in the last decades.
My experience is the AI has somehow thrown discipline and accountability out the door.
Folks break everything all the time and looking at their PRs and messages are clearly just having the AI respond to everything for them and actually have no idea what they're doing.
And then other people doing the same thing approve their work (presumably not reading or understanding any of it) because then the other person will do the same for theirs.
The way I think about it is that understanding is formed in a top-down fashion now, instead of bottom-up. You can look at a feature developed by AI from the outside, and keep peeling the layers and examining them (or having AI explain them to you) until you learn how it works. It's like reading a textbook. It is different than writing the code yourself, or practicing the topic of the textbook yourself. But if you invest the time, it can be just as effective.
The issue of course is that if you do invest the time, then you're no longer saving time by using AI. You're just spending it reading and trying to understand something you didn't write. And that can be unpleasant in its own way.
My hot take is that for parts of a system that can be considered its core, forming a deep understanding is almost always important, and so is knowing how the different business domains integrate and where the connection points are. For many others, a high level understanding is sufficient. The difference is that now, with AI, you can make that choice. Before, you had to write everything yourself, and for any sufficiently complex and long-lived system it became impossible to hold all of it in your head.
I disagree that reading a textbook is just as effective as solving problems yourself (this is what i read your statement to mean). It seems fairly obvious from university experience that doing the homework yourself gets you better test grades in the end. You can of course have the AI write the homework but you can't then just watch the AI solve it lest it be drastically muted in effect. Trying things that fail, working hard towards a problem, banging your head against something impossible, these are all worthwhile endeavors. Of course reading the textbook is useful too, very useful! But the textbook does not teach you experience.
> Trying things that fail, working hard towards a problem, banging your head against something impossible, these are all worthwhile endeavors.
I believe that this is the most important thing... and something that people are afraid of. I've got dozens of personal projects and more than a few branches in my employers repo of things that didn't work... things that I tried, figured out it wasn't going anywhere and went to try some other approach.
I suspect that there's a bit of sunk cost fallacy going on elsewhere. "If I don't know how to do it, I'm not even going to try" and "I got this far, I'll keep doing it despite it being wrong."
As a programmer, I am often disappointed at the lack of curiosity in the language and how things could be done. My example would be people writing Java code as if it was still Java 7 - no streams, no Optionals... The fear of having to go back and do it again if using something new doesn't work they'll be in a worse position than if they did it the old way and not realizing that learning what doesn't work or seeing how things that didn't work in this situation may be the right thing for some other future problem.
Sure, but what level of understanding do you actually need? Only you can be the judge of that obviously, but I posit that for the vast, perhaps even the overwhelming majority of any system's components, a high level understanding is sufficient. In fact that's how it already works: it is rarely the case that one person knows the team's entire codebase inside out. Instead different people specialize in different areas. With AI it's the same thing: you can achieve the level of understanding you want with the most important parts (either by having AI explain it to you, or architecting it and writing the code yourself), and delegate everything else to AI.
There are still some places that operate that way, as outlined in this Jane Street talk https://www.youtube.com/watch?v=zR9PpXWsKFQ (Production Engineering When Trading Billions of Dollars a Day).
> capacity to reason about it (during critical downtime) and communicate it
That's one of the things that's put into limelight in that talk.
I personally think that most of "Web Scale" software is inconsequential (inconsequential for its creators, not for users) - as there are no consequences for bugs and outages at all. A data breach -> slap on a wrist. Reputational damage because on an outage/data loss amidst general public? - almost impossible (clownstrike, anyone?)
The funny thing is that web scale software that's consequential is often in an ethically grey zone - but at least you won't be surrounded by colleagues who don't give a shit.
> When you write something you constantly remodel your understanding through refactors and rewrites until you internalize it.
Unfortunately I learned that not everybody thinks this way. Some orgs do imperfectly fine without good engineering discipline, and that has been the case before AI... AI has only made it easier to give the appearance of good engineering, which is exactly the pre-AI goal of many orgs
> It boggles me we completely forgot that the world operated like this just 4 years ago
I'd think that's what they call paradigm shift, and this probably repeated across generations from the introduction of the printing press, PC, the wheel, the internet to stochastic parrots that reduced what we still stubbornly insist require our special neurons to mere statistical modelling that can be aggressively scaled.
As part of a reorg, my team received a new system to maintain. I sat down with a great engineer and together we refactored that shit to get into something more maintainable. I became the go to guy for that system because our redactor led to such a deep understanding of the codebase.
Now in the age of AI... we adopted another system from a team and when we asked for knowledge sharing to prep for the handover the answer we got was "do we still do that? Just ask Claude".
It's now been a few months. We've shipped features in this new system. I still have no idea how it works. Okay I kind of know, but only at a very shallow level.
I am finding this, professionally, not so dramatic - but mostly because the industries in which I've consistently shipped code at scale involve review as a first principle, as in no un-tested, un-reviewed code gets shipped, because: safety critical/realtime requirements and certification specs, say so.
And having AI code to review is no different than any other code that ever was to review, so the review tooling is - as it necessitates - also benefited by lugubrious application of .. more AI. But: all AI is human reviewed.
So it's not a big impact. We just don't ship code that isn't 100% human reviewed, If that's insurmountable: you're doing it wrong. Use AI to make code readable again.
And then, also, put AI back in its box. Don't give devs 100% full-time API access to subscriptions: give them actual hardware to use, to go 100% local.
Local AI is, thus, the best AI, folks. Don't use more than you can run locally, is a great way to keep AI code properly maintainable.
The industry will prove this, itself, sooner or later: If you can't put your AI in its box for safe-keeping, you're doing it wrong, anyway... and should've already learned this practice as a habit, decades ago, vis a vis future-proof tooling... (See also: not logging everything you do with an AI? Big fail.)
Sure, the absolutely intoxicating addiction of Big Metal AI™ is going to put a lot of consumers in a deep, deep pit of Neo-Illiteracy - however: 'good' AI code is actually just good code.
> I am finding this, professionally, not so dramatic - but mostly because the industries in which I've consistently shipped code at scale involve review as a first principle, as in no un-tested, un-reviewed code gets shipped,
> And having AI code to review is no different than any other code that ever was to review
If your company is sticking to "everything needs (human) review" then you wont run into most of this. This issue is that a lot of companies are using AI as an excuse to remove that review process (either partially or entirely).
Yes, that is very real to me, the AI "revolution" is more of a battle between traditional devs trying to replace their managers with AI, and professional managers trying to get rid of the devs the same way. There is a wide degree and scope of knicker-twister'ing in these trenches.
Everyone can write code these days. Trouble is, all code has to be SIL-4 code now, because, human, you will never know if your compiler trusts your AI until you trust your compiler. This rule will be true for decades into the future, I'm willing to wager...
But, ultimately, software has to follow certain rules, or it just doesn't work. Proper workflows - involving review - are needed. Because security is pretty much over, otherwise.
Folks are finding it easier to make their own software now, too - rather than use others. I predict an end to the app stores - or at least, the primary interface is going to end up being "describe the app you want to use today" instead of picking words from a list ..
> And then, also, put AI back in its box. Don't give devs 100% full-time API access to subscriptions: give them actual hardware to use, to go 100% local. Local AI is, thus, the best AI, folks. Don't use more than you can run locally, is a great way to keep AI code properly maintainable.
This glosses over the large capability app between local options and frontier options. Which applies in many ways, but includes drafting more maintainable code. I would definitely not want to limit my devs to what we can run on our own hardware.
The thing is, fully-test-your-code wasn't economical for software houses before the rise of AI. Perhaps it will become economical now but I'm not sure what would force the issue.
I see this everyday. The problem is code is the wrong abstraction for the work we do. LLMs have solved coding, but they haven't solved systems, collaboration or system maintenance.
Edit: Since I seem to have touched a nerve - I've been working on a project to solve this: https://www.archme.io if you want to know my thoughts on the right abstraction
I would say it means you are building software but no longer are concerned with writing or editing literal code. Your concerns have moved up the stack to managing requirements, context, and verification processes.
Just to make this clear: if you can define a really good PRD and sophisticated technical specs, and a strong set of tests cases to pass, at the right level of architectural granularity, plus adversarial code review processes that triangulate and weed out most mistakes, SOTA agents can write the code autonomously, at or above the quality level of most human coding teams. I call that "solved" but only if you meet those context requirements. Which is still hard, not solved, at that layer.
Solving writing is not a good analogy IMO. Writing is for human consumption, and cannot be wrapped in objective requirements and verification processes. Certain forms of writing perhaps could be (can't think of one at the moment but I don't doubt some exist), and those forms might be good analogies for being "solvable" or "solved."
I agree with you, but it's a moot point for software engineering.
> Your concerns have moved up the stack to managing requirements, context, and verification processes.
This has always been the concern. "Add oauth to this app, there are no requirements beyond oauth working" has been an intern level task for ages. What makes software engineering hard is when the requirements start adding "well it has to use this oauth backend that isn't technically spec compliant, and the user is going to send some kind of random token you need to translate to oauth, and...". The problem isn't in writing code that will do the thing, it's figuring out exactly how that backend isn't oauth compliant and what chain of API calls I have to make to convert their random token into an oauth one, and etc.
Producing software that complies with a test suite isn't really novel. You've been able to outsource that forever. This falls apart in the same places outsourcing does; I'm sure India/Phillipines/etc/ is more than capable of iterating on code until it passes a test suite.
> no longer are concerned with writing or editing literal code
I'm not typing keys, but I am very much still concerned about the quality and nature of the code. Coding to me is more than pushing keys
> if you can define a really good PRD and sophisticated technical specs
I still believe we cannot waterfall software, the idea seems like taking a step backwards. How often do we learn about an unforeseen complexity only after getting into the implementation?
In my experience with agents, it's better to be iterative and in-the-loop. Start with a decent description, have them research the code/issue, write up an initial plan/design, work iteratively on writing code and updating design doc, review and finalize the code and markdown. Then future agents will have some resources to shortcut understanding the code base.
I'm not talking about not just punching the keyboard to type out the identifiers in the code. I'm talking about not making decisions about most classes and functions, most type definitions, most of the weedy logic within most modules, etc. If it can't be described in natural language as requirements, or in a typescript type or other data shape DSL (I think key data model types are probably still important to own), it's probably too weedy.
Natural language test cases still define the expectations both at the product and architectural level and are essential for triangulating the agents on successful outcomes.
A requirements and verification approach with agents is not waterfall any more than TDD is waterfall. Does thinking ahead and doing some planning equal waterfall? Does describing how a feature works to an end user, and making some key technical decisions, before you build it, mean waterfall? Does having some sense of what you're building first mean waterfall? With agents, you can specify (with PRD and technical specs) what you THINK it should do, and in minutes or hours or at most days, have the result, which you then learn from and iterate. If you didn't fully think it through, the agents will do one of three things: 1) decide for you, which you learn from 2) stop and ask, which you learn from 3) introduce bugs, which you learn from.
How are you thinking about technical debt in this new agentic age? (also markdown debt?)
It's one of my bigger concerns and I use this iterative approach to try and reduce it... "go look at the ./cli directory and generate me a report of inconsistencies blah blah..." or "review ./research/something.md, validate consistency, correctness, claims, ..."
... pseudo prose, have we made that a thing yet like pseudo code?
"solved coding" is type of thing you say if you want to sound smart.
Saying that LLMs have "reduced the cost of coding" would be boring. And using your analogy, pencils, typewriters and computers have all reduced the cost of writing, but writers are still around.
I'd separate "coding" from software engineering. Software engineering is not solved. The new challenge in software engineering is how to properly use AI to maximum leverage and nobody has solved that.
But, "coding" is, except that solving it means using techniques that are still barely understood and barely described today (something I'm hoping to be able to communicate better myself about). It is now possible (at least for most consumer software, I would not say applications where human life or extreme risk is involved) to work exclusively in the domain of natural language requirements, natural language architectural/design decisions, and natural language test cases, and deliver product of equal or better software quality than average human code authors could have produced. Even in a language that you plausibly don't actually know how to code in, because the language itself can often be separable from the requirements and verification process.
If someone who doesn't know a language or framework can plausibly produce better software using it, and faster, than a team of people who do know the language/framework (and I would strongly argue that is more than plausible now with models like Astra and Fable) then it's not just "coding is less expensive."
It's like saying computers make complex calculations less expensive. That's true, but it's missing the real paradigm shift.
I'm not trying to sound smart, I just think it encompasses what most people now acknowledge. LLMs write code faster and generally better than needed for most cases, and that seems to be becoming the prevalent opinion seeing as most folks have stopped doing PR reviews.
The problem is, without PR reviews & strict oversight, we're losing knowledge, system design & control of our codebases & products. Which is why, IMO, the coding is solved but the other parts which used to be so tightly coupled to programming are cropping up as their own issues.
This ties into my experience: greenfield projects turn bad way faster than legacy ones, because AI has no established patterns to key off and it does increasingly dumb shit.
"Reducing the cost of coding" would imply that you still have to code, but with current LLMs you no longer have to. You can write 100s of thousands of lines of code without ever having to touch a single line of code. Literally "here is a git repo, here is an issue, please fix".
You might still need to nudge the LLM in the right direction or stop it from going off weird tangents, but none of that involves touching actual code yourself.
Opening the IDE and typing program code. With LLMs you don't have to use an IDE, you don't have look at program code, you don't have to care about coding standards. You ask the chatbot to write you a program/feature/fix and chatbot does it.
Is chatting with a chatbot still "coding"?
The part that isn't fully solved is just the software architecture side of things, do you want library A or library B, or write it all from scratch? LLM can do all three, but if you aren't careful it might go down a route that you don't like. But that again can be fixed with chat, "replace A with B", not coding.
So lets say you plan the software architecture and take a person off the street, give them a frontier model and tell them to clone Figma given your architecture
As a developer, a good way to reframe this and step away from your own domain expertise, is to read Terence Tao's conversation with ChatGPT and ask yourself "could I have done that?"
It solved coding as in it's significantly less of a bottleneck than it was before.
At least that's how I experience it. In the before times each non-trivial code change had a real opportunity cost as it would easily consume two days until I could even estimate whether this is worth looking deeper into.
Given a working system, like a web service or services, and their code, non technical people can now make effective changes with LLMs.
For example, nobody on our team writes manual code anymore, we have basically set up a harness where an engineer types up the requirements for a change, the system implements it given certain constraints, we have automated unit and integration tests that are ran, and if any errors pop up, they get fed back into the loop until fixed.
But to do that, you need to actually know what you are doing - you have to have good instructions to keep the agents in check and not start making mods outside of their bounds especially when the issue is with a dependant service that is causing errors.
I'm well on my way down this path too, but that is a process implemented in markdown. I don't see coding as solved, or a solvable thing at all, like writing
To solve something, there must be a defined problem, what is the problem that was solved. Or perhaps it is just "coding is solved" is the turn of phrase de jour be ause we haven't yet found a more succinct and accurate way to describe the paradigm shift
When it comes to non technical people using Ai to build things on code, the outcomes are on average pretty poor, which i see as evidence that the driver and their expertise behind the Ai matters a lot. A notable example is Terence Tao's conversation with ChatGPT, us math normies could never have done that. The same applies to coding agents ime
Writing is a means of expression and communication. Code can be those things, but that's not its primary purpose.
I think "solved coding" is taking it too far, but for many projects, the mechanical aspect of it has been removed or reduced greatly.
LLMs will have a much harder time "solving writing", because they cannot develop their own style and so are severely limited, creatively. This is less important for coding.
I agree that writing is much harder, largely because there is no way to get concrete feedback for "does it work"
I still think they produce shoddy or sus code too often, an artifact of the current generation's training to try anything and everything until it "completes the task". They have a hard time even with that concept, which is part of "coding" imo
maybe it’s more like “abstracted” away, in the same way higher-level languages “solved” needing to code via machine instruction sets. Higher level languages allowed humans to think more like themselves. AI puts another abstraction in front of the outcome, making it even more generally open to human thinking and less defined by the need to give machines exactly what they expect.
Today, human-language outlines / briefs / prompts are “compiled” to code which is itself then adapted to hardware. We are stretching less and less across the divide, doing less and less work on the terms of the machine. Now the farthest we’ll stretch is often formatted markdown - the most basic application of machine-parseable structure to very organic human thinking. Because we’re given the chance to be less precise, coherence suffers.
even setting aside the artistic side of the profession, I see these things make all sorts of basic mistakes, so even the mechanical part doesn't seem "solved" ime
Wow, I think you have never had a chance to communicate with someone selling their expertise for $2/h. Unless your time is also worth $2/h, I think you would get better results prompting local 27B model.
Same problem though :). If an overseas employee knows the code and you don't, you have ceded ownership, problem solving ability & future direction of the project.
>The problem is code is the wrong abstraction for the work we do
My team recently spent two weeks on a wild goose chase trying to figure out why TensorFlow Lite was generating nonsensical OpenCL kernels. Well it turns out that LLVM had a few bugs in the RISC-V assembly for our platform that was leading to silent garbage. It took combing through assembly dumps, hexdumps, a lot of pain staking debugging, and going through the TensorFlow Lite source code to to track this down.
In your opinion, if code is the wrong abstraction to be working at, how do you approach this scenario?
Fair question - IMO it's the wrong abstraction for building and collaborating on a new product with a team.
To your point, it's not the wrong abstraction for solving code level bugs. Just like python is not the right abstraction for solving memory corruption or pointer mis-alignments.
See my edit on the right abstraction - how do we get humans able to collaborate with agents effectively, where knowledge flows both ways, without needing a human to read 10k LOC, or even just lots of long LLM responses, to follow along
Or more broadly, LLMs fundamentally don't "understand". They can simulate understanding and generate text/code/whatever, but they don't have will to engage with something holistically and "own" it.
I have been trying to define "understanding". Is it when you can predict something that you understood it? Or maybe when you can explain it? Or how about when you can control it? Or invent it.
This time I add another definition "when you can own it".
When you can own it - I like that. Humans are still needed to own systems and drive/direct changes. The question is how can we collaborate as organizations and teams to still own it when we don't write the code and can't keep up with the output
tbh, I don't think of LLMs as junior-devs. Maybe 6 months ago, but now they are v. senior code-monkeys. They are masters of their craft, but their craft is narrow and lacks big picture, collaboration, org goals, etc.
You can immediately tell someone has absolutely no fucking clue what they're talking about and dismiss everything they say as soon as they say "solved coding". You must write the most heinous code imaginable if you think LLMs are better, or even anywhere near, what a competent professional can produce. So funny to listen to terrible coders talk up terrible code.
In the first place, what does it even mean to say that code is the "wrong abstraction" for the work we do? You can't actually abstract away reality. Maybe you wish we could, that programming computers wasn't about contending with physical constraints. But it is. It will never not be important to have control over what the hardware is actually doing. Abstractions are temporary conveniences, not a replacement for understanding what is being abstracted away.
Thanks my friend - I'm not an AI evangelist but I think it's pretty clear that AI codes very well at this point. So well that I'd hazard a guess that most software engineers don't write code anymore. Sure, maybe a professional could do better if they spent way more time on it, but it has always been a tradeoff we programmers have had to make. Getting things done + technical debt or writing the perfect code and over optimizing. And LLMs write working code, so the rest is just semantics on that scale.
Code being the wrong abstraction is similar to assembly being the wrong abstraction if you're trying to write a web browser game. Sure it's possible to do it and yes, you may end up with more optimized code, but it's not necessary and much slower to do that. Just use Javascript.
And now, people are building larger applications & functions quicker, and it no longer makes sense time-wise to look at Javascript for-loops when understanding the work. And that's because those for-loops are generally correct, if maybe a bit under optimized. Instead, engineers need a new layer of abstraction, to understand what the code is doing without having to read each line.
LLMs are faster until the technical debt comes due, at which point all supposed productivity gains evaporate and you're now 10x slower than doing anything by hand. They have a use if the debt never comes due, eg. for one-off scripts or for prototypes you will responsibly dispose of. The debt does come due in production, very quickly.
LLMs write code that compiles, which they accomplish mostly by robotically attempting the task and repeatedly fixing compile errors in a loop. That's a very narrow definition of "code that works", and I would argue it is the starting point, not the finish line. My definition of code that works is more like: secure, stable, maintainable, efficient, performant, effectively bug-free, with an ergonomic interface. LLMs fail on every single fucking count. I routinely observe generated code that leaves trivial 10x or 100x gains on the table, while having severe deficiencies in every measurable approach.
Given that you talk about assembly as a bogeyman, though, I gather that you're from the generation of "software engineer" who was already writing insecure, unstable, unmaintainable, 100x inefficient, 100x non-performant, bug-ridden JS for everything. LLMs can replace this class of people, it's true.
I feel related to it. I switched companies at the end of July. First 4 weeks AI was giving me the feeling of freedom. No longer I need to understand tens thousand of lines of legacy codebases. Never onboarding was so easy. Just ask Claude and it tells me what happens here and how.
But after 2 months it starts to backfire me. I still know nothing. I have some understanding of the system design and core components but I have zero clue about how certain things are done under the hood. Because AI read code for me and code for me and I take it as my own understanding.
In last week I end up limiting my AI usage and forcing myself (it is really hard) to read and code at least a bit by myself to start having any idea about what is going on here.
AI is not for programmers, they will always complain and given time and resources can of course produce better atomic code. AI is for a combination of a business analyst, architect and product manager in one person who has capacity in coding but decided this is limited role in the whole software engineering lifecycle. With the right attitude and of course certain level of micro management over AI (which they had to do with human programmers as well) they can achieve the same or similar results with much less communication friction, less people to maintain that teamwork and get closer to solving issues they find rather than discussing philosophy of programming and other manifestos of the craft that stopped being prestigious or unique and that’s the key point of complaint when AI is given wrong people to use.
That's my recent experience as well. You have to know the domain and have technical knowledge so that you understand (not in every detail) what AI is actually doing and why. You also have to understand what you want and why you want it.
But once you do, working with AI is like having a very capable team of programmers knowing all programming languages you might need and understanding your goals.
Can they really achieve similar results though? If quality suffers greatly, and velocity eventually slows down due to huge tech debt, isn't that just a worse outcome?
My 12+ months learning is that quality suffers primarily when I cannot find the right working process to assure it. It's not software development as a process but software engineering as a whole holistic picture across all departments and traditional roles (dev,test,devops/sysops, pm, etc..)
I also found that what is commonly called "spec-driven development" is just a different CMMI alike model that works better with AI than anything AGILE. which is also nothing new.. just realisation that AI is the ultimate proof agile development/manifestos is a dead end.
there are many other reflections about the whole process to ensure quality, good architecture, minimizing regressions and providing business continuity and proper, reliable, SLA driven maintenance.
Actually not that much different than certain human-centric models.. just need to be adapted to the AI reality to truly benefit from it.
when that is done, from leadership perspective, it is a working process that brings similar outcomes and quality. not exactly cheaper atomically, but faster and brings more scalable opportunity to drive multiple projects within the same resource pool.
I think that's what is mostly experimented today and that is also of course exhausting traditional developers.
If you believe software still can be sold. as I do believe further development of AI may completely redefine buy vs. build capabilities and potential directions.
My bet is that half of the software developed so far may be obsolete 10 years from now and so the businesses behind them. SaaS specifically.
I'm seeing this right now in my domain. Much stronger lean towards build than years past. Not enough signal yet to really make a call but I suspect a significant downward re-pricing is going to have to happen to convince folks to buy. At the moment the arguments of "your core business isn't software so why build?" or "you'll have to maintain this and there's no one else on the planet that knows it", etc are not working since some of these folks have fully bought in and just say "yeah so, AI will do it." Also the number of features expected, and many non-trivial, is pretty high. wild times!
It's not a given. It's about willpower and care. LLMs are the ultimate crutch. I over-rely on the crutch, more and more people will over-rely on the crutch. But it is still inherently a psychological problem.
You can still know things and get force multiplication out of LLMs, if you are disciplined and caring enough. In practice, most people won't be. And you can't force other people to be. But you can force yourself to be.
> You can still know things and get force multiplication out of StackOverflow, if you are disciplined and caring enough. In practice, most people won't be. And you can't force other people to be. But you can force yourself to be.
AI is going to make, long term, Software Engineering more important. Getting requirements, user feedback, tests, product vision. These are key differentiators now.
If you can really get a good set of requirements, go and write all of your test cases out, and then throw it at an AI that will one shot it. Its perfect.
You're basically describing a waterfall process. That never worked in a pre-AI world and won't work in an AI world. Why? Because it's just not possible to gather absolutely every requirement up front. Software development has always worked well as an iterative process. You come up with better solutions as you go along and learn about the problems you're solving and make improvements. While AI will be able to one-shot a set of requirements, you'll miss out on those improvements because you'll have skipped over those phases.
I've tried this out. Even with relatively small applications and with spending hours reviewing the spec documents, there was always something I missed or something that wasn't quite right when seeing it live.
"Getting requirements, user feedback, tests, product vision", that also apply to iterations. Iterative processes are just a cascade of small waterfalls - either a scrum week or just a single agent session (define, prompt, plan, implement, verify)
But today you can do even better, with AI can do true waterfall by rewriting from scratch many times.
> But today you can do even better, with AI can do true waterfall by rewriting from scratch many times.
This does nothing other than ensure you end up in the "joy" of running a v0.0.1 product but for years on end instead of for a few months.
I hear this sort of thing a lot, and I can't help but internally translate it to "I've never had to take oncall for a product directly after a rewrite". It will be broken; not even because the AI is "wrong", but because the rewrite has new edge cases. No one rewrites a project to have the exact same edge cases. Those edge cases will become outages. No one will learn anything, because a month from now it will be rewritten and those edge cases will get swapped for something else; you can pick which edge of the CAP theorem you want to live on, but you can't pick "none of them".
Really I was thinking about exact example announced recently on HN: Postgres rewirte in Rust, it has 3 versions looks like each one generated from scratch https://github.com/malisper/pgrust
In this moment there are a million ways to do things wrong.
But there are also some ways, maybe less than a million, to do things right.
Here's an anecdote about a way to do this wrong.
I have found with AI coding methods that there's a line where it becomes a hail-mary (in the American Football sense).
A hail-mary is when you throw the ball to the end zone and just pray someone catches it. This is almost always at the end of the game.
This moment with AI code is indicative that you can't put together a coherent plan so you just tell the agent to "make it good". It used to be that the results here would suck, but now the agents are really competent, so the results might be good.
But at that moment, that's your cue to back up. Because as soon as you take a solution that's so far detached from your understanding, you're underwater. The hail mary is not part of a larger game plan. It's the last play of the game. There's nothing after.
So as soon as you reach that moment in your coding, you're signaling that you're done understanding not just the code, but even the way it works at a high level. If you're still going to work with this code after, then back up and work with the AI to get more understanding of the problem.
Yeah we forgot the goal of programming isn’t just to tell the computer what to do, it’s to program the programmer into thinking a deeper understanding of the problem.
As a programmer, today I can confirm with no room for doubt that programming is dead (programming by humans to be precise, the act of typing code on a premise to present a solution to a problem)
We may consider ourselves the last programmers before the AI era
Each to their own, and the market will decide in the end? If a company is going to (implicitly or explicitly) incentivize everyone to directly ship Claude’s output straight to production, and prioritize features above everything else, that’s a choice. Providing that I have a long enough runway, I’d love to have this company as my competitor? Yeah, year 1 will suck, because they’ll pull ahead, but if I can stay in the game until years 2 and 3…
Aside: this reminds me of running a “negative split” in a long distance race, where you aim to run the second half faster than the first. It’s very hard to do this because you have to be willing to let everyone else in your pace group pull waaay ahead, and running above race pace in the beginning feels “free” with all of the adrenaline. But if you do manage to stay disciplined, it’s a fantastic feeling to reach half-way with gas in the tank, and then start to reel in all those runners who sped by in the beginning.
> because they’ll pull ahead, but if I can stay in the game until years 2 and 3…
well as you said, the market will decide in the end. You may be even further behind in years 2 and 3. btw, we're already in "year 2 or 3" territory for some, i wonder how those companies are doing vs their competitors who adopted AI full throttle?
Most people say full agentic coding only got good enough for production at end of last year.
This year was the turning point for most companies. A lot of experiments going now right now.
Of course things can change drastically in the next years, as they already have in the last few.
Only thing I know is that it is really stressful working on this rat race
Yes, customers are sticky, but every customer has their limit.
And of course, this is very much dependent on the switching costs. Which sucks, because tech companies are great at making this as high as possible; the EU is trying to do something about this with their data portability legislation.
Anecdote: I worked at a company providing infrastructure as service. We lost some bids the first time around, likely for a variety of reasons (price, features, ...). And, some customers came "back" to us after using our competitors' offerings and being burned by their reliability. Yes, switching costs were real, but the pain of losing their customers was even higher.
We've had team changes and product manager changes and lack of documentation for so long that this was the state of our team anyway: no one knows why it was done and no one wants to break it.
> But the final boss is, and always will be, maintainability.
I think this article misses the mark in a few ways, but this is the biggest. I see AI as the ultimate solution for maintenance, in three ways:
* AI does a great job of refactoring and so tech debt becomes shallow. If something is not architected right, it can be fixed much more easily than ever before.
* Bug fixing is also a great AI strength. In the future there won't be a backlog of all the bugs that never got fixed. AI can fix them as quickly as they come in.
* AI doesn't get bored, doesn't get tired, and doesn't care how crufty the code is. It is happen to maintain any application without judgement.
I some very real way this has been true for a very long time. Does anyone know how a AMD Ryzen processor works? I mean does anyone understand in any detail how it works from machine code down to the transistors?
Does anyone single person understand what’s happening when compiling a large C++ code base? Meaning, can anyone track the basket cast C++ language constructs down to the Clang IR to the optimized machine instructions? From there can any one person follow those machine instructions all the way through to the actual registers etc to actually running the code?
I'm understanding my role as a developer in the AI era through the lens of the company hierarchy at my first job.
In 2018 I began my career at a management consulting firm which also did tech consulting for a very specific industry.
The hierarchy was as such - associates at the bottom who did all the manual coding work and got into the weeds with the tech, consultants on top of them who orchestrated the work and kept the wheels moving, and managers on top of them who had a more higher level view of the system and knew why decisions were being made (partners were involved with future deals and driving the business forward). The managers who knew how the system worked had no idea about the actual code underneath it all because it was abstracted away from them through multiple layers of hierarchy. The actual code which was mostly written by 21 or 22 year olds who largely had no idea what was going on other than the fact they had to make it work or risk the ire of higher ups was a mess, unless the consultant orchestrating it all had real technical and system design chops.
I was an associate then and more often than not I had no idea what i was doing but i was fortunate to work with consultants and team leads who did. Right now, I'm a software developer akin to a consultant, with more ownership of the code but no associates to guide because all of that work has been pawned off to AI agents. I see my role gradually morphing into what the manager role was at the consultancy, where i have a bird's eye view of the system and know where the project is moving and why decisions are being made, but with a very limited knowledge of what is actually in the code.
I've noted a similar shift, but I think it's a short-sighted path.
My experience (both in and out of management, and as a very high level IC at a mix of companies) is that while the management layers generally didn't understand what was going on in the code - the decent managers tended to have a "gut check" person they trusted to go and ask technical questions, and bounce ideas around with.
This was their "expert" who they trusted to be close enough to ground truth to tell them if a thing will work, how hard it will be, if there are shortcuts, etc... Often times, it wasn't even strictly technical questions that would get answered, it will be things like "how much of this project is technically in my sphere of control, and how much do I need to go align other teams? How many teams? Which teams? Which projects? etc..."
My problem is that AI SUCKS at this job. So yes - we're getting pushed into management roles as ICs... but instead of a decent subject matter expert who still understands ground truth... you get an LLM blabbering on about seams and load-bearing joints, and clickbait style headline summaries, but completely non-nonsensical judgement calls about how to interact with the people who make up the company.
Ultimately, work is also politics - and it's not just "writing code" that matters. It's knowing what code is worth writing, and when you should write it. And an AI will happily tell you that a single change-set in a single repo is "substantially larger then expected" and put a timeline of "two weeks" on the completion (especially if anything Jira related happens to be in context... blegh), and then knock it out in 5 minutes time... While also spouting off about how easily it can add "new feature X" it just needs to get PRs in across 5 different repos, owned by different parts of the org with non-aligned managers.
I don't really understand all the negativity - I've delivered amazing robust solutions at about 10x my previous rate I think and can make dramatic cross code changes on awful legacy code bases with ease. I just added some complex prev/next navigation across a load of detail pages pages that rely on the current search, a new PDF generation system that uses a queue and workers and absolutely tones of changes to a legacy ERP system that would be unworkable without AI.
Dealing with legacy messes I used to get frustrated and bored of making the improvements, now it's easy to clean up code bases and write loads of tests. I asked Astra to come up with a plan for Playwright testing the whole App, I have not built it yet but the flows suggested were fantastic as was the ephemeral database we plan to create for CI.
I built my friends portfolio website almost entirely vibe coded in 4 hours and it looks unbelievable, we added so much slickness (he's a designer) just prompting together. I used a CMS I had never used once before and it was so so easy to do without any of the usual need to read docs about everything.
I've done so much devops now I'm actually fairly confident that me and an AI can do anything you want in terms of deployment/infra and scaling from AWS to Terraform to whatever.
Anyway my main concern about this technology is not that it is crap at coding it's that the improvements in how it codes and thinks are absolutely dramatic which is extremely scary - it has come so far in a year I wonder what the next year will bring.
When coding agents first came out, the overwhelming narrative was how bad they were, they don't know what they're doing, the code they write just sucks and they hallucinate APIs.
While these criticisms were technically true, they were stated mostly out of a sense of insecurity from people whose jobs were basically spending years and years just glueing code together and mixing APIs to display some CRUD apps rather than out of a genuine concern for whether LLMs were actually producing poor products.
Nowadays those concrete criticisms don't really work anymore, LLMs are pretty good now and surpass most developers when it comes to writing the majority of shovelware that people have been employed, and so the narrative is changing from concrete criticisms about how LLMs were genuinely not capable of writing software... to these kinds of abstract and philosophical arguments that are really hard to argue against because they make no concrete claims.
If you say an LLM can't implement a feature, we'll we can test that claim concretely and LLMs are getting much better with every new release. If you say its code is slower, buggier, or less maintainable, those claims too can be measured and once again they're getting really good at these. If you say it takes longer to complete a task or requires more human intervention, we can compare it and measure etc...
But now the objection has shifted not to LLMs are incapable, but people are now incapable and LLMs represent a degradation of the "craft". And here there is nothing left to test or falsify. The argument has stopped being about whether the LLMs work, because that's verifiable and they are now at a point where it's hard to argue against their ability to actually produce functioning software, so now the argument is about whether people are morally, culturally, or intellectually permitted to use it.
I'm with you here. If we skate to the puck, LLMs will continue to write tighter and tighter code, comments, unit tests, integration tests, etc. I would also assume that it will become increasingly good at code organization, architecture, and understanding intent. i feel like I am a series of monthly shifting of the goal posts...."LLMs can't autocomplete code", "LLMs can't write a high quality function", "LLM's can't understand small repos". "LLM's can't understand legacy code bases".....
If it's this easy for you it's just as easy for anyone else. What do you think will happen to your salary when no particular skills are required to do your job?
Next year you may be unemployed (because your boss will cut by half the engineering department), or perhaps your salary will be cut by half (because your boss will route your other half of your salary to openai/anthropic), or perhaps you will get half insane (because your boss will ask you 10X the output you used to have but within the same timeframe)
Whatever the path is, regular engineers (90% of the people around here) will get screwed up one way or another. But hey, playing with LLM agents is cool!
Yes I agree we have all signed our own death warrants. But this does not just apply to coding.
Are you suggesting not using the tools? A kind of technical version of the Amish way of life?
I dunno I think my guidance and testing is very important and I make sure not to ship things with bugs and code that is really awful. I'm still just about necessary for now.
You may not be at a company where people are dropping AI slop into the codebase faster than it can be cleaned up. I've thought about my options for going "single operator", so I don't end up with a mismatch between people who use AI thoughtfully and those those who don't.
Writers disagree on what effect word processing will have on the quality of our written language.
Some writers are concerned that computer assistance may promote dry bland writing.
I don't mean to be mean, but the problem is that AI has exposed how much programming is designed around human usability. C/Rust/Go wrap assembly. TypeScript wraps JS. Java/Python wrap C.
Do you need all these layers of abstraction when the human is no longer looking at the code?
Best analogy is forgetting how to use a slide rule following the advent of calculators. The former was made to make hand-calculation of logarithms easy. The latter does these calculations directly (obviating the need for a slide rule at all).
I think what humans still need to learn are the theory and domain fundamentals for their industry. If that industry is computer science, that means algorithms, calculus, linear algebra, etc. I think the future of CS is then (a) theoretical human-drive design and (b) prompt engineering to implement and verify that design.
It would also be helpful to have domain knowledge outside of CS as having the skills to build something is nearly commoditized (outside of the above fundamentals).
There was a post a few days ago about why control-flow-macros aren't that bad, and their thesis was basically "because AI can understand code well now", though I feel like that sort of missed a bigger point of "if AI can understand the code easily, what's the point of macros at all?"
Even hygenic macros still end up being there primarily to increase legibility and ergonomics. Good ones, like core.async, make it easy to understand how threads and the like are glued together, but ultimately it's just syntax-sugar on steroids.
If the goal is not for humans to read the code at all, I'm not entirely sure of the point of syntax macros; the LLM could just generate the expanded code.
I think it is not needed to own every line of code anymore , but you should still have control and idea about the architetcure of the system. You should understand what componenets are there and what is the repsonsibility , how are the orchestrated and what are my quality gates where i messure if what i requested match the results. Thats why i build archkeel (https://github.com/rapiddweller/archkeel) and datamimic (https://github.com/rapiddweller/datamimic) ... to increaes the transparency and review surface for human ... to make the results easier to judge ... i think when we stop knowing anything, we can also stop burning token and ressources
I agree writing completely new systems with LLMs is now near impossible to keep in your head, however, the onus is STILL on you the individual engineer. You are mistaken if you think that's changed.
So, if you're pooping out code, and committing it because tests still pass, and that's all you know, you're in for a treat. When an executive wants to know why a b0rked feature lost their department millions of dollars, guess who will have to answer for it, and its not the LLM.
My advice is to find ways to keep on top of how it all works, and if you're the only one who cares, well, then, that makes you even more valuable, not less.
> the onus is STILL on you the individual engineer. You are mistaken if you think that's changed.
In a lot of shops right now, this leads to an accountablility-authority gap. Where AI changes are merged in quickly without review and without my input, how can I be responsible for understanding the system? I can't. Its the same with reliability.
A lot of good engineers take it a a personal duty to understand the system and keep it up. The way a lot of businesses are using AI makes that impossible. And your job might just be cranking out features, with regard to little else.
I think this is what is stressing a lot of engineers out. It might be worth having a conversation with your boss about what you are actually accountable for. If your boss agrees you are not responsible for uptime, reliability, security, or even understanding the system you might find yourself much happier. If you your boss wants you to be accountable for those things, then you should feel empowered to ask for authority over the things that give you control of the outcome.
The post seems to be in line with some of my own direct observations. I've been pondering recently whether for many devs the use of AI is the death of the mental model.
We all interact with systems through mental models, but if many devs are just prompting claude when something doesn't work, they might read what claude found, but lose out on the exploration, debugging, and work that builds and reinforces the correct mental model and discourages the wrong one. And if devs are missing out on the mental models, will they actually be capable of driving efficient solutions to problems as the mental models get worse.
that tweet about the fast moving startups looks like almost the hypothetical AI nightmare situation just dreamed up. Assuming it's real, i mean yeah, people shouldn't work for these stupid high-moving startups I guess. From my perspective, "when did startups NOT suck?". The code was always garbage at startups, the push to work 12 hour days was always at startups, "nobody is resolving bugs" ha well yes, welcome to a startup? I hope the guy is at least getting paid and doesnt have to hire a lawyer to get his checks like I did. startups suck
I'm curious how much people who code this way now are spending in tokens every month? Where I work currently, we operate on a $200/month token usage limit. For me personally, this has prevented me from just endlessly prompting claude to fix bugs.
Even with unlimited spend, it seems immensely beneficial to dig into the code base and fix a certain amount of bugs oneself. Oftentimes this is ends up being quicker than having to type out a detailed explanation of an issue in plain language, with the added benefit of maintaining intimate knowledge of the code base.
The architecture and the intent don't matter to business folks, it never has, and with LLMs it matters less than ever.
Vibe code into production is satisfactory regardless of architectural understanding or intent. If there are issues, just have the LLM spin up some agents to play whackamole until the issues are pushed beyond visibility.
Ultimately, the idea that we can hang onto fleeting engineering disciplines misunderstands where the industry is going, regardless of any assessment of the LLMs capabilities.
I don't think it guarantees job security, I think the future is an adaptation to the prodslop paradigm where the answer to vibe-coded monstrosities is more LLMs.
Ultimately, the industry will be forced to adapt to the extinction of the junior to senior pipeline, but investment in more tokens is going to be more attractive than paying specialist engineer salaries, especially since LLMs probably have a few more generations of qualitative improvements before diminishing returns truly set in.
After experience on both ends of the spectrum, I arrived at the position that what we basically need is to be a "Responsible Human in the Loop (RHITL)" [0]
> You may not write the code by hand but you understand it enough to investigate and fix it when it fails. It is how I think we should leverage AI instead of becoming a meat proxy.
Don't touch the code for 10 years and even the brightest will experience serious dip in skill. Most probably much earlier, our minds are lazy by default since creative thinking is energetically draining. Then try to debug some harder problem llm created in its own code. Look at what its doing to students.
Its like saying you can keep fluent level of French as a foreign language to you by just reading it. Nope you won't - do it for 2 years solidly and you will have problems forming basic sentences correctly if you don't use language actively, forming foreign language constructs in your mind.
Those few of us who are largely apart from current llm craze (due to my mega corporation and its obscure rules and procedures) are not missing much I see. I still happily code changes by hand and at most consult basic available llms via prompt, its beautiful process to create fix to a problem which seems cryptic at the beginning, and knowing it all came just from my mind. By far the best part of my corporate job, will keep hanging to it for as long as possible.
People keep saying this, but how realistic is that in reality? Even say a person is doing their best to do that, we all know that the alarm that doesn't trip 99% of the time isn't going to be closely monitored. Just look at what happens when people get used to "hands free" cruise control.
Not only that, but even if we assume best effort on the engineer, the business pressures don't often allow that. I'm under constant pressure to deliver more, faster with less people. The performance eval ladder at my company was just reworked to double the amount of deliverable features expected per job level per year. Our CEO told us we should be able to deliver what took us the past decade to deliver in a quarter, every quarter going forward.
How could you possibly have a human anywhere near that loop with those demands?
My boss-boss asked me to do a presentation on a topic I actively research for quite a while now.
Scheduled a quick call to align me on what he expects - normally he wouldn't do that but he has attached a big agenda written by Claude what the presentation could show, and invited two other product colleagues of mine.
I came to a Miro board of the Claude Agenda, put into Miro using the MCP.
Honestly, just tiring. Asked my colleagues if they would just put the Claude agenda into Miro with Claude, what they need me for when an AI could just narrate it.
No the problem is allowing changing of system architecture or intent because the AI just did it and being totally trained to mentally surrender to the whims of it.
You could argue the same with high-level vs low-level programming languages. But I do get that English is so abstracted and non-deterministic that we're kinda clueless of what's going on. At the same time, I do think that maintaining code does require some level of understanding. And unless you straight up one-shot something perfectly, you still want to test manually and tweak with a few more prompts, adding a level of 'how does this work?'.
This is also freeing up time to explore things that before you wouldn't have been able to even start. New fields in tech are opening up. It's all about the model, compute, plugins, third parties... and more to come!
And this is not limited to code, but also how the world works. It's all being abstracted into prompts. Funnily enough, I'm also learning that way...just taking less time to get to the point. But this is not the first time we go through this. Eg. Google vs a library. And like anything, if no one knows anything anymore how do we distinguish from one another? There's a level of wanting to understand in order to distinguish ourselves from the rest in the serendipity of everyday life.
Call me naive, but something tells me we're going to start being much more open to just exploring the world with all this time we just bought ourselves thanks to technology. We were always gonna get to this point and there'll undeniably be bumps ahead.
This page has been live long before AI appeared, I used Quartz [1] v3, now at v5. But added some features I like with AI, yes, but still keeping my beloved color theme and web style as I had it for a long, long time. The whole code for the second brain (ssp.sh/brain/*) is on GitHub, in case you want to check it [2], but you can also see the "Changelog" in the footer to see recent visual changes. But I'd be curious what makes you think it's AI-generated?
good point, thanks for the honesty. I will change that, I think the color inside pop up make sense (red removed, green added words), but the green dot does not add value.
How humans can differentiate the things they make has been something I’ve been thinking about a bit recently. What markers signal to you this is AI generated?
I mean, the content doesn't look AI generated. I couldn't care less if someone AI generates their blog template, the rest of the content seems meaningful.
I predict we will see more and more of these convoluted rationales (i.e. excuses). All these basically boil down to: “I know AI is crap, but I have found a way to make it useful. Trust me.”
I am gonna appeal to Occam’s razor here and say, the integral variable here is AI, and the only variable you need to know is AI. The problem is AI.
This is the perspective, not code, coding is solved. The wrong input will produce wrong outputs, so a mediocre dev using AI may produce mediocre results. AI can do everything asked for, code, tests, reviews, fixes, etc but still it can not think, for now
Intelligence is the new currency, but it will be short lived
The future of engineering is product management. I don't believe there is any world left for people whose primary responsibility is opening pull requests; and we're seeing the angst against this happen from both directions. Engineers hate it. Leadership feels they don't need them. The truth is somewhere in the hazy middle; we're just in an uncanny valley right now where neither side can take the leap to cross the valley: the agents aren't good enough yet, and the bigger problem is that there's no job title or corpus of experience leadership can look to and say "Yeah that's the person we need in this role".
The labs saw this early, and thus many roles at the labs are "Member of Technical Staff". That's the future for every software team. You're not a software engineer anymore, but you're also not a PM, nor a designer. Think horizontal slices, not vertical: Every human's responsibility is to leverage AI to be an expert on everything necessary to deliver some vertical slice of the business.
Product Managers have been claiming this for years now, but put an AI in front of them and they also just start asking it to do their product management job for them.
Worst ones I witnessed were just using it to hallucinate tickets. I saw one even fired because of that. Endless mountains of text going absolutely nowhere, both me and the CPO thought we were going insane from reading so much Claude-ish.
Best one I know used Copilot to code a "Notion to Jira" exporter so it copies requests from business people into Jira. Automated their own work, did nothing else other than monitoring the Python script daily and running ceremonies.
They can pontificate all they want about taste but: modern apps are all copycats of others, work badly, monetization strategies are spaghetti against the wall...
Product management will be replaced by AI way sooner than engineers.
Wouldn't engineering then be the safest place to be? Engineers have always been embedding loose requirements into runtime law. They (supposedly) understand the business model better than most and have the technical know how to validate that
Yes I think that's correct. But I'm worried less about safety and more about fit. Articles like this convince me that there are some engineers who will struggle to adapt to a world of "no one knows what's going on". Engineers have always had the benefit of making abstraction concrete, through the systemization of abstract business requirements into concrete code that the team would build a shared understanding of over time. But that doesn't seem to be where the industry is trending.
PMs have always lived in the world of dealing with hazy abstraction in both directions: Unclear requirements coming in, turned into unclear system capability whom they have to rely on the engineers to parse. This is where Engineers will have to get comfortable living now: Unclear requirements coming in, unclear code coming out. Its clear that many engineers aren't ready for this, and I don't blame us; it SUCKS. If I had ten dollars for every time I've heard a PM say "no one has any idea what's going on" over the past fifteen years, I wouldn't have to work anymore.
Engineers are probably still the role most suited to adapt to this new world, as you say, but I think people are still vastly underestimating how much they will be personally impacted by the changing industry. If you hate your job now, for reasons like those the article communicates, you'll hate it ten times more in a year.
Understanding code is trivial, with AI support. LLMs are better at summarizing and identifying side effects than writing code. Granted, not perfect, but about as good as a person.
Looking at an architecture, module boundaries, and comparing the organization to the domain is the part LLMs don't do well and is unironically difficult for humans. The less you care about the domain (ie having a mental model), the harder it is.
You could say the same thing about the transition from assembler to C in the 1970s and early-1980s, albeit that was at vastly smaller scale of impact. It’s not that nobody knows _anything_. We still direct the machines, just in a different way. And if what comes out the other side satisfies our needs, does it matter what lies beneath?
Tail risks have always existed in software development. The tail risk of a bug introduced by some dev who quit five years ago is similar to the tail risk of a bug introduced by Claude six months ago. Deal with it by building better visibility into how your systems work. Demand that your agents write good documentation to accompany their code-writing.
If you’re doing it right these days, it means you’re thinking of a much bigger picture and containing downside risks as boldly as you’re expanding the frontier of upside opportunities.
You did not have to know about the assembly most of the time _because_ the assembly output of the compiler would be guaranteed (bugs notwithstanding) to conform to certain rules. Not the case with an LLM
"Nobody is resolving bugs" is weird. Coding agents are great at resolving bugs! So much of what I see posted here seems more about how coding agents are being misused rather than anything inherent to them.
Feel like this has always been a problem, especially with large legacy codebases. Very few people ever knew everything. We just reach the problem faster now; it used to take years of churn and turnover.
> But the final boss is, and always will be, maintainability.
Always has been, always will be.
I am actually hopeful that AI will finally break the industry and force a reckoning around this. Some of it goes to our economic system. New builds are usually capitalizable, flashy, and a great way to get promoted.
Doing ten to fifteen years of thankless maintenance, keeping a critical system alive with high quality? Usually nobody cares, and it's OPEX, not sexy.
I'm not convinced that this matters anymore either. Knowing architecture was a vibe coding strength about three months ago. The modern models seem to handle it very well.
You can't. You need to look at and review at least the core pieces of the code as it's being changed. You need to be able to connect the details to the higher level system architecture and go through the process of asking questions to help discover missing pieces of understanding.
When you simply large amounts of information, the knowledge probably shifts to upper level reasoning, then complex reasoning develops on top of it again. But who knows.
pretty AI-sympathetic vibes in this thread. I generally agree with the author despite everyones opinion (including my own) in this thread that AI can produce great code. Just becuase the code is good doesnt absolve the human-in-the-loop from understanding and clearly documenting and communicating what the system does.
At the end of the day I'm not paying an engineer to send me claude all day. I'm paying someone to become an expert on a system, even if AI assisted. The product will be better if i have someone that deeply understands the system and where to point AI to. Someone that can understand the full big picture can also anticipate future needs - something AI cannot do at all.
I'm in EE/embedded/FPGA work and you can make an absolute hell of a mess with AI in that world, so maybe everyones opinions are more based around front end software or something. I think people forget there are fields that do not have the huge data training base that frontend/backend software does. a large majority of good designs in FPGA are proprietary at big defense corps, not on stackoverflow and github.
If you don't understand how your system works, your ability to make good decisions about future work on that system quickly degrades.
I don't think you need to review every line of code, but you absolutely do need to be able to describe how the system works and its high level structure.
As is so often the case with coding agents, having experience as a tech lead or engineering manager really helps here. You are responsible for a large system that has been worked on by multiple different collaborator (both human and agentic). You need to be able to make smart, informed decisions about that system, and talk with credibility to other stakeholders about what it can and cannot do and sensible next steps for the project.
>I don't think you need to review every line of code, but you absolutely do need to be able to describe how the system works and its high level structure.
I just don't see how you can truly reason about a system without delving into code. Tests aren't enough, running the software isn't enough, high level system architecture isn't enough.
>As is so often the case with coding agents, having experience as a tech lead or engineering manager really helps here ... You need to be able to make smart, informed decisions about that system
In my experience, engineering managers are too detached from the system to accurately reason about the system. Tech leads on the other hand usually can given enough time, but they tend to defer judgement to senior ICs on the team who are more familiar with the code.
Point is: there's no way to have your cake and eat it to. You either read the fucking code and keep a mental model of how the system works in your head, or you have an overstated confidence in your ability to reason about the system (and this has been a problem well before LLMs).
What will the path to a tech lead look like when it's no longer paved by thousands of hours of experience internalizing code? How do future tech leads avoid becoming like people who never got comfortable with fractions because they offloaded all arithmetic to calculators from an early age?
My hope/hunch is that the kids will be alright. Not learning effectively is a choice: if you want to get good, the paths to getting good are all still available to you.
We have never had as abundant a supply of tools to help us learn our craft. I expect that many people will thrive.
People who are a bit lazy and prone to cheating will be able to hurt themselves even more.
I had agents deep diving OpenCode source over the weekend, writing research docs, and helping me with a set of plugins. I learned quite a bit from the process and markdown walls, enough so that in later sessions I was able to point the model at where it was conflating concepts and help narrow its search.
Ai can help you learn if you are intentional about it.
This is the kind of stuff AI-skeptics have warned about daily on this very site for the last year or two. We are met with "you'll be left behind" and "you're just coping" and "the future is coming".
This shit is blatantly obvious. And if people keep pushing and pushing for LLMs to generate things that are actually used directly, the problem will grow so large that most people will barely remember a time when their job was something that could be understood.
This article assumes that this problem did not exist before AI. Especially in large companies like Google, the number of people who actually knew what they were doing was relatively small. The vast majority were just piggybacking on other people's work, which I absolutely hate. At least AI has made this obvious, and the difference between people is now their taste, which, for the majority of people, is bad news.
This! I work for a company that has several systems that are older than 20 years and no one really knows how they work. We are using AI to actually get insights in how they really works, to be able to rewrite and modernize the applications.
I can't read the source code since it's 8 million lines of code and written in a programming language I don't know and in a language I don't speak.
> I can't read the source code since it's 8 million lines of code and written in a programming language I don't know and in a language I don't speak.
^ This is the real world. Some comments talked about how maintainability is king and you just can't keep a mental model together of what the LLM produced. In real life software there is no single person with a mental model of how the system works end to end. In the most ideal scenario you have an architecture diagram, some readme's, and a runbook of how to use the system or get it running in a dev environment. Everything else is manually tracing through mountains of code of wildly varying quality.
coding harnesses are god sent tools when it comes to analysis and maintainability of existing code bases (including code they have produced).
>We’ve had to know everything about the product/business from day 1
I know this is being hyperbolic but I thought this was an odd post to include. I've met plenty of data engineers that don't have great knowledge of the business/product and SWEs that do have that. ¯\_(ツ)_/¯
This problem has existed long before AI became useful. As technology becomes more accessible, less knowledge is required to operate it. As that knowledge becomes less necessary, fewer people acquire it. Eventually the abstraction becomes so effective that entire layers of knowledge disappears from common practice.
As a person who's a solid generalist with over 30 years in various roles, I am completely and utterly shocked at how little foundational knowledge people in "senior" roles possess across a wide variety of technical fields. I'm often treated like some wizard or oracle for knowing things that everybody in the field used to know, I just haven't retired yet.
AI didn't create this phenomenon, it's just the latest (and probably the fastest) iteration of it. I recall another particularly large iteration happened when Windows NT 4.0 Server saw mass adoption. Suddenly, people who were effectively IT technicians were now sysadmins.
And, to be clear, I don't really place any judgments on this.
Unarguably, the world's technical capabilities have increased by this shift (while decreasing the required technical understanding required of the people managing it). Albeit, with some security implications.
The job of a software engineer has always been to turn a real-world idea or problem into a precise specification a computer can execute.
That specification used to be assembler, then higher-level languages like C and Python. Now it might be detailed Markdown files that AI turns into code.
So our understanding moves from the C/Python level to the Markdown specification level. We still need to understand exactly what we're building and why.
The hard part was never translating the spec into machine code. The hard part is defining the right spec. That is still the job of the software engineer.
In some ways I'm sympathetic to this: after all I'm trying to make money by being that person who does know stuff. But honestly that capability never seemed to be terribly valued by the people with the money. Anyway, it would be nice to see some examples where AI tools didn't "know" about architecture or intent. My experience has been they know plenty. More than most human developers. They do tend to follow certain fashions, which can be overridden by debate, or by introducing new fashion via skill text. But that's today. Surely these tools are only going to get much better in the next 10-20 years. I'm skeptical there's going to be a line of thinking that "it's smart but doesn't know the things humans know" long term. Whatever it is that we think only humans know today will be learned by the tools tomorrow.
"People are working 12 to 13 hours a day just to press enter. Nobody is reading anything."
Really? Seems like they're not making proper use of the tool. I've been reading MORE not less, and also learning more along the way. Just hitting "enter" is a choice, these tools are so powerful if you invest your curiosity, time and experience.
Sounds like they don't care about what they're doing in the first place, writing code by hand won't fix that.
This is almost the same as saying "The problem is not the fentanyl, but that everyone keeps getting addicted or dying." In a very literal sense, yes, absolutely. There is a way to safely use fentanyl. But from a holistic point of view, the way to do so is so specific (ie multiple medical professionals prescribing and possibly even administering and monitoring vitals afterwards) and limited that for the general public is much more useful to just say "It's unsafe to use fentanyl unless following instructions from your medical provider." There's absolutely a way to use AI for coding that doesn't result in not actually knowing about the produced code, but it's so damn hard to get that right and you won't even realize you've fallen away from that knowledge until it's too late.
The point about PMs is I think illustrative of this issue. Yes, great PMs who actually understand the product they want to build and can then turn that into something using engineering teams as a black box system exist. But by and large, no they don't. A massive part of why waterfall fell out of fashion is because it turns out most orgs don't have and can't find PMs/PM adjacent people who can build that knowledge, and you can see how agile attempts to correct for this by turning a pipeline into an OODA loop. If you can prevent pipeline flushes, it will beat that OODA loop, but it turns out as an industry we can't actually prevent enough pipeline flushes for that to be the case and we can (for the most part) use that OODA loop to eventually iterate into something a paying customer might give us money for.
Part of the issue is that we're treating the AI like it's just another compiler or static code generator. It's not. The reason they're not is because they require a complete specification of what to output, ie the code. That level of specificity is what allows engineers to mostly stop caring about the assembler and just stay in their higher level code (the main exception being extremely hot loops in places too complex for the compiler to optimize well) and legitimately be able to say they understand the code without ever looking at the compiler/linker output. The entire point of AI for code generation is taking an incomplete specification and turning it into a completed one. If you review and understand every line of output the way almost no one did even before AI was a thing, then you're not going to fall into the trap of not knowing anything about your system anymore. But at the same time, you're not really going to get much benefit from AI either, because you're just replacing time spent coding with time spent reviewing code. It might even take you longer to review than it would to just code it yourself (i think every senior engineer who's mentored a junior dev knows this exact feeling).
To actually get the benefit from AI you have to let go of looking at the code output at all, because looking at that output is the slow part and because just looking and reviewing still won't give you the same knowledge as actually implementing. The passenger aviation industry handles the second issue by preferring/requiring manual control in critical areas (takeoffs and landing) and using simulators to extensively train for manual recovery outside of the abilities of autopilot. For developers the second bit is actually why reviewing AI output is a fool's errand - it will give you the false confidence in your understanding of the system. That doesn't mean we have to embrace vibe-coding or accept slop. It means changing the engineering process from one that deals with a deterministic system to a stochastic one. As an aside, I think this is why upper level managers and executive are leaning into AI code generation so hard - engineering was already a stochastic black box system to them.
A man has to write program for himself and a man has to use the program. A man needs to define in code what the program does. If AI defines what code does then man has not written program for himself.
Too many young people in startups without enough experience.
I was in a Hackathon for students which quite a few staff, like myself, infiltrated. The results were completely unfair, staff and teams with staff (like mine) cleaned up the awards.
In my case I was working with a student who was much better at writing platformers in Unity than I was and an another student who could draw the art we needed even if she'd been trained to think every problem we had interacting with each other had something to do with "the patriarchy".
Myself I'd been in many startups where the game was make a half-baked demo that you could demo on stage and get people excited about it. So everything from presenting broken software on stage and making it look not just perfect but enticing and developing software that has the qualities it takes to present it that way was routine for me, the bit that isn't routine is onboarding unexperienced people to this life in two days.
The more things are unprecedented, the more you need a longer view with more experience.
I was reflecting on this on Saturday in an unformed way, trying to trace the lineage of a decision made at work.
The code change itself doesn't specifically matter. But suffice to say, it was about an AI feature in one of our products.
The code was stamped by Claude driven by a prompt. The prompt was for a ticket generated with the Atlassian AI integration. Atlassian had digested docs made with AI. The docs came from strategy memos I'm 90% sure were written entirely by Claude.
The strategy was chosen by management at the urging of exec leadership. The execs now communicate mostly via AI written memos. I do not know how they make decisions, but they reference tech influencers, market conditions, customer expectations.
This gave me pause. Who had actually made the decision then? Arguably there has been several layers of human review, but the actual source of the decision was hard to pin down.
We were not building the feature because we wanted it. We were building it because we thought other people expected it.
Perhaps reflecting on the state of the market, I thought, could indicate who was actually in control.
Where do investor and customer expectations come from in 2026? It is very murky, at least in tech. There appears to be hype. Some hype comes from true believers, some comes from cynics. But both respond to market incentives that reward bigger and bigger claims.
Where does the market's "action" come from? What is the driver?
Investors do not really seem to understand what the tech is or its limitations. Some are passive operators. Others are just responding to the overall froth and speculation in the market - which becomes a runaway feedback cycle.
This left me lost.
Nobody in this ecosystem, I thought, is actually in control here.
Nobody is actually orienting work and action to real, concrete goals. It's all based on speculation and anxiety about the future.
So it is not only that nobody understands what the code does. It is that we cannot, or at least I cannot, explain the motivation. There doesn't seem to "be" any form of "intention" in this environment.
It has all been hollowed out, replaced either be inscrutable machines, or inscrutable incentives.
Ironically it rather resembles the kind of "misaligned" superintelligence we are supposed to be avoiding.
In many (most perhaps, but hard to know personally) companies, most people don't really understand the market they are in, the competition, their own products, the customers etc well enough to actually make good decisions. They also aren't likely to be around (or held responsible) for decisions as they play out over years. So what has historically happened is that people use proxies for good decisions and understanding - which are clever sounding documents and presentations.
This is a long winded way of saying "people made up clever/sensible sounding stuff". Now it's easier to do it with AI so the problem is worse. However, I'm not sure what you were looking for was ever really there - the "inscrutable machines" and "inscrutable incentives" were always quite inscrutable.
> most people don't really understand the market they are in, the competition, their own products, the customers etc well enough to actually make good decisions
"Product" studies this data and tells engineers what features they want. I am rarely instructed to A/B anything...excepting when it's defensive, to ensure the first rule of "don't interrupt the flow of money" is upheld, if at all. Most of the features are obvious improvements anyway (determined from plain conceptual planning, manual testing, and personal usage).
ofc I don't understand the customer or market or anything else. I'm paid with the expectation that I'm not going to share an opinion about it, especially since am never exposed to the raw data and inner circle decisioning except during a quarterly meeting...maybe. This is part of why AI is so successful. Humans in large organizations are specialized with little creative input and lots of mechanical process. Coding is largely a mechanical black box (turing machine) from the outside looking in. It works seamlessly because I don't need to know about the things product wants, so the AI doesn't know and everyone carries on faster than we were before.
Yes, people can be bad at their jobs. Yes, they can work in an industry and not know anything about the market but I refuse to believe that most people in most companies are just stumbling around faking it. I've never worked in a place like that. People like this exist but there are only a handful of people like this at every job.
The hyperbole feels like it's been contrived to fit into the classic AI counterpoint: But humans do this too!
It's more nuanced than that - I never said faking it, people usually believe what they write in those documents/ppts. And they do actually sound clever. Next time you see a decision of any magnitude, especially something which sounds "strategic" - trace the decision back as far as you can, consider the question from several angles, and really question it. You will see gaping holes, decisions made using "frameworks", anecdotes, hopes and dreams.
I can't argue that people don't always consider every possible angle and nth-order consequence to a decision before making it. AI doesn't do this either. Also, this seems different than your original position.
It still sounds like what you're saying is people are bad at decision making so AI isn't doing that much worse than people. That's very bizarre thinking because it opens the door to AI making decisions for us.
I'm saying something even worse - humans are bad at decision making, we usually don't even most of the context required to make good decisions so we use heuristics to make decisions, and proxies for "good decision making".
If the above appears to be bizarre thinking, take a couple extra laps around the block.
Now I'm not saying AI is better, in fact above I mention that it's worse - but don't hold onto some nostalgic idea that decisions were made by people with lots of context thinking rationally and incentivized properly.
It's not counterpoint, it's a fact. Can you say with your hand on your heart that the typical user experience was acceptable before AI arrived?
The reply was always "It's hard, and there are always going to be bugs."
But it's not just about bugs. It's about systems that are hard to use, poorly designed, and opaque. And sometimes the opacity is deliberate. There are dark patterns, outright lies about what happens to data, and more or less obvious grifts.
Was any of this truly great before AI arrived?
Is it an accident that you can't kill an MS365 subscription if you have more than X amount of GB on OneDrive, but the actual usage includes all your spam and email attachments, and it doesn't show up unless you know where to find it, and the location is very non-obvious, and so is the cleanup and deletion process?
Or that the eBay fee structure for international sales is utterly incomprehensible without automated help?
Or that if you select Subscribe and Save on Amazon your cancellation date is something like two weeks before the next delivery?
Or that if you sign up to Discord you need a mobile and an email for activation, but the site doesn't tell you this, and nor do tech support, who insist that a mobile is optional?
And software quality - services down, records lost, records stolen, photos and documents deleted - is a whole other layer on top of that.
AI is a moderately good solution to the first set of problems, because it can search and assemble information far more quickly than you can. So IME it's pretty damn good at tech support - not perfect, but better than DIY in many cases.
Software quality? We'll see what happens. If tech stacks collapse over the next couple of years we'll know AI was terrible thing.
I'm in the 'Too early to tell' camp. There's a fair chance they might. But if so it will be because of poor testing and design, not because of code review or lack of hand-rolled code. And it's not completely clear that quality levels wouldn't have dropped anyway.
AI is here and things don't seem like they are getting any better any faster than they were before. Some people get it wrong and some people get it right...just like before.
Also, you're completely missing my point. The author is talking about his company inadvertantly offloading decision-making to AI. The comment I was responding to was like "Well, it won't do that much worse than people". That type of thinking is a little dangerous to the future of our species.
> Can you say with your hand on your heart that the typical user experience was acceptable before AI arrived?
I remember using Windows XP (SP3) and Windows 7, Photoshop 7 and CS3, Office 2007, Blender, Winamp, Linuxmint and my computer was a joy to use (with HDD and slow ram)
The internet was mostly a repository of knowledge and communication. I was online maybe 30 minutes every few days. This was around 2010.
From my experience in big and medium sized companies or even startups I would say that this was already the state of things before AI. Most leaders and top management were following internal committees, that were following consultants recommendations, that were themselves recommending what others were doing and was hype and/or following Gartner like "studies". The motivation was equally dubious, and many developers already had problem with management and product direction, but many didn’t bother to question it. AI just makes it obvious.
To put a finger on a word: David Hume’s “is vs ought” problem.
Data can describe to you what exists. But it can’t tell you what you value.
What you describe is people who can’t tell the difference, and who let the machine (data) make the value judgments.
I don't think the problem OP has is anything so metaphysical as the fact/value distinction. This is a business, after all. It has goals (e.g., making money) that the model surely can grasp. It sounds to me more like a breakdown of organizational control owing to a lack of transparency in the tools and a general ignorance among the management.
Imo, most companies ought not to exist.
The parent comment is interesting, but ultimately I think in this case, the AI is actually revealing something about the true nature about their place of employment, a nature that has always been there versus some mutation caused by the prevalence of the AI itself.
A lot of money can be made purely algorithmically. Think market markers or other algorithmic trading. The ought vs is divide is quite narrow here. It's not a moral question, the "value" is in the money to be made. It's actually not a great example to invoke Hume's problem. Many companies essentially are chasing a similar spread, its just less obvious. Few people ever ask what "ought" to exist. If the power of AI makes more businesses operate more reactively and algorithmically, because of more data or processing power or w/e that really is probably in keeping with their alignment and goals. Because the ultimate ought for a company is we ought to be making more money. So, in many ways the ought is really not that interesting, the is is satisfactory provided the return on whatever their version of a spread is keeps improving.
The number of companies that actually "invent" useful things and thus ask even vaguely meaningful "ought" questions are extremely slim. The vast majority of employees are, at best, accessories to these questions, even in software where even before AI many of us were not doing very interesting work. There is a lot of essentially rebuilding your competitors same layers on top of common libraries and standards where the actual interesting work is done. Really not unlike asking AI to cook you up a boilerplate by leveraging the vast work of a fraction of SWEs who maintain OSS tools. It's the same pattern and the same sort of behavior, just now your "layering" is becoming automated to the point of irrelevance.
What the parent misses - the real promise of AI is paradoxically, that it will allow more people to ask actually interesting ought questions as AI owns more of the spreads. In the same way a human does not compete with an algorithmic trader, and at some level, really doesn't care. The more algorithmic your business becomes, the less any individual human "value judgement" matters. And really this is desirable, because again, most companies are not asking interesting value questions anyway. The end state of this you are missing is these companies are going to cease to exist. In the optimistic case this will free you up to ask more interesting value questions - like how do I value all my UBI enabled free time. In the less optimistic case your value judgments will be more dire - like who do I sacrifice given the Terminators are at the door and we only have x quantity of supplies left.
This is the other paradox. When questions of what ought to happen are of paramount importance, you are probably finding yourself in a very undesirable situation. It's easy to valorize the ought problem from a distance, it is much much harder to actually engage with it when it actually matters. In many ways, the relative luxuries of society and civilization are derived from taking such questions out of most of our hands. This is (perhaps surprisingly) true even as you climb the ladder of power:
I used to think that if there was reincarnation, I wanted to come back as the President or the Pope or as a .400 baseball hitter. But now I would like to come back as the bond market.
For me, at this company, which is struggling to develop a new value proposition, everyone is unfortunately using AI in a blatant way. It’s even worse when executives and management pressure us to move fast. Presentations, documents, and mockups all use AI, rinse and repeat, feeding it context, but somehow, nothing is moving. It’s just staying static.
The word I have for this is “commitment laundering”. People pass around AI artifacts that nobody has necessarily read or considered, and the invented assumptions and tagalong commitments just keep piling up. They look like they’ve got real provenance but nobody can say what is being done or why.
Right up there with responsibility/culpability/liability laundering. These are things being shrugged off and externalized.
And the complement is credit/provenance laundering. These are things being misappropriated.
The grift often does both of these with the same sleight of hand, and this is what gets accelerated with the new tools and cavalier culture around everything.
> The execs now communicate mostly via AI written memos
It bothers me to no end when I get an AI written response, especially from the executive team or any one of my co-workers
Professors in grad school are also simply replying with chatgpt answers to any and all questions, including reviewing papers before they're submitted. I've learned to simply cut the middle man and ask the AI myself, though i cannot afford the pro subscription.
You could tell them: “ChatGPT found 3 minor issues with this paper, that I have since addressed: issue a, b and c.
Most people are trend followers and what trends they see decides what trends they follow. It sounds like the algorithms that chose which influencers to promote made the decision, which was already somewhat true before the current AI trend. This happened when there was a big push from deterministic feeds to algorithmically ranked feeds by facebook and twitter. I guess an argument could be made that the people who chose what content to consume feed the algorithm but really there is a heavy hand on the scales that tilts the algorithm in favor of various things.
Anecdotally, I know people in the industry who tell me their job is "so easy" now because all they need to do is be a meat proxy for claude and take home the paycheck.
Question from someone written in AI? Just answer it in AI and send it back. What was it actually about - who cares? Bug comes in? Post the jira link in claude and don't even bother prompting anything else. If something critically breaks - well, eh, we'll deal with it then. FIRE (early retirement), a prediction their layoff is inevitable, and investing aggressively so you can finally escape actually working are often invoked in the same breath. Everyone feels like they're just trying to punch the drywall and grab as much copper wire out of the walls as they can until they're finally let go and/or the whole company fails.
There's a great deal of nihilism and cynicism in the industry currently, and it feels like LLMs are just greatly enabling it. Where you would've done a halfassed job previously, you'd now do an unchecked AI job.
> Everyone feels like they're just trying to punch the drywall and grab as much copper wire out of the walls as they can until they're finally let go and/or the whole company fails
That’s very graphic.
The thing is, I already felt a bit like that before AI. It’s just money. This quarter’s. The rest doesn’t exist. Make flashy features faster and you’ll be promoted. Make things carefully so that they last and can be maintained easily, and you are be ignored.
I say this not as an engineer that tried to write maintainable software and now is butthurt, but as an (ex)manager who tried to promote such people. And now is butthurt. I found myself telling my engineers that if they wanted to get promotions they had to prioritize the shiny and skip the rest.
I hated it.
Now I am back to Engineering. I do use AI heavily at work because no one cares but if my output is “too slow” I will look bad. I at least try to raise concerns when I see them. The answer tends to be “yeah, we can’t afford to do that properly now”.
I use AI way more sparingly for my personal open source stuff. Because I care.
> a prediction their layoff is inevitable
It probably is. Where I work it has been made clear, as in actually stated by leadership, that any process that does not include AI input is to be considered broken and needing to be fixed. It doesn't matter if it works, if it is 100% human made and maintained, it is broken and needs to be fixed by injecting AI in a critical place. You should be in a position that if you lost access to the AI tools, you are unable to proceed.
This is the step towards replacing people. You don't need highly paid people in that process, you just need someone who can speak a language the AI tools can transcribe. This is no different than moving from a codebase or process that is tightly coupled to a specific technology to a more generic process so that you can easily and quickly change the backend tech on a whim. The technology being removed is the people. The AI is important, you are not.
Well it's the incentive given. A bit like the original comment writes above. If your performance and impact are measured by charts, meeting outputs, some kind of abstract KPIs and other means that traditionally kind of worked because people had agency, you are incentivized to just slopmaxx it. Otherwise you will fall behind your colleagues who slopmaxx and have a better number on their abstract KPI.
If you try to do good work, you won't be able to keep the pace with the slopmaxxers, which will mean you get laid off earlier. You will also be swamped in slop review.
If you get called out on some issue or shitty implementation, you can just make Claude abstract it away behind more complexity to the point where people have a hard time doubting you because they don't have time to get into the details and verify things.
Grab as much as you can, invest in immovable property and other shit that has value after a stock crash and enjoy the ride
> prediction their layoff is inevitable, and investing aggressively
If that happens on a large enough scale those investments won’t be worth much.
The problem with the tragedy of the commons is that everyone knowing it will happen is not enough to prevent it from happening.
One of the more thoughtful observation's I've seen in this "space," one which offers an unsettling counterpoint to the claim in the original post that "AI Can't Drive Itself."
To your point, AI can drive itself; it's just that the fashion in which control occurs is distributed. Which is to say, the process (e.g. a feature being implemented, in some way, or at all) is not spontaneously occurring: it's emergent from the mesh of AI automation.
Niceties aside, it's pretty clear that humans are not in the loop in a meaningful way, much of the time. What emerges may hence not well be not aligned with business or technical needs. What it is aligned to may be impossible for we humans to discern.
Who controls the way a forest grows?
Ask a tree, get an answer, but don't forget, that's not the answer.
This resonates, but also, I'm not convinced anyone can truly understand the workings of any society, or market economy, or what have you.
Who's in control? Everyone is, to some extent. And no one is: when you're hungry for food, are "you" in control of that? You can consciously repress your impulses to go eat something, but your mind didn't create those impulses.
Human societies develop impulses and minds of their own, emerging (weakly) from the impulses and minds that comprise them, and they make decisions in mysterious ways.
Of course, it sure is nice when we can come up with a compelling story for the motivations behind something. Easier said than done…
> Nobody is actually orienting work and action to real, concrete goals. It's all based on speculation and anxiety about the future.
the simulation has become simulacra! cosmic horrors abound.
I just today found a term to describe what is happening: gradual disempowerment. I think it's a likely path. Everybody, because of competition, needs to end up giving control to AI.
Very good post, thanks! You remind me of the countless times I've heard "we need to be AI native", "use AI", "pass it through AI for review".
AI has become the thing you do, and what you do it with, to achieve it. It's a self-fulfilling chicken that is an egg that is a chicken.
> This gave me pause. Who had actually made the decision then? Arguably there has been several layers of human review, but the actual source of the decision was hard to pin down.
Cynically, this might be the real reason managers and investors love AI. It diffuses responsibility. No one is accountable. "Oops the AI messed it up" is a convenient excuse for bad management.
I see this is code in a smaller level. It lands on an alternative but in the end nobody actually chose it, not even by personal taste.
Yet everyone assumes that, like before, there must be a reason.
Worse, we can't tell apart real decisions choices from the dream-machine side effects.
The invisible hand of the market is in control, and we don't understand it either.
You could look at it as a kind of evolutionary pressure, where the AI driven companies that have no tether to reality are eventually extinct. Companies that are able to get a handle on the direction they are being dragged may have a fighting chance of survival.
That is a great example, and kind of frightening how once the market decides to start pumping money into something our world morphs to meet the flow of cash.
AI is definitely impressive but are LLMs the true path to AGI? Is AGI the path to a better life/world? These tools are really impressive but it would be nice if they did something actually useful. Can a rogue OpenAI agent draw investor attention by curing a disease instead of hacking into the CIA?
Ultimately competitive pressure is what drive decisions.
> Ironically it rather resembles the kind of "misaligned" superintelligence we are supposed to be avoiding.
yes, and it is the exact same system that is producing the "misaligned" superintelligence. Funny how that works, and begs the question: exactly how are you supposed to avoid building "misaligned" superintelligence?
At some point deferring all decisions to AI will be the competitive thing to do, regardless if its aligned or not.
> The code was stamped by Claude driven by a prompt. The prompt was for a ticket generated with the Atlassian AI integration. Atlassian had digested docs made with AI. The docs came from strategy memos I'm 90% sure were written entirely by Claude
Was there never a developer in the loop? I hope my org won't give up control of their business to an AI like this soon, sounds like a nightmare to figure out what's going on.
It's like old fashioned Wordpress, edited by someone directly in production, on a shared host that probably is infested with who knows what, with little to no oversight except inscrutable emails, which in the old days were like a CEO of a small business emailing 'hey, the thing we spoke about on the phone, is it coming?' and no actual document trail.
Except for the enterprise customer and at enterprise prices.
> Nobody in this ecosystem, I thought, is actually in control here.
Profound, your comment provoked this thought: we know that AI in organizations is disruptive, but we still don’t understand some of the shifting patterns of power, communications, agency, etc.
If AI is not well-suited for traditional models of stratified management then maybe we need different models of bureaucracy?
Thank you for taking the time to write this. Unfortunately, there are still people on this forum who argue that a turtles-all-the-way-down approach will somehow work out in the end and that the best way to verify LLM work is to add another layer of AI agents, whereas the only way to stop it is to ensure that the entire thing is built on some human verified last line of defense. Unfortunately, the higher up you go the more insecure people get when you ask for their supporting sources. They'll try to hand wave it saying they researched it or got others to research it for them instead of owning up to deferring the decision to AI. You know whats even scarier? When a C-suite doesn't try to hide the fact that they had the AI make the decision. Then it's just time to leave, or alternatively time to turn your brain off and collect your pay check until the music stops.
>Perhaps reflecting on the state of the market, I thought, could indicate who was actually in control. Where do investor and customer expectations come from in 2026?
This looks like ill-fated Gartner driven development on crack. Where does customer expectations come from in 2026? How about from your customers?
If you're doing enterprise software and talk to your users, you'll end up learning that the actual users of your software don't use 80% of your features. You might even learn they don't use your software at all, and that the software was mandated top-down from the C-suites or that the execs in charge forgot what purpose your software served but are too insecure to question it, unless and until there is a sudden pressure to cut costs.
So AI is already in control with humans just rubber stamping the output?
Exactly the same issues we are facing right now. And, well why wouldn't it end up like this? The promises of going 'faster' while ignoring the organisational processes capability to handle it is - and continues to be - the greatest delusion of this new age. But, even in the face of all of this, we are seeing leaders asking for more. At some point the culture will collapse in on itself, and no one will understand anything anymore.
So it’s tulips all the way down?
Actually reminds me a lot of 'Cube'
> there is no conspiracy. Nobody is in charge. It's a headless blunder operating under the illusion of a master plan.
this is a good summary of the problems of AI decision making, but its assuming corporate decision making before was any better.. its probably about the same
ie we have companies making decisions just as badly before AI as with AI..
You need human vision and leadership to move an org.. AI is just a tool at this point
This is why we are building Origin (originhq.com). Having a record of every prompt, tool call and model response enterprise wide allows you to answer these kinds of questions.
No human will read that though.
It is for execs
When you write something you constantly remodel your understanding through refactors and rewrites until you internalize it. By internalizing it you gain the capacity to reason about it (during critical downtime) and communicate it. An entire team that can communicate can solve problems together, from one guy's vision to products white boarding to engineering's infrastructure to UX and UI's artistry.
It boggles me we completely forgot that the world operated like this just 4 years ago
I am simultaneously deep down in the agentic treadmill and also supremely frustrated by this.
And I don't think it was ever necessary to go to the point where people just gave up authorship. These were choices made by adopting the "I'll do everything for you" agentic "harness" model that shipped with Claude Code but it was never inevitable.
e.g. we completely dropped fill-in-the-middle completion OG CoPilot auto completion model. That combined AI authorship with a human always in the mix and I actually really enjoyed it. It's just that the models involved were pretty stupid. We totally could have had IDE / shell / tooling integration that kept people in the driver's seat while automating parts of the drudgery away. Instead what we got was a simple chat loop with "oh, whatever, you go do it" being the ultimate result. Cuz, you'll totally review everything after and understand it, right?
The things should end by quizzing you on what was just made and if you don't pass, just throw it away. That'd be funny to watch.
Our brains are hard-wired to enjoy (at least in the short term) immediate gratification, and the removal of friction. It's a hard battle to fight.
Have you ever asked a long distance runner how are they feeling during a run?
Long distance runner here, feels amazing from start to finish. Ok, amazing maybe only during runners high, but definitely good all the way through. Running sucks until it doesn't.
Running (especially long distance) is both immediate and long term satisfaction to me. Exhausting but very enjoyable.
Imagine if you will two product proposals coming across Dario's desk.
One is:
1) Complex IDE integration where our AI model is called to assist users in making decisions, assisting them to do the things they already intended to do. It appears more as a function of the IDE rather than something separate. It shows off the intelligence of the model but doesn't show off any autonomy.
2) A separate tool that can be branded Anthropic. It takes over completely. It advances the narrative that we've already been spreading that AI is going to make some human jobs obsolete. It looks to employers (the actual people who spend money) like it replaces an expensive developer even if at first it requires one. It has minimal to no integration point.
Which do you think they'll approve. I think it has less to do with the user motivation and more with the producer's.
I feel this in my bones. My team is one of the most communicative in our org and have garnered a widely positive reputation I think largely as a result of our coming together every day to solve problems. Trying to get information out of the less communicative groups is like pulling teeth, i wonder how they get anything done at all
Assuming we don't speak about some vibed "one-shot-deploy" situation (for which there were options in the old world as well, with the same bad results), in a well disciplined team, why would AI make this worse?
You can still write anything by doing it either small steps, or at once followed by a lot of refactors while skimming over code and asking tons of questions / making refinements via prompts, guidelines, test guardrails. A team can still reason and whitepaper about the same things. Devs who were previously shy to ask some specific details (to not seem dumb) can now confidently survey big codebases and get insights in whatever style they can swallow.
The bigger problem I see is that all this requires a lot of communication, and most importantly writing skills, something that disappears in thin air in the last decades.
My experience is the AI has somehow thrown discipline and accountability out the door.
Folks break everything all the time and looking at their PRs and messages are clearly just having the AI respond to everything for them and actually have no idea what they're doing.
And then other people doing the same thing approve their work (presumably not reading or understanding any of it) because then the other person will do the same for theirs.
Hopefully my experience is an anomaly though :)
The way I think about it is that understanding is formed in a top-down fashion now, instead of bottom-up. You can look at a feature developed by AI from the outside, and keep peeling the layers and examining them (or having AI explain them to you) until you learn how it works. It's like reading a textbook. It is different than writing the code yourself, or practicing the topic of the textbook yourself. But if you invest the time, it can be just as effective.
The issue of course is that if you do invest the time, then you're no longer saving time by using AI. You're just spending it reading and trying to understand something you didn't write. And that can be unpleasant in its own way.
My hot take is that for parts of a system that can be considered its core, forming a deep understanding is almost always important, and so is knowing how the different business domains integrate and where the connection points are. For many others, a high level understanding is sufficient. The difference is that now, with AI, you can make that choice. Before, you had to write everything yourself, and for any sufficiently complex and long-lived system it became impossible to hold all of it in your head.
I disagree that reading a textbook is just as effective as solving problems yourself (this is what i read your statement to mean). It seems fairly obvious from university experience that doing the homework yourself gets you better test grades in the end. You can of course have the AI write the homework but you can't then just watch the AI solve it lest it be drastically muted in effect. Trying things that fail, working hard towards a problem, banging your head against something impossible, these are all worthwhile endeavors. Of course reading the textbook is useful too, very useful! But the textbook does not teach you experience.
> Trying things that fail, working hard towards a problem, banging your head against something impossible, these are all worthwhile endeavors.
I believe that this is the most important thing... and something that people are afraid of. I've got dozens of personal projects and more than a few branches in my employers repo of things that didn't work... things that I tried, figured out it wasn't going anywhere and went to try some other approach.
I suspect that there's a bit of sunk cost fallacy going on elsewhere. "If I don't know how to do it, I'm not even going to try" and "I got this far, I'll keep doing it despite it being wrong."
As a programmer, I am often disappointed at the lack of curiosity in the language and how things could be done. My example would be people writing Java code as if it was still Java 7 - no streams, no Optionals... The fear of having to go back and do it again if using something new doesn't work they'll be in a worse position than if they did it the old way and not realizing that learning what doesn't work or seeing how things that didn't work in this situation may be the right thing for some other future problem.
Sure, but what level of understanding do you actually need? Only you can be the judge of that obviously, but I posit that for the vast, perhaps even the overwhelming majority of any system's components, a high level understanding is sufficient. In fact that's how it already works: it is rarely the case that one person knows the team's entire codebase inside out. Instead different people specialize in different areas. With AI it's the same thing: you can achieve the level of understanding you want with the most important parts (either by having AI explain it to you, or architecting it and writing the code yourself), and delegate everything else to AI.
There are still some places that operate that way, as outlined in this Jane Street talk https://www.youtube.com/watch?v=zR9PpXWsKFQ (Production Engineering When Trading Billions of Dollars a Day).
> capacity to reason about it (during critical downtime) and communicate it
That's one of the things that's put into limelight in that talk.
I personally think that most of "Web Scale" software is inconsequential (inconsequential for its creators, not for users) - as there are no consequences for bugs and outages at all. A data breach -> slap on a wrist. Reputational damage because on an outage/data loss amidst general public? - almost impossible (clownstrike, anyone?)
The funny thing is that web scale software that's consequential is often in an ethically grey zone - but at least you won't be surrounded by colleagues who don't give a shit.
> When you write something you constantly remodel your understanding through refactors and rewrites until you internalize it.
Unfortunately I learned that not everybody thinks this way. Some orgs do imperfectly fine without good engineering discipline, and that has been the case before AI... AI has only made it easier to give the appearance of good engineering, which is exactly the pre-AI goal of many orgs
> It boggles me we completely forgot that the world operated like this just 4 years ago
I'd think that's what they call paradigm shift, and this probably repeated across generations from the introduction of the printing press, PC, the wheel, the internet to stochastic parrots that reduced what we still stubbornly insist require our special neurons to mere statistical modelling that can be aggressively scaled.
As part of a reorg, my team received a new system to maintain. I sat down with a great engineer and together we refactored that shit to get into something more maintainable. I became the go to guy for that system because our redactor led to such a deep understanding of the codebase.
Now in the age of AI... we adopted another system from a team and when we asked for knowledge sharing to prep for the handover the answer we got was "do we still do that? Just ask Claude".
It's now been a few months. We've shipped features in this new system. I still have no idea how it works. Okay I kind of know, but only at a very shallow level.
AI is the worst thing to happen to our industry.
I am finding this, professionally, not so dramatic - but mostly because the industries in which I've consistently shipped code at scale involve review as a first principle, as in no un-tested, un-reviewed code gets shipped, because: safety critical/realtime requirements and certification specs, say so.
And having AI code to review is no different than any other code that ever was to review, so the review tooling is - as it necessitates - also benefited by lugubrious application of .. more AI. But: all AI is human reviewed.
So it's not a big impact. We just don't ship code that isn't 100% human reviewed, If that's insurmountable: you're doing it wrong. Use AI to make code readable again.
And then, also, put AI back in its box. Don't give devs 100% full-time API access to subscriptions: give them actual hardware to use, to go 100% local.
Local AI is, thus, the best AI, folks. Don't use more than you can run locally, is a great way to keep AI code properly maintainable.
The industry will prove this, itself, sooner or later: If you can't put your AI in its box for safe-keeping, you're doing it wrong, anyway... and should've already learned this practice as a habit, decades ago, vis a vis future-proof tooling... (See also: not logging everything you do with an AI? Big fail.)
Sure, the absolutely intoxicating addiction of Big Metal AI™ is going to put a lot of consumers in a deep, deep pit of Neo-Illiteracy - however: 'good' AI code is actually just good code.
> I am finding this, professionally, not so dramatic - but mostly because the industries in which I've consistently shipped code at scale involve review as a first principle, as in no un-tested, un-reviewed code gets shipped,
> And having AI code to review is no different than any other code that ever was to review
If your company is sticking to "everything needs (human) review" then you wont run into most of this. This issue is that a lot of companies are using AI as an excuse to remove that review process (either partially or entirely).
Yes, that is very real to me, the AI "revolution" is more of a battle between traditional devs trying to replace their managers with AI, and professional managers trying to get rid of the devs the same way. There is a wide degree and scope of knicker-twister'ing in these trenches.
Everyone can write code these days. Trouble is, all code has to be SIL-4 code now, because, human, you will never know if your compiler trusts your AI until you trust your compiler. This rule will be true for decades into the future, I'm willing to wager...
But, ultimately, software has to follow certain rules, or it just doesn't work. Proper workflows - involving review - are needed. Because security is pretty much over, otherwise.
Folks are finding it easier to make their own software now, too - rather than use others. I predict an end to the app stores - or at least, the primary interface is going to end up being "describe the app you want to use today" instead of picking words from a list ..
> And then, also, put AI back in its box. Don't give devs 100% full-time API access to subscriptions: give them actual hardware to use, to go 100% local. Local AI is, thus, the best AI, folks. Don't use more than you can run locally, is a great way to keep AI code properly maintainable.
This glosses over the large capability app between local options and frontier options. Which applies in many ways, but includes drafting more maintainable code. I would definitely not want to limit my devs to what we can run on our own hardware.
The thing is, fully-test-your-code wasn't economical for software houses before the rise of AI. Perhaps it will become economical now but I'm not sure what would force the issue.
I see this everyday. The problem is code is the wrong abstraction for the work we do. LLMs have solved coding, but they haven't solved systems, collaboration or system maintenance.
Edit: Since I seem to have touched a nerve - I've been working on a project to solve this: https://www.archme.io if you want to know my thoughts on the right abstraction
When you say "solved coding," what does this mean, what does it look like?
I have strong disagreement because it sounds like, by analogy or proxy, we have also "solved writing"
I would say it means you are building software but no longer are concerned with writing or editing literal code. Your concerns have moved up the stack to managing requirements, context, and verification processes.
Just to make this clear: if you can define a really good PRD and sophisticated technical specs, and a strong set of tests cases to pass, at the right level of architectural granularity, plus adversarial code review processes that triangulate and weed out most mistakes, SOTA agents can write the code autonomously, at or above the quality level of most human coding teams. I call that "solved" but only if you meet those context requirements. Which is still hard, not solved, at that layer.
Solving writing is not a good analogy IMO. Writing is for human consumption, and cannot be wrapped in objective requirements and verification processes. Certain forms of writing perhaps could be (can't think of one at the moment but I don't doubt some exist), and those forms might be good analogies for being "solvable" or "solved."
I agree with you, but it's a moot point for software engineering.
> Your concerns have moved up the stack to managing requirements, context, and verification processes.
This has always been the concern. "Add oauth to this app, there are no requirements beyond oauth working" has been an intern level task for ages. What makes software engineering hard is when the requirements start adding "well it has to use this oauth backend that isn't technically spec compliant, and the user is going to send some kind of random token you need to translate to oauth, and...". The problem isn't in writing code that will do the thing, it's figuring out exactly how that backend isn't oauth compliant and what chain of API calls I have to make to convert their random token into an oauth one, and etc.
Producing software that complies with a test suite isn't really novel. You've been able to outsource that forever. This falls apart in the same places outsourcing does; I'm sure India/Phillipines/etc/ is more than capable of iterating on code until it passes a test suite.
> no longer are concerned with writing or editing literal code
I'm not typing keys, but I am very much still concerned about the quality and nature of the code. Coding to me is more than pushing keys
> if you can define a really good PRD and sophisticated technical specs
I still believe we cannot waterfall software, the idea seems like taking a step backwards. How often do we learn about an unforeseen complexity only after getting into the implementation?
In my experience with agents, it's better to be iterative and in-the-loop. Start with a decent description, have them research the code/issue, write up an initial plan/design, work iteratively on writing code and updating design doc, review and finalize the code and markdown. Then future agents will have some resources to shortcut understanding the code base.
I'm not talking about not just punching the keyboard to type out the identifiers in the code. I'm talking about not making decisions about most classes and functions, most type definitions, most of the weedy logic within most modules, etc. If it can't be described in natural language as requirements, or in a typescript type or other data shape DSL (I think key data model types are probably still important to own), it's probably too weedy.
Natural language test cases still define the expectations both at the product and architectural level and are essential for triangulating the agents on successful outcomes.
A requirements and verification approach with agents is not waterfall any more than TDD is waterfall. Does thinking ahead and doing some planning equal waterfall? Does describing how a feature works to an end user, and making some key technical decisions, before you build it, mean waterfall? Does having some sense of what you're building first mean waterfall? With agents, you can specify (with PRD and technical specs) what you THINK it should do, and in minutes or hours or at most days, have the result, which you then learn from and iterate. If you didn't fully think it through, the agents will do one of three things: 1) decide for you, which you learn from 2) stop and ask, which you learn from 3) introduce bugs, which you learn from.
It's extremely iterative.
How are you thinking about technical debt in this new agentic age? (also markdown debt?)
It's one of my bigger concerns and I use this iterative approach to try and reduce it... "go look at the ./cli directory and generate me a report of inconsistencies blah blah..." or "review ./research/something.md, validate consistency, correctness, claims, ..."
... pseudo prose, have we made that a thing yet like pseudo code?
"solved coding" is type of thing you say if you want to sound smart.
Saying that LLMs have "reduced the cost of coding" would be boring. And using your analogy, pencils, typewriters and computers have all reduced the cost of writing, but writers are still around.
I'd separate "coding" from software engineering. Software engineering is not solved. The new challenge in software engineering is how to properly use AI to maximum leverage and nobody has solved that.
But, "coding" is, except that solving it means using techniques that are still barely understood and barely described today (something I'm hoping to be able to communicate better myself about). It is now possible (at least for most consumer software, I would not say applications where human life or extreme risk is involved) to work exclusively in the domain of natural language requirements, natural language architectural/design decisions, and natural language test cases, and deliver product of equal or better software quality than average human code authors could have produced. Even in a language that you plausibly don't actually know how to code in, because the language itself can often be separable from the requirements and verification process.
If someone who doesn't know a language or framework can plausibly produce better software using it, and faster, than a team of people who do know the language/framework (and I would strongly argue that is more than plausible now with models like Astra and Fable) then it's not just "coding is less expensive."
It's like saying computers make complex calculations less expensive. That's true, but it's missing the real paradigm shift.
I'm not trying to sound smart, I just think it encompasses what most people now acknowledge. LLMs write code faster and generally better than needed for most cases, and that seems to be becoming the prevalent opinion seeing as most folks have stopped doing PR reviews.
The problem is, without PR reviews & strict oversight, we're losing knowledge, system design & control of our codebases & products. Which is why, IMO, the coding is solved but the other parts which used to be so tightly coupled to programming are cropping up as their own issues.
This ties into my experience: greenfield projects turn bad way faster than legacy ones, because AI has no established patterns to key off and it does increasingly dumb shit.
"Reducing the cost of coding" would imply that you still have to code, but with current LLMs you no longer have to. You can write 100s of thousands of lines of code without ever having to touch a single line of code. Literally "here is a git repo, here is an issue, please fix".
You might still need to nudge the LLM in the right direction or stop it from going off weird tangents, but none of that involves touching actual code yourself.
What is coding? Pushing keys or knowing the right keys to push?
Ai can push a lot of keys very fast, but not always the right ones
if they need to be reminded to follow the coding standards, visible in the very code they are working on, what has been solved?
> What is coding?
Opening the IDE and typing program code. With LLMs you don't have to use an IDE, you don't have look at program code, you don't have to care about coding standards. You ask the chatbot to write you a program/feature/fix and chatbot does it.
Is chatting with a chatbot still "coding"?
The part that isn't fully solved is just the software architecture side of things, do you want library A or library B, or write it all from scratch? LLM can do all three, but if you aren't careful it might go down a route that you don't like. But that again can be fixed with chat, "replace A with B", not coding.
So lets say you plan the software architecture and take a person off the street, give them a frontier model and tell them to clone Figma given your architecture
How far would they get?
As a developer, a good way to reframe this and step away from your own domain expertise, is to read Terence Tao's conversation with ChatGPT and ask yourself "could I have done that?"
It solved coding as in it's significantly less of a bottleneck than it was before.
At least that's how I experience it. In the before times each non-trivial code change had a real opportunity cost as it would easily consume two days until I could even estimate whether this is worth looking deeper into.
Given a working system, like a web service or services, and their code, non technical people can now make effective changes with LLMs.
For example, nobody on our team writes manual code anymore, we have basically set up a harness where an engineer types up the requirements for a change, the system implements it given certain constraints, we have automated unit and integration tests that are ran, and if any errors pop up, they get fed back into the loop until fixed.
But to do that, you need to actually know what you are doing - you have to have good instructions to keep the agents in check and not start making mods outside of their bounds especially when the issue is with a dependant service that is causing errors.
I'm well on my way down this path too, but that is a process implemented in markdown. I don't see coding as solved, or a solvable thing at all, like writing
To solve something, there must be a defined problem, what is the problem that was solved. Or perhaps it is just "coding is solved" is the turn of phrase de jour be ause we haven't yet found a more succinct and accurate way to describe the paradigm shift
When it comes to non technical people using Ai to build things on code, the outcomes are on average pretty poor, which i see as evidence that the driver and their expertise behind the Ai matters a lot. A notable example is Terence Tao's conversation with ChatGPT, us math normies could never have done that. The same applies to coding agents ime
Writing is a means of expression and communication. Code can be those things, but that's not its primary purpose.
I think "solved coding" is taking it too far, but for many projects, the mechanical aspect of it has been removed or reduced greatly.
LLMs will have a much harder time "solving writing", because they cannot develop their own style and so are severely limited, creatively. This is less important for coding.
I agree that writing is much harder, largely because there is no way to get concrete feedback for "does it work"
I still think they produce shoddy or sus code too often, an artifact of the current generation's training to try anything and everything until it "completes the task". They have a hard time even with that concept, which is part of "coding" imo
maybe it’s more like “abstracted” away, in the same way higher-level languages “solved” needing to code via machine instruction sets. Higher level languages allowed humans to think more like themselves. AI puts another abstraction in front of the outcome, making it even more generally open to human thinking and less defined by the need to give machines exactly what they expect.
Today, human-language outlines / briefs / prompts are “compiled” to code which is itself then adapted to hardware. We are stretching less and less across the divide, doing less and less work on the terms of the machine. Now the farthest we’ll stretch is often formatted markdown - the most basic application of machine-parseable structure to very organic human thinking. Because we’re given the chance to be less precise, coherence suffers.
It means "for my requirements they're good enough."
They haven't solved coding.
Programming is an art form. And the better you get at it, the better kinds of ideas (abstractions) you can create.
This is something today's AI cannot do.
If everyone were to permanently switch to AI for software development, software innovation would cease.
even setting aside the artistic side of the profession, I see these things make all sorts of basic mistakes, so even the mechanical part doesn't seem "solved" ime
Outsourcing solved coding a long time ago. You can go on Fiverr and prompt a real human developer for $2/hr - even cheaper than LLMs!
Wow, I think you have never had a chance to communicate with someone selling their expertise for $2/h. Unless your time is also worth $2/h, I think you would get better results prompting local 27B model.
Same problem though :). If an overseas employee knows the code and you don't, you have ceded ownership, problem solving ability & future direction of the project.
>The problem is code is the wrong abstraction for the work we do
My team recently spent two weeks on a wild goose chase trying to figure out why TensorFlow Lite was generating nonsensical OpenCL kernels. Well it turns out that LLVM had a few bugs in the RISC-V assembly for our platform that was leading to silent garbage. It took combing through assembly dumps, hexdumps, a lot of pain staking debugging, and going through the TensorFlow Lite source code to to track this down.
In your opinion, if code is the wrong abstraction to be working at, how do you approach this scenario?
Fair question - IMO it's the wrong abstraction for building and collaborating on a new product with a team.
To your point, it's not the wrong abstraction for solving code level bugs. Just like python is not the right abstraction for solving memory corruption or pointer mis-alignments.
Even if they "solved" that, the problem is it's the LLM that "knows" it, not the team.
Which is really the same problem with coding.
The agentic model of it just taking over and doing everything is poisonous to effective long term team work.
We're well past the point where it's about the quality of the work they produce. It's the way they integrate (or rather, don't) into human practices.
See my edit on the right abstraction - how do we get humans able to collaborate with agents effectively, where knowledge flows both ways, without needing a human to read 10k LOC, or even just lots of long LLM responses, to follow along
Or more broadly, LLMs fundamentally don't "understand". They can simulate understanding and generate text/code/whatever, but they don't have will to engage with something holistically and "own" it.
I have been trying to define "understanding". Is it when you can predict something that you understood it? Or maybe when you can explain it? Or how about when you can control it? Or invent it.
This time I add another definition "when you can own it".
When you can own it - I like that. Humans are still needed to own systems and drive/direct changes. The question is how can we collaborate as organizations and teams to still own it when we don't write the code and can't keep up with the output
I find it helps to imagine what is/isn't solved (however, we choose to label it) by an indefinite number of cheap junior-developers.
Except a little worse, since they were raised alone in a library, act mostly the same, and have harsh limits on personal growth.
tbh, I don't think of LLMs as junior-devs. Maybe 6 months ago, but now they are v. senior code-monkeys. They are masters of their craft, but their craft is narrow and lacks big picture, collaboration, org goals, etc.
You can immediately tell someone has absolutely no fucking clue what they're talking about and dismiss everything they say as soon as they say "solved coding". You must write the most heinous code imaginable if you think LLMs are better, or even anywhere near, what a competent professional can produce. So funny to listen to terrible coders talk up terrible code.
In the first place, what does it even mean to say that code is the "wrong abstraction" for the work we do? You can't actually abstract away reality. Maybe you wish we could, that programming computers wasn't about contending with physical constraints. But it is. It will never not be important to have control over what the hardware is actually doing. Abstractions are temporary conveniences, not a replacement for understanding what is being abstracted away.
Thanks my friend - I'm not an AI evangelist but I think it's pretty clear that AI codes very well at this point. So well that I'd hazard a guess that most software engineers don't write code anymore. Sure, maybe a professional could do better if they spent way more time on it, but it has always been a tradeoff we programmers have had to make. Getting things done + technical debt or writing the perfect code and over optimizing. And LLMs write working code, so the rest is just semantics on that scale.
Code being the wrong abstraction is similar to assembly being the wrong abstraction if you're trying to write a web browser game. Sure it's possible to do it and yes, you may end up with more optimized code, but it's not necessary and much slower to do that. Just use Javascript.
And now, people are building larger applications & functions quicker, and it no longer makes sense time-wise to look at Javascript for-loops when understanding the work. And that's because those for-loops are generally correct, if maybe a bit under optimized. Instead, engineers need a new layer of abstraction, to understand what the code is doing without having to read each line.
LLMs are faster until the technical debt comes due, at which point all supposed productivity gains evaporate and you're now 10x slower than doing anything by hand. They have a use if the debt never comes due, eg. for one-off scripts or for prototypes you will responsibly dispose of. The debt does come due in production, very quickly.
LLMs write code that compiles, which they accomplish mostly by robotically attempting the task and repeatedly fixing compile errors in a loop. That's a very narrow definition of "code that works", and I would argue it is the starting point, not the finish line. My definition of code that works is more like: secure, stable, maintainable, efficient, performant, effectively bug-free, with an ergonomic interface. LLMs fail on every single fucking count. I routinely observe generated code that leaves trivial 10x or 100x gains on the table, while having severe deficiencies in every measurable approach.
Given that you talk about assembly as a bogeyman, though, I gather that you're from the generation of "software engineer" who was already writing insecure, unstable, unmaintainable, 100x inefficient, 100x non-performant, bug-ridden JS for everything. LLMs can replace this class of people, it's true.
I feel related to it. I switched companies at the end of July. First 4 weeks AI was giving me the feeling of freedom. No longer I need to understand tens thousand of lines of legacy codebases. Never onboarding was so easy. Just ask Claude and it tells me what happens here and how.
But after 2 months it starts to backfire me. I still know nothing. I have some understanding of the system design and core components but I have zero clue about how certain things are done under the hood. Because AI read code for me and code for me and I take it as my own understanding.
In last week I end up limiting my AI usage and forcing myself (it is really hard) to read and code at least a bit by myself to start having any idea about what is going on here.
AI is not for programmers, they will always complain and given time and resources can of course produce better atomic code. AI is for a combination of a business analyst, architect and product manager in one person who has capacity in coding but decided this is limited role in the whole software engineering lifecycle. With the right attitude and of course certain level of micro management over AI (which they had to do with human programmers as well) they can achieve the same or similar results with much less communication friction, less people to maintain that teamwork and get closer to solving issues they find rather than discussing philosophy of programming and other manifestos of the craft that stopped being prestigious or unique and that’s the key point of complaint when AI is given wrong people to use.
That's my recent experience as well. You have to know the domain and have technical knowledge so that you understand (not in every detail) what AI is actually doing and why. You also have to understand what you want and why you want it.
But once you do, working with AI is like having a very capable team of programmers knowing all programming languages you might need and understanding your goals.
Can they really achieve similar results though? If quality suffers greatly, and velocity eventually slows down due to huge tech debt, isn't that just a worse outcome?
My 12+ months learning is that quality suffers primarily when I cannot find the right working process to assure it. It's not software development as a process but software engineering as a whole holistic picture across all departments and traditional roles (dev,test,devops/sysops, pm, etc..)
I also found that what is commonly called "spec-driven development" is just a different CMMI alike model that works better with AI than anything AGILE. which is also nothing new.. just realisation that AI is the ultimate proof agile development/manifestos is a dead end.
there are many other reflections about the whole process to ensure quality, good architecture, minimizing regressions and providing business continuity and proper, reliable, SLA driven maintenance.
Actually not that much different than certain human-centric models.. just need to be adapted to the AI reality to truly benefit from it.
when that is done, from leadership perspective, it is a working process that brings similar outcomes and quality. not exactly cheaper atomically, but faster and brings more scalable opportunity to drive multiple projects within the same resource pool.
I think that's what is mostly experimented today and that is also of course exhausting traditional developers.
"Bet on it and start your own company"
If you believe software still can be sold. as I do believe further development of AI may completely redefine buy vs. build capabilities and potential directions.
My bet is that half of the software developed so far may be obsolete 10 years from now and so the businesses behind them. SaaS specifically.
I'm seeing this right now in my domain. Much stronger lean towards build than years past. Not enough signal yet to really make a call but I suspect a significant downward re-pricing is going to have to happen to convince folks to buy. At the moment the arguments of "your core business isn't software so why build?" or "you'll have to maintain this and there's no one else on the planet that knows it", etc are not working since some of these folks have fully bought in and just say "yeah so, AI will do it." Also the number of features expected, and many non-trivial, is pretty high. wild times!
It's not a given. It's about willpower and care. LLMs are the ultimate crutch. I over-rely on the crutch, more and more people will over-rely on the crutch. But it is still inherently a psychological problem.
You can still know things and get force multiplication out of LLMs, if you are disciplined and caring enough. In practice, most people won't be. And you can't force other people to be. But you can force yourself to be.
I remember when:
> You can still know things and get force multiplication out of StackOverflow, if you are disciplined and caring enough. In practice, most people won't be. And you can't force other people to be. But you can force yourself to be.
The tools constantly push you away from any conscientiousness about the work and towards just letting them do everything.
AI is going to make, long term, Software Engineering more important. Getting requirements, user feedback, tests, product vision. These are key differentiators now.
If you can really get a good set of requirements, go and write all of your test cases out, and then throw it at an AI that will one shot it. Its perfect.
You're basically describing a waterfall process. That never worked in a pre-AI world and won't work in an AI world. Why? Because it's just not possible to gather absolutely every requirement up front. Software development has always worked well as an iterative process. You come up with better solutions as you go along and learn about the problems you're solving and make improvements. While AI will be able to one-shot a set of requirements, you'll miss out on those improvements because you'll have skipped over those phases.
I've tried this out. Even with relatively small applications and with spending hours reviewing the spec documents, there was always something I missed or something that wasn't quite right when seeing it live.
"Getting requirements, user feedback, tests, product vision", that also apply to iterations. Iterative processes are just a cascade of small waterfalls - either a scrum week or just a single agent session (define, prompt, plan, implement, verify)
But today you can do even better, with AI can do true waterfall by rewriting from scratch many times.
> But today you can do even better, with AI can do true waterfall by rewriting from scratch many times.
This does nothing other than ensure you end up in the "joy" of running a v0.0.1 product but for years on end instead of for a few months.
I hear this sort of thing a lot, and I can't help but internally translate it to "I've never had to take oncall for a product directly after a rewrite". It will be broken; not even because the AI is "wrong", but because the rewrite has new edge cases. No one rewrites a project to have the exact same edge cases. Those edge cases will become outages. No one will learn anything, because a month from now it will be rewritten and those edge cases will get swapped for something else; you can pick which edge of the CAP theorem you want to live on, but you can't pick "none of them".
Absolutely, I should really put "/s" after that.
Really I was thinking about exact example announced recently on HN: Postgres rewirte in Rust, it has 3 versions looks like each one generated from scratch https://github.com/malisper/pgrust
In this moment there are a million ways to do things wrong. But there are also some ways, maybe less than a million, to do things right.
Here's an anecdote about a way to do this wrong.
I have found with AI coding methods that there's a line where it becomes a hail-mary (in the American Football sense).
A hail-mary is when you throw the ball to the end zone and just pray someone catches it. This is almost always at the end of the game.
This moment with AI code is indicative that you can't put together a coherent plan so you just tell the agent to "make it good". It used to be that the results here would suck, but now the agents are really competent, so the results might be good.
But at that moment, that's your cue to back up. Because as soon as you take a solution that's so far detached from your understanding, you're underwater. The hail mary is not part of a larger game plan. It's the last play of the game. There's nothing after.
So as soon as you reach that moment in your coding, you're signaling that you're done understanding not just the code, but even the way it works at a high level. If you're still going to work with this code after, then back up and work with the AI to get more understanding of the problem.
> A hail-mary is when you throw the ball to the end zone and just pray someone catches it. This is almost always at the end of the game.
So that's why that Andy Weir's book was named that!
This is obvious cultural context for most Americans but probably not many non-Americans.
I didn't read the book but they explicitly mention this in the movie.
https://en.wikipedia.org/wiki/Hail_Mary_pass
Yeah we forgot the goal of programming isn’t just to tell the computer what to do, it’s to program the programmer into thinking a deeper understanding of the problem.
As a programmer, today I can confirm with no room for doubt that programming is dead (programming by humans to be precise, the act of typing code on a premise to present a solution to a problem)
We may consider ourselves the last programmers before the AI era
While I fully agree, this is hard to sell to the other side, be it management, or a customer.
Each to their own, and the market will decide in the end? If a company is going to (implicitly or explicitly) incentivize everyone to directly ship Claude’s output straight to production, and prioritize features above everything else, that’s a choice. Providing that I have a long enough runway, I’d love to have this company as my competitor? Yeah, year 1 will suck, because they’ll pull ahead, but if I can stay in the game until years 2 and 3…
Aside: this reminds me of running a “negative split” in a long distance race, where you aim to run the second half faster than the first. It’s very hard to do this because you have to be willing to let everyone else in your pace group pull waaay ahead, and running above race pace in the beginning feels “free” with all of the adrenaline. But if you do manage to stay disciplined, it’s a fantastic feeling to reach half-way with gas in the tank, and then start to reel in all those runners who sped by in the beginning.
> because they’ll pull ahead, but if I can stay in the game until years 2 and 3…
well as you said, the market will decide in the end. You may be even further behind in years 2 and 3. btw, we're already in "year 2 or 3" territory for some, i wonder how those companies are doing vs their competitors who adopted AI full throttle?
Most people say full agentic coding only got good enough for production at end of last year. This year was the turning point for most companies. A lot of experiments going now right now.
Of course things can change drastically in the next years, as they already have in the last few.
Only thing I know is that it is really stressful working on this rat race
I don't think that's how business works, unfortunately. Customer adoption is pretty sticky.
Yes, customers are sticky, but every customer has their limit.
And of course, this is very much dependent on the switching costs. Which sucks, because tech companies are great at making this as high as possible; the EU is trying to do something about this with their data portability legislation.
Anecdote: I worked at a company providing infrastructure as service. We lost some bids the first time around, likely for a variety of reasons (price, features, ...). And, some customers came "back" to us after using our competitors' offerings and being burned by their reliability. Yes, switching costs were real, but the pain of losing their customers was even higher.
We've had team changes and product manager changes and lack of documentation for so long that this was the state of our team anyway: no one knows why it was done and no one wants to break it.
> But the final boss is, and always will be, maintainability.
I think this article misses the mark in a few ways, but this is the biggest. I see AI as the ultimate solution for maintenance, in three ways:
* AI does a great job of refactoring and so tech debt becomes shallow. If something is not architected right, it can be fixed much more easily than ever before. * Bug fixing is also a great AI strength. In the future there won't be a backlog of all the bugs that never got fixed. AI can fix them as quickly as they come in. * AI doesn't get bored, doesn't get tired, and doesn't care how crufty the code is. It is happen to maintain any application without judgement.
I some very real way this has been true for a very long time. Does anyone know how a AMD Ryzen processor works? I mean does anyone understand in any detail how it works from machine code down to the transistors?
Does anyone single person understand what’s happening when compiling a large C++ code base? Meaning, can anyone track the basket cast C++ language constructs down to the Clang IR to the optimized machine instructions? From there can any one person follow those machine instructions all the way through to the actual registers etc to actually running the code?
These year and a few next ones will be the years when everything was possible.
We have both the tools and the skills.
Later, we will loose the skills because of AI and the depletion of natural resources will lead to the scarcity of the tools.
I'm understanding my role as a developer in the AI era through the lens of the company hierarchy at my first job. In 2018 I began my career at a management consulting firm which also did tech consulting for a very specific industry.
The hierarchy was as such - associates at the bottom who did all the manual coding work and got into the weeds with the tech, consultants on top of them who orchestrated the work and kept the wheels moving, and managers on top of them who had a more higher level view of the system and knew why decisions were being made (partners were involved with future deals and driving the business forward). The managers who knew how the system worked had no idea about the actual code underneath it all because it was abstracted away from them through multiple layers of hierarchy. The actual code which was mostly written by 21 or 22 year olds who largely had no idea what was going on other than the fact they had to make it work or risk the ire of higher ups was a mess, unless the consultant orchestrating it all had real technical and system design chops.
I was an associate then and more often than not I had no idea what i was doing but i was fortunate to work with consultants and team leads who did. Right now, I'm a software developer akin to a consultant, with more ownership of the code but no associates to guide because all of that work has been pawned off to AI agents. I see my role gradually morphing into what the manager role was at the consultancy, where i have a bird's eye view of the system and know where the project is moving and why decisions are being made, but with a very limited knowledge of what is actually in the code.
I've noted a similar shift, but I think it's a short-sighted path.
My experience (both in and out of management, and as a very high level IC at a mix of companies) is that while the management layers generally didn't understand what was going on in the code - the decent managers tended to have a "gut check" person they trusted to go and ask technical questions, and bounce ideas around with.
This was their "expert" who they trusted to be close enough to ground truth to tell them if a thing will work, how hard it will be, if there are shortcuts, etc... Often times, it wasn't even strictly technical questions that would get answered, it will be things like "how much of this project is technically in my sphere of control, and how much do I need to go align other teams? How many teams? Which teams? Which projects? etc..."
My problem is that AI SUCKS at this job. So yes - we're getting pushed into management roles as ICs... but instead of a decent subject matter expert who still understands ground truth... you get an LLM blabbering on about seams and load-bearing joints, and clickbait style headline summaries, but completely non-nonsensical judgement calls about how to interact with the people who make up the company.
Ultimately, work is also politics - and it's not just "writing code" that matters. It's knowing what code is worth writing, and when you should write it. And an AI will happily tell you that a single change-set in a single repo is "substantially larger then expected" and put a timeline of "two weeks" on the completion (especially if anything Jira related happens to be in context... blegh), and then knock it out in 5 minutes time... While also spouting off about how easily it can add "new feature X" it just needs to get PRs in across 5 different repos, owned by different parts of the org with non-aligned managers.
I don't really understand all the negativity - I've delivered amazing robust solutions at about 10x my previous rate I think and can make dramatic cross code changes on awful legacy code bases with ease. I just added some complex prev/next navigation across a load of detail pages pages that rely on the current search, a new PDF generation system that uses a queue and workers and absolutely tones of changes to a legacy ERP system that would be unworkable without AI.
Dealing with legacy messes I used to get frustrated and bored of making the improvements, now it's easy to clean up code bases and write loads of tests. I asked Astra to come up with a plan for Playwright testing the whole App, I have not built it yet but the flows suggested were fantastic as was the ephemeral database we plan to create for CI.
I built my friends portfolio website almost entirely vibe coded in 4 hours and it looks unbelievable, we added so much slickness (he's a designer) just prompting together. I used a CMS I had never used once before and it was so so easy to do without any of the usual need to read docs about everything.
I've done so much devops now I'm actually fairly confident that me and an AI can do anything you want in terms of deployment/infra and scaling from AWS to Terraform to whatever.
Anyway my main concern about this technology is not that it is crap at coding it's that the improvements in how it codes and thinks are absolutely dramatic which is extremely scary - it has come so far in a year I wonder what the next year will bring.
When coding agents first came out, the overwhelming narrative was how bad they were, they don't know what they're doing, the code they write just sucks and they hallucinate APIs.
While these criticisms were technically true, they were stated mostly out of a sense of insecurity from people whose jobs were basically spending years and years just glueing code together and mixing APIs to display some CRUD apps rather than out of a genuine concern for whether LLMs were actually producing poor products.
Nowadays those concrete criticisms don't really work anymore, LLMs are pretty good now and surpass most developers when it comes to writing the majority of shovelware that people have been employed, and so the narrative is changing from concrete criticisms about how LLMs were genuinely not capable of writing software... to these kinds of abstract and philosophical arguments that are really hard to argue against because they make no concrete claims.
If you say an LLM can't implement a feature, we'll we can test that claim concretely and LLMs are getting much better with every new release. If you say its code is slower, buggier, or less maintainable, those claims too can be measured and once again they're getting really good at these. If you say it takes longer to complete a task or requires more human intervention, we can compare it and measure etc...
But now the objection has shifted not to LLMs are incapable, but people are now incapable and LLMs represent a degradation of the "craft". And here there is nothing left to test or falsify. The argument has stopped being about whether the LLMs work, because that's verifiable and they are now at a point where it's hard to argue against their ability to actually produce functioning software, so now the argument is about whether people are morally, culturally, or intellectually permitted to use it.
I'm with you here. If we skate to the puck, LLMs will continue to write tighter and tighter code, comments, unit tests, integration tests, etc. I would also assume that it will become increasingly good at code organization, architecture, and understanding intent. i feel like I am a series of monthly shifting of the goal posts...."LLMs can't autocomplete code", "LLMs can't write a high quality function", "LLM's can't understand small repos". "LLM's can't understand legacy code bases".....
If it's this easy for you it's just as easy for anyone else. What do you think will happen to your salary when no particular skills are required to do your job?
So what exactly are you proposing I do - just ignore the fantastic gains and great software that people are using day in day out?
Next year you may be unemployed (because your boss will cut by half the engineering department), or perhaps your salary will be cut by half (because your boss will route your other half of your salary to openai/anthropic), or perhaps you will get half insane (because your boss will ask you 10X the output you used to have but within the same timeframe)
Whatever the path is, regular engineers (90% of the people around here) will get screwed up one way or another. But hey, playing with LLM agents is cool!
Yes I agree we have all signed our own death warrants. But this does not just apply to coding.
Are you suggesting not using the tools? A kind of technical version of the Amish way of life?
I dunno I think my guidance and testing is very important and I make sure not to ship things with bugs and code that is really awful. I'm still just about necessary for now.
You may not be at a company where people are dropping AI slop into the codebase faster than it can be cleaned up. I've thought about my options for going "single operator", so I don't end up with a mismatch between people who use AI thoughtfully and those those who don't.
The Computer Chronicles – Word Processing (1983)
https://www.youtube.com/watch?v=Jt0OoXluC8g at 4:08:
I don't mean to be mean, but the problem is that AI has exposed how much programming is designed around human usability. C/Rust/Go wrap assembly. TypeScript wraps JS. Java/Python wrap C.
Do you need all these layers of abstraction when the human is no longer looking at the code?
Best analogy is forgetting how to use a slide rule following the advent of calculators. The former was made to make hand-calculation of logarithms easy. The latter does these calculations directly (obviating the need for a slide rule at all).
I think what humans still need to learn are the theory and domain fundamentals for their industry. If that industry is computer science, that means algorithms, calculus, linear algebra, etc. I think the future of CS is then (a) theoretical human-drive design and (b) prompt engineering to implement and verify that design.
It would also be helpful to have domain knowledge outside of CS as having the skills to build something is nearly commoditized (outside of the above fundamentals).
> C/Rust/Go wrap assembly. TypeScript wraps JS. Java/Python wrap C.
LLMs rely on the semantic richness of these languages in order to capture the the gradient of human intent from their training.
There was a post a few days ago about why control-flow-macros aren't that bad, and their thesis was basically "because AI can understand code well now", though I feel like that sort of missed a bigger point of "if AI can understand the code easily, what's the point of macros at all?"
Even hygenic macros still end up being there primarily to increase legibility and ergonomics. Good ones, like core.async, make it easy to understand how threads and the like are glued together, but ultimately it's just syntax-sugar on steroids.
If the goal is not for humans to read the code at all, I'm not entirely sure of the point of syntax macros; the LLM could just generate the expanded code.
I think it is not needed to own every line of code anymore , but you should still have control and idea about the architetcure of the system. You should understand what componenets are there and what is the repsonsibility , how are the orchestrated and what are my quality gates where i messure if what i requested match the results. Thats why i build archkeel (https://github.com/rapiddweller/archkeel) and datamimic (https://github.com/rapiddweller/datamimic) ... to increaes the transparency and review surface for human ... to make the results easier to judge ... i think when we stop knowing anything, we can also stop burning token and ressources
You could at least write the READMEs
I agree writing completely new systems with LLMs is now near impossible to keep in your head, however, the onus is STILL on you the individual engineer. You are mistaken if you think that's changed.
So, if you're pooping out code, and committing it because tests still pass, and that's all you know, you're in for a treat. When an executive wants to know why a b0rked feature lost their department millions of dollars, guess who will have to answer for it, and its not the LLM.
My advice is to find ways to keep on top of how it all works, and if you're the only one who cares, well, then, that makes you even more valuable, not less.
> the onus is STILL on you the individual engineer. You are mistaken if you think that's changed.
In a lot of shops right now, this leads to an accountablility-authority gap. Where AI changes are merged in quickly without review and without my input, how can I be responsible for understanding the system? I can't. Its the same with reliability.
A lot of good engineers take it a a personal duty to understand the system and keep it up. The way a lot of businesses are using AI makes that impossible. And your job might just be cranking out features, with regard to little else.
I think this is what is stressing a lot of engineers out. It might be worth having a conversation with your boss about what you are actually accountable for. If your boss agrees you are not responsible for uptime, reliability, security, or even understanding the system you might find yourself much happier. If you your boss wants you to be accountable for those things, then you should feel empowered to ask for authority over the things that give you control of the outcome.
The post seems to be in line with some of my own direct observations. I've been pondering recently whether for many devs the use of AI is the death of the mental model.
We all interact with systems through mental models, but if many devs are just prompting claude when something doesn't work, they might read what claude found, but lose out on the exploration, debugging, and work that builds and reinforces the correct mental model and discourages the wrong one. And if devs are missing out on the mental models, will they actually be capable of driving efficient solutions to problems as the mental models get worse.
that tweet about the fast moving startups looks like almost the hypothetical AI nightmare situation just dreamed up. Assuming it's real, i mean yeah, people shouldn't work for these stupid high-moving startups I guess. From my perspective, "when did startups NOT suck?". The code was always garbage at startups, the push to work 12 hour days was always at startups, "nobody is resolving bugs" ha well yes, welcome to a startup? I hope the guy is at least getting paid and doesnt have to hire a lawyer to get his checks like I did. startups suck
Exactly my thoughts..thank you for doing the working of actually writing it up.
As I was reading the OP I kept thinking..well this sounds exactly like what used to happen before AI.
The more things change...
I'm curious how much people who code this way now are spending in tokens every month? Where I work currently, we operate on a $200/month token usage limit. For me personally, this has prevented me from just endlessly prompting claude to fix bugs.
Even with unlimited spend, it seems immensely beneficial to dig into the code base and fix a certain amount of bugs oneself. Oftentimes this is ends up being quicker than having to type out a detailed explanation of an issue in plain language, with the added benefit of maintaining intimate knowledge of the code base.
I have the $100 subscription and it pretty much does allow me to endlessly prompt Claude to fix bugs (and other stuff).
There is no "problem".
The architecture and the intent don't matter to business folks, it never has, and with LLMs it matters less than ever.
Vibe code into production is satisfactory regardless of architectural understanding or intent. If there are issues, just have the LLM spin up some agents to play whackamole until the issues are pushed beyond visibility.
Ultimately, the idea that we can hang onto fleeting engineering disciplines misunderstands where the industry is going, regardless of any assessment of the LLMs capabilities.
Great job security for us that understand programming fundamentals in the future when the vibe-coded monstrosities collapse under their own weight.
I don't think it guarantees job security, I think the future is an adaptation to the prodslop paradigm where the answer to vibe-coded monstrosities is more LLMs.
Ultimately, the industry will be forced to adapt to the extinction of the junior to senior pipeline, but investment in more tokens is going to be more attractive than paying specialist engineer salaries, especially since LLMs probably have a few more generations of qualitative improvements before diminishing returns truly set in.
After experience on both ends of the spectrum, I arrived at the position that what we basically need is to be a "Responsible Human in the Loop (RHITL)" [0]
> You may not write the code by hand but you understand it enough to investigate and fix it when it fails. It is how I think we should leverage AI instead of becoming a meat proxy.
[0]: https://raahelbaig.com/entry/responsible-human-in-the-loop/
Don't touch the code for 10 years and even the brightest will experience serious dip in skill. Most probably much earlier, our minds are lazy by default since creative thinking is energetically draining. Then try to debug some harder problem llm created in its own code. Look at what its doing to students.
Its like saying you can keep fluent level of French as a foreign language to you by just reading it. Nope you won't - do it for 2 years solidly and you will have problems forming basic sentences correctly if you don't use language actively, forming foreign language constructs in your mind.
Those few of us who are largely apart from current llm craze (due to my mega corporation and its obscure rules and procedures) are not missing much I see. I still happily code changes by hand and at most consult basic available llms via prompt, its beautiful process to create fix to a problem which seems cryptic at the beginning, and knowing it all came just from my mind. By far the best part of my corporate job, will keep hanging to it for as long as possible.
People keep saying this, but how realistic is that in reality? Even say a person is doing their best to do that, we all know that the alarm that doesn't trip 99% of the time isn't going to be closely monitored. Just look at what happens when people get used to "hands free" cruise control.
Not only that, but even if we assume best effort on the engineer, the business pressures don't often allow that. I'm under constant pressure to deliver more, faster with less people. The performance eval ladder at my company was just reworked to double the amount of deliverable features expected per job level per year. Our CEO told us we should be able to deliver what took us the past decade to deliver in a quarter, every quarter going forward.
How could you possibly have a human anywhere near that loop with those demands?
That human could write the harness that enforces the guardrails, for example, working at one level higher of abstraction.
If that was possible it would have been done. AI is a best a mediocre self regulator
My boss-boss asked me to do a presentation on a topic I actively research for quite a while now.
Scheduled a quick call to align me on what he expects - normally he wouldn't do that but he has attached a big agenda written by Claude what the presentation could show, and invited two other product colleagues of mine.
I came to a Miro board of the Claude Agenda, put into Miro using the MCP.
Honestly, just tiring. Asked my colleagues if they would just put the Claude agenda into Miro with Claude, what they need me for when an AI could just narrate it.
No the problem is allowing changing of system architecture or intent because the AI just did it and being totally trained to mentally surrender to the whims of it.
You could argue the same with high-level vs low-level programming languages. But I do get that English is so abstracted and non-deterministic that we're kinda clueless of what's going on. At the same time, I do think that maintaining code does require some level of understanding. And unless you straight up one-shot something perfectly, you still want to test manually and tweak with a few more prompts, adding a level of 'how does this work?'.
This is also freeing up time to explore things that before you wouldn't have been able to even start. New fields in tech are opening up. It's all about the model, compute, plugins, third parties... and more to come!
And this is not limited to code, but also how the world works. It's all being abstracted into prompts. Funnily enough, I'm also learning that way...just taking less time to get to the point. But this is not the first time we go through this. Eg. Google vs a library. And like anything, if no one knows anything anymore how do we distinguish from one another? There's a level of wanting to understand in order to distinguish ourselves from the rest in the serendipity of everyday life.
Call me naive, but something tells me we're going to start being much more open to just exploring the world with all this time we just bought ourselves thanks to technology. We were always gonna get to this point and there'll undeniably be bumps ahead.
Clicks on blog, sees AI generated template, leaves.
Enough, already.
This page has been live long before AI appeared, I used Quartz [1] v3, now at v5. But added some features I like with AI, yes, but still keeping my beloved color theme and web style as I had it for a long, long time. The whole code for the second brain (ssp.sh/brain/*) is on GitHub, in case you want to check it [2], but you can also see the "Changelog" in the footer to see recent visual changes. But I'd be curious what makes you think it's AI-generated?
[1] https://github.com/jackyzha0/quartz [2] https://github.com/sspaeti/second-brain-public
Apologies.
The beige coloring with the bright green "recently updated" label somehow triggered my "this template is made by Claude" instinct.
good point, thanks for the honesty. I will change that, I think the color inside pop up make sense (red removed, green added words), but the green dot does not add value.
How humans can differentiate the things they make has been something I’ve been thinking about a bit recently. What markers signal to you this is AI generated?
I mean, the content doesn't look AI generated. I couldn't care less if someone AI generates their blog template, the rest of the content seems meaningful.
Who cares? The writing certainly wasn't AI.
I predict we will see more and more of these convoluted rationales (i.e. excuses). All these basically boil down to: “I know AI is crap, but I have found a way to make it useful. Trust me.”
I am gonna appeal to Occam’s razor here and say, the integral variable here is AI, and the only variable you need to know is AI. The problem is AI.
This is the perspective, not code, coding is solved. The wrong input will produce wrong outputs, so a mediocre dev using AI may produce mediocre results. AI can do everything asked for, code, tests, reviews, fixes, etc but still it can not think, for now
Intelligence is the new currency, but it will be short lived
The future of engineering is product management. I don't believe there is any world left for people whose primary responsibility is opening pull requests; and we're seeing the angst against this happen from both directions. Engineers hate it. Leadership feels they don't need them. The truth is somewhere in the hazy middle; we're just in an uncanny valley right now where neither side can take the leap to cross the valley: the agents aren't good enough yet, and the bigger problem is that there's no job title or corpus of experience leadership can look to and say "Yeah that's the person we need in this role".
The labs saw this early, and thus many roles at the labs are "Member of Technical Staff". That's the future for every software team. You're not a software engineer anymore, but you're also not a PM, nor a designer. Think horizontal slices, not vertical: Every human's responsibility is to leverage AI to be an expert on everything necessary to deliver some vertical slice of the business.
> The future of engineering is product management
Product Managers have been claiming this for years now, but put an AI in front of them and they also just start asking it to do their product management job for them.
Worst ones I witnessed were just using it to hallucinate tickets. I saw one even fired because of that. Endless mountains of text going absolutely nowhere, both me and the CPO thought we were going insane from reading so much Claude-ish.
Best one I know used Copilot to code a "Notion to Jira" exporter so it copies requests from business people into Jira. Automated their own work, did nothing else other than monitoring the Python script daily and running ceremonies.
They can pontificate all they want about taste but: modern apps are all copycats of others, work badly, monetization strategies are spaghetti against the wall...
Product management will be replaced by AI way sooner than engineers.
Reading comprehension check: I never said product managers would be the new engineers: I said that engineering is becoming product management.
Wouldn't engineering then be the safest place to be? Engineers have always been embedding loose requirements into runtime law. They (supposedly) understand the business model better than most and have the technical know how to validate that
Yes I think that's correct. But I'm worried less about safety and more about fit. Articles like this convince me that there are some engineers who will struggle to adapt to a world of "no one knows what's going on". Engineers have always had the benefit of making abstraction concrete, through the systemization of abstract business requirements into concrete code that the team would build a shared understanding of over time. But that doesn't seem to be where the industry is trending.
PMs have always lived in the world of dealing with hazy abstraction in both directions: Unclear requirements coming in, turned into unclear system capability whom they have to rely on the engineers to parse. This is where Engineers will have to get comfortable living now: Unclear requirements coming in, unclear code coming out. Its clear that many engineers aren't ready for this, and I don't blame us; it SUCKS. If I had ten dollars for every time I've heard a PM say "no one has any idea what's going on" over the past fifteen years, I wouldn't have to work anymore.
Engineers are probably still the role most suited to adapt to this new world, as you say, but I think people are still vastly underestimating how much they will be personally impacted by the changing industry. If you hate your job now, for reasons like those the article communicates, you'll hate it ten times more in a year.
The problem is always maintainability, which can only be achieved through human understanding of code.
Understanding code is trivial, with AI support. LLMs are better at summarizing and identifying side effects than writing code. Granted, not perfect, but about as good as a person.
Looking at an architecture, module boundaries, and comparing the organization to the domain is the part LLMs don't do well and is unironically difficult for humans. The less you care about the domain (ie having a mental model), the harder it is.
You could say the same thing about the transition from assembler to C in the 1970s and early-1980s, albeit that was at vastly smaller scale of impact. It’s not that nobody knows _anything_. We still direct the machines, just in a different way. And if what comes out the other side satisfies our needs, does it matter what lies beneath?
Tail risks have always existed in software development. The tail risk of a bug introduced by some dev who quit five years ago is similar to the tail risk of a bug introduced by Claude six months ago. Deal with it by building better visibility into how your systems work. Demand that your agents write good documentation to accompany their code-writing.
If you’re doing it right these days, it means you’re thinking of a much bigger picture and containing downside risks as boldly as you’re expanding the frontier of upside opportunities.
You did not have to know about the assembly most of the time _because_ the assembly output of the compiler would be guaranteed (bugs notwithstanding) to conform to certain rules. Not the case with an LLM
"Nobody is resolving bugs" is weird. Coding agents are great at resolving bugs! So much of what I see posted here seems more about how coding agents are being misused rather than anything inherent to them.
Of course that's what people are complaining about...?
Feel like this has always been a problem, especially with large legacy codebases. Very few people ever knew everything. We just reach the problem faster now; it used to take years of churn and turnover.
> But the final boss is, and always will be, maintainability.
Always has been, always will be.
I am actually hopeful that AI will finally break the industry and force a reckoning around this. Some of it goes to our economic system. New builds are usually capitalizable, flashy, and a great way to get promoted.
Doing ten to fifteen years of thankless maintenance, keeping a critical system alive with high quality? Usually nobody cares, and it's OPEX, not sexy.
I'm not convinced that this matters anymore either. Knowing architecture was a vibe coding strength about three months ago. The modern models seem to handle it very well.
This. AI can make sound architectural decisions on small to medium-size problems, and even root-cause and generate a well-factored fix. In one shot.
It absolutely won't be long until product managers are the only humans who actually need to be involves with the software development process.
How can one learn system architecture while mainly vibe coding?
You can't. You need to look at and review at least the core pieces of the code as it's being changed. You need to be able to connect the details to the higher level system architecture and go through the process of asking questions to help discover missing pieces of understanding.
When you simply large amounts of information, the knowledge probably shifts to upper level reasoning, then complex reasoning develops on top of it again. But who knows.
The problem is not eating McDonald's for every meal. It's that we've forgotten how to make healthy choices.
Agent Smith had it right
https://youtu.be/JrBdYmStZJ4
Why was the title of the HN post changed from the actual title of the blog post? I thought it was clear in its meaning.
I think there is definitely gonna be a shrinking of critical thinking as lots of people are gonna be lazy.
[dead]
Has t this sort of thought been the constant refrain of every generation at the advent of new technology?
Companies will pay hard and high...
pretty AI-sympathetic vibes in this thread. I generally agree with the author despite everyones opinion (including my own) in this thread that AI can produce great code. Just becuase the code is good doesnt absolve the human-in-the-loop from understanding and clearly documenting and communicating what the system does.
At the end of the day I'm not paying an engineer to send me claude all day. I'm paying someone to become an expert on a system, even if AI assisted. The product will be better if i have someone that deeply understands the system and where to point AI to. Someone that can understand the full big picture can also anticipate future needs - something AI cannot do at all.
I'm in EE/embedded/FPGA work and you can make an absolute hell of a mess with AI in that world, so maybe everyones opinions are more based around front end software or something. I think people forget there are fields that do not have the huge data training base that frontend/backend software does. a large majority of good designs in FPGA are proprietary at big defense corps, not on stackoverflow and github.
If you don't understand how your system works, your ability to make good decisions about future work on that system quickly degrades.
I don't think you need to review every line of code, but you absolutely do need to be able to describe how the system works and its high level structure.
As is so often the case with coding agents, having experience as a tech lead or engineering manager really helps here. You are responsible for a large system that has been worked on by multiple different collaborator (both human and agentic). You need to be able to make smart, informed decisions about that system, and talk with credibility to other stakeholders about what it can and cannot do and sensible next steps for the project.
>I don't think you need to review every line of code, but you absolutely do need to be able to describe how the system works and its high level structure.
I just don't see how you can truly reason about a system without delving into code. Tests aren't enough, running the software isn't enough, high level system architecture isn't enough.
>As is so often the case with coding agents, having experience as a tech lead or engineering manager really helps here ... You need to be able to make smart, informed decisions about that system
In my experience, engineering managers are too detached from the system to accurately reason about the system. Tech leads on the other hand usually can given enough time, but they tend to defer judgement to senior ICs on the team who are more familiar with the code.
Point is: there's no way to have your cake and eat it to. You either read the fucking code and keep a mental model of how the system works in your head, or you have an overstated confidence in your ability to reason about the system (and this has been a problem well before LLMs).
Delving into the code is not the same thing as reading every line.
What will the path to a tech lead look like when it's no longer paved by thousands of hours of experience internalizing code? How do future tech leads avoid becoming like people who never got comfortable with fractions because they offloaded all arithmetic to calculators from an early age?
My hope/hunch is that the kids will be alright. Not learning effectively is a choice: if you want to get good, the paths to getting good are all still available to you.
We have never had as abundant a supply of tools to help us learn our craft. I expect that many people will thrive.
People who are a bit lazy and prone to cheating will be able to hurt themselves even more.
I had agents deep diving OpenCode source over the weekend, writing research docs, and helping me with a set of plugins. I learned quite a bit from the process and markdown walls, enough so that in later sessions I was able to point the model at where it was conflating concepts and help narrow its search.
Ai can help you learn if you are intentional about it.
The plural of anecdote is not fact.
Nor is it data!
This is the kind of stuff AI-skeptics have warned about daily on this very site for the last year or two. We are met with "you'll be left behind" and "you're just coping" and "the future is coming".
This shit is blatantly obvious. And if people keep pushing and pushing for LLMs to generate things that are actually used directly, the problem will grow so large that most people will barely remember a time when their job was something that could be understood.
This is why generative AI sucks.
This article assumes that this problem did not exist before AI. Especially in large companies like Google, the number of people who actually knew what they were doing was relatively small. The vast majority were just piggybacking on other people's work, which I absolutely hate. At least AI has made this obvious, and the difference between people is now their taste, which, for the majority of people, is bad news.
This! I work for a company that has several systems that are older than 20 years and no one really knows how they work. We are using AI to actually get insights in how they really works, to be able to rewrite and modernize the applications.
I can't read the source code since it's 8 million lines of code and written in a programming language I don't know and in a language I don't speak.
> I can't read the source code since it's 8 million lines of code and written in a programming language I don't know and in a language I don't speak.
^ This is the real world. Some comments talked about how maintainability is king and you just can't keep a mental model together of what the LLM produced. In real life software there is no single person with a mental model of how the system works end to end. In the most ideal scenario you have an architecture diagram, some readme's, and a runbook of how to use the system or get it running in a dev environment. Everything else is manually tracing through mountains of code of wildly varying quality.
coding harnesses are god sent tools when it comes to analysis and maintainability of existing code bases (including code they have produced).
>We’ve had to know everything about the product/business from day 1
I know this is being hyperbolic but I thought this was an odd post to include. I've met plenty of data engineers that don't have great knowledge of the business/product and SWEs that do have that. ¯\_(ツ)_/¯
This problem has existed long before AI became useful. As technology becomes more accessible, less knowledge is required to operate it. As that knowledge becomes less necessary, fewer people acquire it. Eventually the abstraction becomes so effective that entire layers of knowledge disappears from common practice.
As a person who's a solid generalist with over 30 years in various roles, I am completely and utterly shocked at how little foundational knowledge people in "senior" roles possess across a wide variety of technical fields. I'm often treated like some wizard or oracle for knowing things that everybody in the field used to know, I just haven't retired yet.
AI didn't create this phenomenon, it's just the latest (and probably the fastest) iteration of it. I recall another particularly large iteration happened when Windows NT 4.0 Server saw mass adoption. Suddenly, people who were effectively IT technicians were now sysadmins.
Edit: grammar
And, to be clear, I don't really place any judgments on this.
Unarguably, the world's technical capabilities have increased by this shift (while decreasing the required technical understanding required of the people managing it). Albeit, with some security implications.
The job of a software engineer has always been to turn a real-world idea or problem into a precise specification a computer can execute.
That specification used to be assembler, then higher-level languages like C and Python. Now it might be detailed Markdown files that AI turns into code.
So our understanding moves from the C/Python level to the Markdown specification level. We still need to understand exactly what we're building and why.
The hard part was never translating the spec into machine code. The hard part is defining the right spec. That is still the job of the software engineer.
In some ways I'm sympathetic to this: after all I'm trying to make money by being that person who does know stuff. But honestly that capability never seemed to be terribly valued by the people with the money. Anyway, it would be nice to see some examples where AI tools didn't "know" about architecture or intent. My experience has been they know plenty. More than most human developers. They do tend to follow certain fashions, which can be overridden by debate, or by introducing new fashion via skill text. But that's today. Surely these tools are only going to get much better in the next 10-20 years. I'm skeptical there's going to be a line of thinking that "it's smart but doesn't know the things humans know" long term. Whatever it is that we think only humans know today will be learned by the tools tomorrow.
"People are working 12 to 13 hours a day just to press enter. Nobody is reading anything."
Really? Seems like they're not making proper use of the tool. I've been reading MORE not less, and also learning more along the way. Just hitting "enter" is a choice, these tools are so powerful if you invest your curiosity, time and experience.
Sounds like they don't care about what they're doing in the first place, writing code by hand won't fix that.
People don't get fired for dishonesty or persistent failure like you'd expect.
Quite the contrary, they get promoted with salary raises
This is almost the same as saying "The problem is not the fentanyl, but that everyone keeps getting addicted or dying." In a very literal sense, yes, absolutely. There is a way to safely use fentanyl. But from a holistic point of view, the way to do so is so specific (ie multiple medical professionals prescribing and possibly even administering and monitoring vitals afterwards) and limited that for the general public is much more useful to just say "It's unsafe to use fentanyl unless following instructions from your medical provider." There's absolutely a way to use AI for coding that doesn't result in not actually knowing about the produced code, but it's so damn hard to get that right and you won't even realize you've fallen away from that knowledge until it's too late.
The point about PMs is I think illustrative of this issue. Yes, great PMs who actually understand the product they want to build and can then turn that into something using engineering teams as a black box system exist. But by and large, no they don't. A massive part of why waterfall fell out of fashion is because it turns out most orgs don't have and can't find PMs/PM adjacent people who can build that knowledge, and you can see how agile attempts to correct for this by turning a pipeline into an OODA loop. If you can prevent pipeline flushes, it will beat that OODA loop, but it turns out as an industry we can't actually prevent enough pipeline flushes for that to be the case and we can (for the most part) use that OODA loop to eventually iterate into something a paying customer might give us money for.
Part of the issue is that we're treating the AI like it's just another compiler or static code generator. It's not. The reason they're not is because they require a complete specification of what to output, ie the code. That level of specificity is what allows engineers to mostly stop caring about the assembler and just stay in their higher level code (the main exception being extremely hot loops in places too complex for the compiler to optimize well) and legitimately be able to say they understand the code without ever looking at the compiler/linker output. The entire point of AI for code generation is taking an incomplete specification and turning it into a completed one. If you review and understand every line of output the way almost no one did even before AI was a thing, then you're not going to fall into the trap of not knowing anything about your system anymore. But at the same time, you're not really going to get much benefit from AI either, because you're just replacing time spent coding with time spent reviewing code. It might even take you longer to review than it would to just code it yourself (i think every senior engineer who's mentored a junior dev knows this exact feeling).
To actually get the benefit from AI you have to let go of looking at the code output at all, because looking at that output is the slow part and because just looking and reviewing still won't give you the same knowledge as actually implementing. The passenger aviation industry handles the second issue by preferring/requiring manual control in critical areas (takeoffs and landing) and using simulators to extensively train for manual recovery outside of the abilities of autopilot. For developers the second bit is actually why reviewing AI output is a fool's errand - it will give you the false confidence in your understanding of the system. That doesn't mean we have to embrace vibe-coding or accept slop. It means changing the engineering process from one that deals with a deterministic system to a stochastic one. As an aside, I think this is why upper level managers and executive are leaning into AI code generation so hard - engineering was already a stochastic black box system to them.
[flagged]
[dead]
[dead]
[dead]
A man has to write program for himself and a man has to use the program. A man needs to define in code what the program does. If AI defines what code does then man has not written program for himself.
Too many young people in startups without enough experience.
I was in a Hackathon for students which quite a few staff, like myself, infiltrated. The results were completely unfair, staff and teams with staff (like mine) cleaned up the awards.
In my case I was working with a student who was much better at writing platformers in Unity than I was and an another student who could draw the art we needed even if she'd been trained to think every problem we had interacting with each other had something to do with "the patriarchy".
Myself I'd been in many startups where the game was make a half-baked demo that you could demo on stage and get people excited about it. So everything from presenting broken software on stage and making it look not just perfect but enticing and developing software that has the qualities it takes to present it that way was routine for me, the bit that isn't routine is onboarding unexperienced people to this life in two days.
The more things are unprecedented, the more you need a longer view with more experience.
she'd been trained to think every problem we had interacting with each other had something to do with "the patriarchy
Who is teaching these young girls such foolish things?
Of course its unfair. You're competing against students
I am interested in the question of "how does a group of staff and students outperform a group of either?"