> On the one hand, we’re pretty sure these systems don’t do anything like what humans do to produce or classify language or images. They use massive amounts of data, whereas we seem to use relatively little;
This isn't entirely correct; humans work with a roughly 16hr/day audio-visual feed running at very high resolution. That seems to be more data than ChatGPT was trained on. We spend less time looking at character glyphs, but the glyphs are the end of a process for building up language. When we say that cats sit on mats, that is linked to us having seen cats, mats and a lot of physics.
Although that strongly supports that humans learn in a way different from an LLM. And humans seem to have a strategy that involves seeking novelty that I don't think the major LLMs have cracked yet. But we use more data than they do.
Where such direct, numerical comparison is possible, it's my understanding that Weatherby is correct. Both adults and children are exposed to far fewer words than an LLM before achieving comparable fluency, and LLMs have, statistically or in aggregate, perfect recall, whereas humans do not.
I would claim that any reasonable "bright line" critique of AI is going to be a "remainder" theory. If one models and "tightly" articulates a thing that AI can't do, well, one has basically created a benchmark that systems are going to gradually (or quickly) move to surpassing. But the ability to surpass benchmarks isn't necessarily an ability to do anything and one can still sketch which remainders tend to remain.
The thing is, high social science theorists like the person interviewed, want to claim a positive theory rather than a remainder theory because such a theory seems more substantial. But for the above reason, I think such substance is basically an illusion.
Anecdotally, LLMs as a whole haven't made my life noticeably any better. I see some great use cases and some impressive demos, but they are just that. I look at how many things that LLMs have noticeably made worse and by my own impression it outweighs improvements.
- I asked when a software EOL will be, the LLM response (incorrectly) provided past tense for an event yet to happen.
- The replacement of Google Assistant with Gemini broke using my phone while locked and the home automation is noticeably less reliable.
- I asked an LLM about whether a device "phones home" and the answer was wrong.
- I asked an LLM to generate some boiler plate code with very specific instructions and the generated code was unusable.
- I gave critical feedback to a company that works with LLMs regarding a poor experience (along with some suggestions) and they seemed to have no interest in making adjustments.
- I've seen LLM note takers with incorrect notes, often skipping important or nuanced details.
I have had good experiences with LLMs and other ML models, but most of those experiences were years ago before LLMs were being unnecessarily shoved into every possible scenario.
At the end of the day, it doesn't matter if the experience is powered by an LLM, it matters whether the experience is effective overall (by many different measures).
I have an extensive, strong traditional CS background. I built and shipped a production grade SaaS in 2 months that has paying users. I've built things in day that would have taken me 3+ days manually. Through all of that, I hardly wrote a single line of code. It was all GPT-4.1 and o3.
Granted, I think you need quite a lot of knowledge and experience to know how to come up with coherent prompts and to be able to do the surgery necessary to get yourself out of a jam. But LLMs have easily 3x'd my productivity by very quantifiable metrics, like number of features shipped, for example.
I've noticed people who actually build stuff agree with me. That's because it's such a tremendous addition of value to our lives. Armchair speculators seem to see only the negative side.
What you're describing sounds to me like absolute hell on earth.
I'm not interested in reaching the finish line with maximum speed and bypassing the hard work of struggling with and solving problems myself.
Partly this is because working this way has real benefits that are difficult to quantify. One example: I've recently dumped an enormous amount of time into investigating performance problems in the tools my team use. I've spent more time making dumb mistakes than actually improving anything. I've also learned a tremendous amount, to the point that I was able to diagnose in seconds the cause of a serialization error in one of the tools we use for testing. Others were convinced that these crashes were expected. I was able to show them that, and why, this was wrong. I've likely saved multiple people on my team days' worth of confusion and struggling, because they were trying to solve the wrong problems. If they'd charged ahead with their intended fix, I suspect the result would have been an outage in a global service that has stringent requirements for availability.
An LLM may have been able to tell me in seconds how to solve the performance problem that started my investigations and dumb mistakes. But I'd have learned basically nothing.
If your goal is to make something specific and code is both the obstacle and the means of reaching that goal, sure, great, I'm glad LLMs work so well for you.
I just want to program. I want to solve problems, understand, and become better at working with programming languages, software, and systems. I haven't seen any evidence that LLMs will help me do this. As far as I can tell, they'd do the opposite. They strike me as a layer of awful, chipper bureaucracy between me and what I actually want to work on. I call this meeting-based programming -- and if that's what software engineering becomes, I'd rather leave the field than adopt that style of workong. And maybe that's a good thing. Maybe LLMs will enable more people to make better stuff faster, and maybe that'll be better for everyonr.
I suspect it won't though. I think it would be a dangerous Faustian bargain, and I'm pretty sure I'd rather die than cede intellectual work -- the thing I love most -- to a machine.
I agree there --- if you want to program, don't use an LLM!
Sometimes I do turn off the LLM on purpose because it is intrinsically enjoyable to program. I like to do things like Project Euler and I would never see the point of having an LLM do it for you, unless you were explicitly reading its code to try to learn something new.
Easier said than done though. Like many programmers, I'm finally facing upper management that expect everybody to start using AI tools. I'm half expecting to lose my job in the near future for refusing.
The future is coming, they keep telling us, (or it's already here) and if I don't actively strive to turn their fantasies into reality, I think they'll have no use for me.
I've noticed that people who build greenfield projects solo or on small teams love AI, while people who are stuck maintaining software written a decade ago haven't gotten the same value and are more critical of it.
You should see some of the security holes that copilot has tried to introduce into our code.
My hypothesis is that the greenfield projects make it easier to learn AI. I find it pretty easy to get value out of Cursor on 500k LOC legacy code bases, but I've also spent a few hundred hours on green field projects.
> I've noticed people who actually build stuff agree with me. That's because it's such a tremendous addition of value to our lives. Armchair speculators seem to see only the negative side.
I'm glad LLMs have "3x'd" your perceived productivity, but disguised insults are not necessary or constructive.
If your venture sustains, that's great and I do hope you share your deep insights when that happens.
> On the one hand, we’re pretty sure these systems don’t do anything like what humans do to produce or classify language or images. They use massive amounts of data, whereas we seem to use relatively little;
This isn't entirely correct; humans work with a roughly 16hr/day audio-visual feed running at very high resolution. That seems to be more data than ChatGPT was trained on. We spend less time looking at character glyphs, but the glyphs are the end of a process for building up language. When we say that cats sit on mats, that is linked to us having seen cats, mats and a lot of physics.
Although that strongly supports that humans learn in a way different from an LLM. And humans seem to have a strategy that involves seeking novelty that I don't think the major LLMs have cracked yet. But we use more data than they do.
Where such direct, numerical comparison is possible, it's my understanding that Weatherby is correct. Both adults and children are exposed to far fewer words than an LLM before achieving comparable fluency, and LLMs have, statistically or in aggregate, perfect recall, whereas humans do not.
I would claim that any reasonable "bright line" critique of AI is going to be a "remainder" theory. If one models and "tightly" articulates a thing that AI can't do, well, one has basically created a benchmark that systems are going to gradually (or quickly) move to surpassing. But the ability to surpass benchmarks isn't necessarily an ability to do anything and one can still sketch which remainders tend to remain.
The thing is, high social science theorists like the person interviewed, want to claim a positive theory rather than a remainder theory because such a theory seems more substantial. But for the above reason, I think such substance is basically an illusion.
Anecdotally, LLMs as a whole haven't made my life noticeably any better. I see some great use cases and some impressive demos, but they are just that. I look at how many things that LLMs have noticeably made worse and by my own impression it outweighs improvements.
- I asked when a software EOL will be, the LLM response (incorrectly) provided past tense for an event yet to happen. - The replacement of Google Assistant with Gemini broke using my phone while locked and the home automation is noticeably less reliable. - I asked an LLM about whether a device "phones home" and the answer was wrong. - I asked an LLM to generate some boiler plate code with very specific instructions and the generated code was unusable. - I gave critical feedback to a company that works with LLMs regarding a poor experience (along with some suggestions) and they seemed to have no interest in making adjustments. - I've seen LLM note takers with incorrect notes, often skipping important or nuanced details.
I have had good experiences with LLMs and other ML models, but most of those experiences were years ago before LLMs were being unnecessarily shoved into every possible scenario. At the end of the day, it doesn't matter if the experience is powered by an LLM, it matters whether the experience is effective overall (by many different measures).
My experience is the opposite.
I have an extensive, strong traditional CS background. I built and shipped a production grade SaaS in 2 months that has paying users. I've built things in day that would have taken me 3+ days manually. Through all of that, I hardly wrote a single line of code. It was all GPT-4.1 and o3.
Granted, I think you need quite a lot of knowledge and experience to know how to come up with coherent prompts and to be able to do the surgery necessary to get yourself out of a jam. But LLMs have easily 3x'd my productivity by very quantifiable metrics, like number of features shipped, for example.
I've noticed people who actually build stuff agree with me. That's because it's such a tremendous addition of value to our lives. Armchair speculators seem to see only the negative side.
What you're describing sounds to me like absolute hell on earth.
I'm not interested in reaching the finish line with maximum speed and bypassing the hard work of struggling with and solving problems myself.
Partly this is because working this way has real benefits that are difficult to quantify. One example: I've recently dumped an enormous amount of time into investigating performance problems in the tools my team use. I've spent more time making dumb mistakes than actually improving anything. I've also learned a tremendous amount, to the point that I was able to diagnose in seconds the cause of a serialization error in one of the tools we use for testing. Others were convinced that these crashes were expected. I was able to show them that, and why, this was wrong. I've likely saved multiple people on my team days' worth of confusion and struggling, because they were trying to solve the wrong problems. If they'd charged ahead with their intended fix, I suspect the result would have been an outage in a global service that has stringent requirements for availability.
An LLM may have been able to tell me in seconds how to solve the performance problem that started my investigations and dumb mistakes. But I'd have learned basically nothing.
If your goal is to make something specific and code is both the obstacle and the means of reaching that goal, sure, great, I'm glad LLMs work so well for you.
I just want to program. I want to solve problems, understand, and become better at working with programming languages, software, and systems. I haven't seen any evidence that LLMs will help me do this. As far as I can tell, they'd do the opposite. They strike me as a layer of awful, chipper bureaucracy between me and what I actually want to work on. I call this meeting-based programming -- and if that's what software engineering becomes, I'd rather leave the field than adopt that style of workong. And maybe that's a good thing. Maybe LLMs will enable more people to make better stuff faster, and maybe that'll be better for everyonr.
I suspect it won't though. I think it would be a dangerous Faustian bargain, and I'm pretty sure I'd rather die than cede intellectual work -- the thing I love most -- to a machine.
I agree there --- if you want to program, don't use an LLM!
Sometimes I do turn off the LLM on purpose because it is intrinsically enjoyable to program. I like to do things like Project Euler and I would never see the point of having an LLM do it for you, unless you were explicitly reading its code to try to learn something new.
Right on.
Easier said than done though. Like many programmers, I'm finally facing upper management that expect everybody to start using AI tools. I'm half expecting to lose my job in the near future for refusing.
The future is coming, they keep telling us, (or it's already here) and if I don't actively strive to turn their fantasies into reality, I think they'll have no use for me.
I've noticed that people who build greenfield projects solo or on small teams love AI, while people who are stuck maintaining software written a decade ago haven't gotten the same value and are more critical of it.
You should see some of the security holes that copilot has tried to introduce into our code.
My hypothesis is that the greenfield projects make it easier to learn AI. I find it pretty easy to get value out of Cursor on 500k LOC legacy code bases, but I've also spent a few hundred hours on green field projects.
> I've noticed people who actually build stuff agree with me. That's because it's such a tremendous addition of value to our lives. Armchair speculators seem to see only the negative side.
I'm glad LLMs have "3x'd" your perceived productivity, but disguised insults are not necessary or constructive.
If your venture sustains, that's great and I do hope you share your deep insights when that happens.