This was a fantastic article and beautifully captured something I had been circling but hadn't quite put into words:
> The notetaking people—and I say this with all the love in the world—are never, like, a researcher at the cutting edge of their field, building this vast cathedral of knowledge, note-by-note, so they can derive new insights. Never a historian who has to read tens of millions of words across thousands of sources to synthesize the life of some historical person. It’s never someone doing something hard. It’s always some blogger. Their “digital garden” is about how to keep a digital garden. It’s very solipsistic: there’s no output, no deliverables. The deliverable is you take a screenshot of your Obsidian graph and tweet about it to show off how much it looks like an incomprehensible ball of twine.
> Sometimes, tools don’t move the needle because there’s no needle to move.
It reminds me of something my old CS mentor, now elderly, had said about LLMs a few months ago: "it's a force multiplier, but there has to be some force to multiply."
This likewise is a basic fact I encounter over and over:
> Knowledge is another limiting factor. I find that even very educated people tend to underrate the importance of knowledge. A lot of people have this attitude that you can just Google everything just-in-time as it comes up. Like Babbage, I can’t rightly apprehend the confusion of ideas that would lead someone to think this.
"Most people are not autodidacts because most people have no material reason to learn a specific topic (i.e. their job does not require it) and the problem with learning for the sake of learning is opportunity cost: there is no a priori reason to learn one thing over another, so better to do nothing and wait for something to appear which actually grabs your interest. Again, this is likely rational! Could you imagine if you found everything interesting? You’d spend years living in a basement curating a wiki of late Soviet military hardware or something."
This is all very right, but I'd like to add this: As your capacity to deal with abstraction (which is a function of intelligence and executive function and, to some extent, knowledge) grows, you become more and more constrained by the extent to which you can manipulate symbols. AI-based solutions for that are potentially really powerful, and that's the mechanism through which, as TFA says, "intelligent, educated people with working reward circuitry stand to gain more from AI."
And I'd also add that AI strongly disaggregates the returns to different levels of the capability to deal with abstraction -- higher levels get more, lower levels get less -- rather than uniformly boosting returns across the board (unfortunately). Of course, this has been the trend of information technology since at least the '80s, but now the slice at the top is really small and the returns very high.
> Could you imagine if you found everything interesting? You’d spend years living in a basement curating a wiki of late Soviet military hardware or something.
This was a good read but this felt like a personal attack :)
As a product manager I have found people's understanding of what they want to be tenuous at best. They know they want it to be easier, less frustrating, and do the darn thing they want. But collecting the input of all the people with the power to make choices, boiling that all down, and actually figuring out what will work is a total bottle neck. Technology and programming have hardly ever actually been a blocker. Legacy systems, conflicting requirements, and ever shifting choices represent a much harder problem.
As an aside, the term 'human bottleneck' resonated with me because I've been trying to describe situations that look like bottlenecks but are actually just "the process everything else is working towards".
I think using 'bottleneck' to describe a process that isn't amenable to automation frames the situation incorrectly in my head. 'positive bottleneck' isn't any help.
Great read, I feel many of the topics discussed firsthand.
It gives me some relief to know there are others out there who struggle with some similar issues, but I was hoping the piece may offer some guidance, but sadly I do not feel it has.
I completely agree with the author. As I've been saying repeatedly: AI is not magic. It's just another tool. An amazing search engine that simplifies a lot of mundane manual work to find what you're looking for. But of course, you have to have some idea of what you're looking for.
> Rather: if you don’t have the knowledge, you don’t understand the question, or why it matters, or how to judge the answers, and you won’t ever think to ask. You’re in a completely different continent from “writing the prompt”.
Reminds me of a time someone asked an influencer how to write better blog posts with LLMs. They responded "oh, it's easy", and then crafted a very specific and niche prompt about comparing and contrasting very specific things with technical details they clearly had deep knowledge in.
Excellent read - thanks! Always felt there was a ‘self-licking ice cream cone’ at the heart of the present moment. If only AI had better context than the messy reality that is human knowledge…oh wait, we have AI to help with that problem
This was a fantastic article and beautifully captured something I had been circling but hadn't quite put into words:
> The notetaking people—and I say this with all the love in the world—are never, like, a researcher at the cutting edge of their field, building this vast cathedral of knowledge, note-by-note, so they can derive new insights. Never a historian who has to read tens of millions of words across thousands of sources to synthesize the life of some historical person. It’s never someone doing something hard. It’s always some blogger. Their “digital garden” is about how to keep a digital garden. It’s very solipsistic: there’s no output, no deliverables. The deliverable is you take a screenshot of your Obsidian graph and tweet about it to show off how much it looks like an incomprehensible ball of twine.
> Sometimes, tools don’t move the needle because there’s no needle to move.
It reminds me of something my old CS mentor, now elderly, had said about LLMs a few months ago: "it's a force multiplier, but there has to be some force to multiply."
This likewise is a basic fact I encounter over and over:
> Knowledge is another limiting factor. I find that even very educated people tend to underrate the importance of knowledge. A lot of people have this attitude that you can just Google everything just-in-time as it comes up. Like Babbage, I can’t rightly apprehend the confusion of ideas that would lead someone to think this.
"Most people are not autodidacts because most people have no material reason to learn a specific topic (i.e. their job does not require it) and the problem with learning for the sake of learning is opportunity cost: there is no a priori reason to learn one thing over another, so better to do nothing and wait for something to appear which actually grabs your interest. Again, this is likely rational! Could you imagine if you found everything interesting? You’d spend years living in a basement curating a wiki of late Soviet military hardware or something."
uhoh
i've been spotted
This is all very right, but I'd like to add this: As your capacity to deal with abstraction (which is a function of intelligence and executive function and, to some extent, knowledge) grows, you become more and more constrained by the extent to which you can manipulate symbols. AI-based solutions for that are potentially really powerful, and that's the mechanism through which, as TFA says, "intelligent, educated people with working reward circuitry stand to gain more from AI."
And I'd also add that AI strongly disaggregates the returns to different levels of the capability to deal with abstraction -- higher levels get more, lower levels get less -- rather than uniformly boosting returns across the board (unfortunately). Of course, this has been the trend of information technology since at least the '80s, but now the slice at the top is really small and the returns very high.
> Could you imagine if you found everything interesting? You’d spend years living in a basement curating a wiki of late Soviet military hardware or something.
This was a good read but this felt like a personal attack :)
As a product manager I have found people's understanding of what they want to be tenuous at best. They know they want it to be easier, less frustrating, and do the darn thing they want. But collecting the input of all the people with the power to make choices, boiling that all down, and actually figuring out what will work is a total bottle neck. Technology and programming have hardly ever actually been a blocker. Legacy systems, conflicting requirements, and ever shifting choices represent a much harder problem.
As an aside, the term 'human bottleneck' resonated with me because I've been trying to describe situations that look like bottlenecks but are actually just "the process everything else is working towards".
I think using 'bottleneck' to describe a process that isn't amenable to automation frames the situation incorrectly in my head. 'positive bottleneck' isn't any help.
Great read, I feel many of the topics discussed firsthand.
It gives me some relief to know there are others out there who struggle with some similar issues, but I was hoping the piece may offer some guidance, but sadly I do not feel it has.
I completely agree with the author. As I've been saying repeatedly: AI is not magic. It's just another tool. An amazing search engine that simplifies a lot of mundane manual work to find what you're looking for. But of course, you have to have some idea of what you're looking for.
The missing part is usually the real workflow. If there is no output and no cost for being wrong, the AI tool just becomes another toy to maintain.
> Rather: if you don’t have the knowledge, you don’t understand the question, or why it matters, or how to judge the answers, and you won’t ever think to ask. You’re in a completely different continent from “writing the prompt”.
Reminds me of a time someone asked an influencer how to write better blog posts with LLMs. They responded "oh, it's easy", and then crafted a very specific and niche prompt about comparing and contrasting very specific things with technical details they clearly had deep knowledge in.
Excellent read - thanks! Always felt there was a ‘self-licking ice cream cone’ at the heart of the present moment. If only AI had better context than the messy reality that is human knowledge…oh wait, we have AI to help with that problem