The phrasing “bias against AI” seems to beg the question here. The article takes it for granted that people are wrong to say they’re more interested in stories written by people than by AI, because they can’t tell the difference if they’re misled.
Compare with a hypothetical study saying: people say they prefer true inspirational personal stories to fake inspirational personal stories. But if you lie to them, they think the fake ones are just as good!
Obviously, this would not prove that they are “wrong” or “biased”. The whole point of stories written by people is that a _person_ wrote it, based on their actual human thoughts and experiences.
All the study objectively shows is people prefer stories they believe are written by humans.
You might find a similar effect with attractive authors vs ugly authors. If you show people the photos they probably prefer stories they believe are written by attractive authors.
If we call that bias in the second case, why not call it bias in the first case?
> The whole point of stories written by people is that a _person_ wrote it, based on their actual human thoughts and experiences.
I thought the whole point of stories was that they were entertaining or had some pertinent message. Truth doesn't have to come from someone's thoughts or experiences. Would you reject a math proof if it was generated by an AI?
I imagine at the time of the printing press, someone argued "The whole point of stories is that a person wrote it"
I think there's a fundamental difference between a story and a math proof. A math proof is mostly there to give you new knowledge.
While a story definitely can do that, for many people, they're also about human connection. Even if the story isn't true, you feel like you're getting a look inside the author's brain by discovering how they weave storylines together. All the life experience they've head that lead them to write this story.
If I was instead told the story was written by AI, I would be far less interested in what data it was trained on to be able to produce this story, because I cannot relate to an AI having any "experience" whatsoever.
Depends on what kind of story is the one we are talking about. Fictional stories just have to be entertaining as you say it. Non-fictional stories have to be entertaining, but also true.
Nobody complains if it turns out that the poor moisture farmer boy from the edge of the galaxy didn't really actually blow up the space station of the evil space empire. It is not that kind of story.
But other types of stories purport to tell about something which really happened. There just being merely entertaining is not enough.
> I thought the whole point of stories was that they were entertaining or had some pertinent message.
I guess it depends on what you as a reader value. The thing that makes any art valuable to me is that it is a human communication. Art made by a machine is far less valuable because, by definition, it isn't a human communication.
Others can value art differently, and they aren't wrong for doing so. That's part of what makes art special, that different people value aspects of it differently.
> Would you reject a math proof if it was generated by an AI?
That's an entirely different thing. A math proof isn't art, nor is it intended to be.
> The thing that makes any art valuable to me is that it is a human communication. Art made by a machine is far less valuable because, by definition, it isn't a human communication.
What is "human communication"? To me that is something that communicates a concept that resonates to a human. It doesn't mean that a human had to create it. For instance, an elaborate bird nest could be art because it communicates things like beauty, symmetry, function, etc.
The title is misleading, this study doesn't even attempt to actually answer preference of human vs AI written stories. It didn't even involve any human written stories!
All this study shows is that people rate an AI generate story worse when they know it's AI and say they'd pay less for it. Who knows what the relative rating or "payment" for a actual human written story would have been!
It reminds me of a genre of videos that would have you believe something is completely spontaneous and natural, but is actually entirely staged. The thing is only interesting under the assumption that it wasn't staged so they trick you into thinking it isn't.
Then someone in the comments inevitably says it is fake and everyone says of course it is. It's a skit, but it is still funny.
Knowing it is written by a computer and not by a human lowers the value of it regardless of how difficult it is to distinguish in my opinion. Thoughts and expressions by fellow humans gives something value and being written by a computer takes away a lot if not all of the purpose behind art.
It's an ok study to show people are predisposed to assume AI works are bad.
However the headline and article imply that people are wrong that the quality of the writing is bad. So a study "independent of the actual quality of the writing" goes directly against the conclusion they are pushing here. (the paper itself does not have this issue).
I had this experience while watching the last season of Black Mirror. I asked GPT for an episode and the outline came out much more interesting than anything on this season [0].
I may be in the minority here, or at least am willing to admit my preference, but I really don't care where my media comes from. I don't believe I have some special relationship with artists. I want people to have jobs and live a fulfilled life, but I don't feel any loyalty to something like listening to music made by humans or basing my consumer decisions based off things like royalty reimbursement rates. If I like it, and there's an incentive for its creation, I'm fine by that. People are still free to create art. I write pretty regularly but I don't expect to be able to do that for a profession, and that's fine.
I don't know if AI books are or will be as good or better than human written, but to me this is the problem - "Even though artificial intelligence is still in its infancy, AI-made books are already flooding the market." There is no scarcity problem in books. There are already way more that I would enjoy reading than I will ever actually read. It's already tough to prioritize which ones to get to without having vastly more to sort through. And people _enjoy_ writing books. I don't want to support automating something away that people enjoy doing, is produced in abundance, and is very low cost to obtain already.
Seriously, we have a Haagen-bot[0] problem ("the robot that eats ice cream so you don't have to") widespread across the field where people are trying to figure out how they can get their piece of the new ML world.
Part of it is that people aren't thinking about what's actually scarce. Let alone what a world more optimized for people might be like.
> I don't want to support automating something away that people enjoy doing, is produced in abundance, and is very low cost to obtain already.
>> "At the very least, this research doesn’t appear to be a reliable indicator of people’s willingness to pay for human-created art."
It's no secret that what does well commercially and what represents good craft doesn't always have much connection. I'm pro-GenAI and still prefer art made by a person. With the aid of AI is fine, but that connection through the choices they make in creating a thing matters to me.
Looking at the actual paper, the story might have biased the results. Would the results be the same if the story wasn't about a story written with the help of AI?
It should be noted the actual study focus is different from how it's framed by the article: "AI Bias for Creative Writing: Subjective Assessment Versus Willingness to Pay"
It seems like the focus was on commercial writing, not writing overall.
> To begin with, the group that knew the story was AI-generated had a much more negative assessment of the work, rating it more harshly on dimensions like predictability, authenticity and how evocative it is. [...]
> Nonetheless, participants were ready to spend the same amount of money and time to finish reading the story whether or not it was labeled as AI.
Have they previously shown that participants willingness to spend money is actually related to "dimensions like predictability, authenticity and how evocative it is" though?
As someone who will usually finish a bad film or book just because I want to know what happens, I'd probably 'pay' to finish even if I wasn't really enjoying it. Getting bored would make me stop, but made-for-tv-christmas-movies are incredibly predictable and I'll usually end up sucked into watching one or two of those over the holidays.
No it doesn't. They didn't test with stories written by humans, they only tested with stories written by AI. The title is completely misleading. Should be
"People say they prefer stories written by humans over AI, study talks about tangential issue"
So what are far-reaching implications? "GPT-4 can't generate sellable novels", "AI hate is universal", and "a proper double blind would have been even worse"?
The amount of work done that is truly new is laughably small. A large percentage of creative output is convergent (combining two existing things) or lateral (bringing an idea from one space to another).
It's a tiresome argument that "An AI can't possibly create anything" To strawman it there's a kind of fetishization of creativity (are your turds creative? is everything you do special?) and there is also Roger Penrose's bogus argument that he can do math because he's a thetan:
The most articulate version is something you hear from religious types which is "you have the power to create because God created you" but if God can delegate its creative power to you why can't you delegate your creative power to something you created?
"Uniqueness" is not real creativity, I mean, the Python script
from uuid import uuid4
print(uuid4())
creates something the world has never seen before and will never see again and would be completely indifferent to except for the fact of its uniqueness. Practically creativity has some element of being useful or expressive within the bounds of some constraints. If I was hiring you to be a writer or I commissioned an artwork I'd have some specification of what I want. Can an AI of some kind do that? Sure.
From the viewpoint of being compelling, some people are easy to satisfy. An AI might be able to satisfy them better you can because what that person needs might be squicky to you or at best tiresome and your "self" will get in the way of seduction, yet the AI can always respond the way it wants you to respond. I'm highly satisfied with the selections that are made by my YOShInOn smart RSS reader even though these are just selections and not "creations", however I'd imagine a system which you work with over a long time and that learns you preferences could "create" things that satisfy you much better than what you can find on the internet.
That's true but we have to ask do humans actually create engaging, new ideas? When you look at all the stories that have been created aren't they just slight variations on the same themes that have been around for, as near as we can tell, all time? When we look back at our historical writings it's the same themes at their core over and over it's just a different way to tell the story. Considering that we can look back 10,000 years in history and see the same common themes being retold it is quite likely that prior to that we were still telling the same stories.
We can find different ways to tell them which can be entertaining and it can feel new but when you examine it it's not really new.
> That's true but we have to ask do humans actually create engaging, new ideas
This is a red herring, and entirely irrelevant
The question of whether or not humans or AI "do" create new ideas is not relevant
The question is whether or not humans or AI can create new ideas
Humans quite obviously can, because humans did. Even if you see the same common themes for 10000 years, at some point those themes were created by humans. We must be capable of creating new ideas, even if we don't actually do it very much.
AI may not even be capable of creating new ideas. It probably isn't in its current state, and may not ever be
The red herring argument you're making here is your own. You were saying because ideas are here therefore humans must have been capable of creating them. There is no evidence of that other than saying because it's here therefore it had to come from a source of your own presupposing.
If you do not know where the original ideas came from or how they originated it is impossible to determine if AI is capable of doing the same thing. Even accepting the premise that humans did originate these ideas, there zero evidence that they did it spontaneously with no inputs. So with that premise in hand if AI is provided the same inputs it is likely they would produce the same outputs. Considering whatever those inputs where is completely unknown and unknowable at this point All of it is conjecture except for the fact that humans are effectively not creating net new ideas therefore there should be no expectation that AI is able to do that either.
> If you do not know where the original ideas came from or how they originated
What are you talking about
We do know where ideas came from! They came from humans! We sent humans to the moon using ideas about rocketry that a human invented, we know that because some of those people were alive when we were alive!
Look at the current AI LLM boom? The people responsible for that technology is alive today, you can talk to them!
I don't need to make a suggestion on how something came to be, It would be up to the person who says it happened in a certain way to provide the evidence to support that. It is not evidence in any meaningful way to say because it's here now therefore it must have been done by humans. It likely was but that doesn't mean it was entirely done by humans absent any external input.
Claiming that these ideas were the sole creation of humans with zero external input seems to be the same argument that intelligent design people make when they look at the complexity of biological systems. Are you a proponent of intelligent design? Because they draw the same conclusions that something of such complexity must have had an initial creator.
If you bother to read up a few comments you will see the entire point of this thread is that there is a base assumption that AI cannot create anything absent the supplied input that they've been given. So refocus on that and my response that humans have not been shown to create anything absent the supplied input that they have been given. To make an assumption that humans created something absent inputs would require an extraordinary amount of proof demonstrating that that's possible.
I have no idea how you would demonstrate that's possible because humans continually receive input from their surroundings all of the time.
(TL;DR - AI art is by design an attempt to reach the lowest common denominator, which makes it easy to digest, and a lot of people like stuff that's easy to digest.)
Someone on HN had a great summation in one of the art discussions that was essentially "Great human art rewards an interested viewer, AI art punishes them". Great art is full of interesting details, connections, and themes that the audience can get more from as their knowledge and attention increase. AI art is opposite. It's full of random details that make no coherent sense. A piece might look good at first glance but the more you pay attention the worse it becomes.
I think that concept applies equally well for writing. For random low value writing or an uninterested audience sure the AI writing will be fine, maybe even good. But as the desire for quality goes up the value of the AI product will go down.
This matches my own experience. Asking an AI to write a very brief story generally goes pretty well, but once you ask for more than a few simplistic paragraphs the quality falls off a cliff.
The phrasing “bias against AI” seems to beg the question here. The article takes it for granted that people are wrong to say they’re more interested in stories written by people than by AI, because they can’t tell the difference if they’re misled.
Compare with a hypothetical study saying: people say they prefer true inspirational personal stories to fake inspirational personal stories. But if you lie to them, they think the fake ones are just as good!
Obviously, this would not prove that they are “wrong” or “biased”. The whole point of stories written by people is that a _person_ wrote it, based on their actual human thoughts and experiences.
All the study objectively shows is people prefer stories they believe are written by humans.
You might find a similar effect with attractive authors vs ugly authors. If you show people the photos they probably prefer stories they believe are written by attractive authors.
If we call that bias in the second case, why not call it bias in the first case?
This isn't a study, it doesn't "objectively show" anything. It's an unreviewed discussion paper with questionable methodology.
The conclusion could just as easily be that an AI is better at writing engaging short-stories than the single author they chose.
> The whole point of stories written by people is that a _person_ wrote it, based on their actual human thoughts and experiences.
I thought the whole point of stories was that they were entertaining or had some pertinent message. Truth doesn't have to come from someone's thoughts or experiences. Would you reject a math proof if it was generated by an AI?
I imagine at the time of the printing press, someone argued "The whole point of stories is that a person wrote it"
I think there's a fundamental difference between a story and a math proof. A math proof is mostly there to give you new knowledge.
While a story definitely can do that, for many people, they're also about human connection. Even if the story isn't true, you feel like you're getting a look inside the author's brain by discovering how they weave storylines together. All the life experience they've head that lead them to write this story.
If I was instead told the story was written by AI, I would be far less interested in what data it was trained on to be able to produce this story, because I cannot relate to an AI having any "experience" whatsoever.
Depends on what kind of story is the one we are talking about. Fictional stories just have to be entertaining as you say it. Non-fictional stories have to be entertaining, but also true.
Nobody complains if it turns out that the poor moisture farmer boy from the edge of the galaxy didn't really actually blow up the space station of the evil space empire. It is not that kind of story.
But other types of stories purport to tell about something which really happened. There just being merely entertaining is not enough.
> I thought the whole point of stories was that they were entertaining or had some pertinent message.
I guess it depends on what you as a reader value. The thing that makes any art valuable to me is that it is a human communication. Art made by a machine is far less valuable because, by definition, it isn't a human communication.
Others can value art differently, and they aren't wrong for doing so. That's part of what makes art special, that different people value aspects of it differently.
> Would you reject a math proof if it was generated by an AI?
That's an entirely different thing. A math proof isn't art, nor is it intended to be.
> The thing that makes any art valuable to me is that it is a human communication. Art made by a machine is far less valuable because, by definition, it isn't a human communication.
What is "human communication"? To me that is something that communicates a concept that resonates to a human. It doesn't mean that a human had to create it. For instance, an elaborate bird nest could be art because it communicates things like beauty, symmetry, function, etc.
To me, it means communication from a human to a human. Something like an elaborate bird's nest can be beautiful, but it cannot be art.
The title is misleading, this study doesn't even attempt to actually answer preference of human vs AI written stories. It didn't even involve any human written stories!
All this study shows is that people rate an AI generate story worse when they know it's AI and say they'd pay less for it. Who knows what the relative rating or "payment" for a actual human written story would have been!
It reminds me of a genre of videos that would have you believe something is completely spontaneous and natural, but is actually entirely staged. The thing is only interesting under the assumption that it wasn't staged so they trick you into thinking it isn't.
Then someone in the comments inevitably says it is fake and everyone says of course it is. It's a skit, but it is still funny.
Knowing it is written by a computer and not by a human lowers the value of it regardless of how difficult it is to distinguish in my opinion. Thoughts and expressions by fellow humans gives something value and being written by a computer takes away a lot if not all of the purpose behind art.
It’s good. Because it shows human bias.
They have to construct the study independent of the actual quality of the writing. And it’s clear here that many people are just biased.
It's an ok study to show people are predisposed to assume AI works are bad.
However the headline and article imply that people are wrong that the quality of the writing is bad. So a study "independent of the actual quality of the writing" goes directly against the conclusion they are pushing here. (the paper itself does not have this issue).
Came here to say exactly this.
I had this experience while watching the last season of Black Mirror. I asked GPT for an episode and the outline came out much more interesting than anything on this season [0].
I may be in the minority here, or at least am willing to admit my preference, but I really don't care where my media comes from. I don't believe I have some special relationship with artists. I want people to have jobs and live a fulfilled life, but I don't feel any loyalty to something like listening to music made by humans or basing my consumer decisions based off things like royalty reimbursement rates. If I like it, and there's an incentive for its creation, I'm fine by that. People are still free to create art. I write pretty regularly but I don't expect to be able to do that for a profession, and that's fine.
[0] https://mleverything.substack.com/p/we-should-just-let-gpt-w...
I don't know if AI books are or will be as good or better than human written, but to me this is the problem - "Even though artificial intelligence is still in its infancy, AI-made books are already flooding the market." There is no scarcity problem in books. There are already way more that I would enjoy reading than I will ever actually read. It's already tough to prioritize which ones to get to without having vastly more to sort through. And people _enjoy_ writing books. I don't want to support automating something away that people enjoy doing, is produced in abundance, and is very low cost to obtain already.
Seriously, we have a Haagen-bot[0] problem ("the robot that eats ice cream so you don't have to") widespread across the field where people are trying to figure out how they can get their piece of the new ML world.
Part of it is that people aren't thinking about what's actually scarce. Let alone what a world more optimized for people might be like.
> I don't want to support automating something away that people enjoy doing, is produced in abundance, and is very low cost to obtain already.
Quoted for truth.
[0] https://www.savagechickens.com/2024/04/haagen-bot.html "the robot that eats ice cream so you don't have to"
>> "At the very least, this research doesn’t appear to be a reliable indicator of people’s willingness to pay for human-created art."
It's no secret that what does well commercially and what represents good craft doesn't always have much connection. I'm pro-GenAI and still prefer art made by a person. With the aid of AI is fine, but that connection through the choices they make in creating a thing matters to me.
Looking at the actual paper, the story might have biased the results. Would the results be the same if the story wasn't about a story written with the help of AI?
It should be noted the actual study focus is different from how it's framed by the article: "AI Bias for Creative Writing: Subjective Assessment Versus Willingness to Pay"
It seems like the focus was on commercial writing, not writing overall.
Title needs to be changed. It completely misrepresents this research. There was no comparison between human written and AI written stories.
> To begin with, the group that knew the story was AI-generated had a much more negative assessment of the work, rating it more harshly on dimensions like predictability, authenticity and how evocative it is. [...]
> Nonetheless, participants were ready to spend the same amount of money and time to finish reading the story whether or not it was labeled as AI.
Have they previously shown that participants willingness to spend money is actually related to "dimensions like predictability, authenticity and how evocative it is" though?
As someone who will usually finish a bad film or book just because I want to know what happens, I'd probably 'pay' to finish even if I wasn't really enjoying it. Getting bored would make me stop, but made-for-tv-christmas-movies are incredibly predictable and I'll usually end up sucked into watching one or two of those over the holidays.
All in all it seems a rather weird study.
No it doesn't. They didn't test with stories written by humans, they only tested with stories written by AI. The title is completely misleading. Should be
"People say they prefer stories written by humans over AI, study talks about tangential issue"
So what are far-reaching implications? "GPT-4 can't generate sellable novels", "AI hate is universal", and "a proper double blind would have been even worse"?
I question whether AI can create engaging new ideas. Isn't everything it outputs already based on existing data?
The amount of work done that is truly new is laughably small. A large percentage of creative output is convergent (combining two existing things) or lateral (bringing an idea from one space to another).
Whereas this comment, seen fifteen thousand plus times on this very website, is evidence of ... your human creativity?
It's a tiresome argument that "An AI can't possibly create anything" To strawman it there's a kind of fetishization of creativity (are your turds creative? is everything you do special?) and there is also Roger Penrose's bogus argument that he can do math because he's a thetan:
https://en.wikipedia.org/wiki/The_Emperor%27s_New_Mind
The most articulate version is something you hear from religious types which is "you have the power to create because God created you" but if God can delegate its creative power to you why can't you delegate your creative power to something you created?
"Uniqueness" is not real creativity, I mean, the Python script
creates something the world has never seen before and will never see again and would be completely indifferent to except for the fact of its uniqueness. Practically creativity has some element of being useful or expressive within the bounds of some constraints. If I was hiring you to be a writer or I commissioned an artwork I'd have some specification of what I want. Can an AI of some kind do that? Sure.From the viewpoint of being compelling, some people are easy to satisfy. An AI might be able to satisfy them better you can because what that person needs might be squicky to you or at best tiresome and your "self" will get in the way of seduction, yet the AI can always respond the way it wants you to respond. I'm highly satisfied with the selections that are made by my YOShInOn smart RSS reader even though these are just selections and not "creations", however I'd imagine a system which you work with over a long time and that learns you preferences could "create" things that satisfy you much better than what you can find on the internet.
That's true but we have to ask do humans actually create engaging, new ideas? When you look at all the stories that have been created aren't they just slight variations on the same themes that have been around for, as near as we can tell, all time? When we look back at our historical writings it's the same themes at their core over and over it's just a different way to tell the story. Considering that we can look back 10,000 years in history and see the same common themes being retold it is quite likely that prior to that we were still telling the same stories.
We can find different ways to tell them which can be entertaining and it can feel new but when you examine it it's not really new.
> That's true but we have to ask do humans actually create engaging, new ideas
This is a red herring, and entirely irrelevant
The question of whether or not humans or AI "do" create new ideas is not relevant
The question is whether or not humans or AI can create new ideas
Humans quite obviously can, because humans did. Even if you see the same common themes for 10000 years, at some point those themes were created by humans. We must be capable of creating new ideas, even if we don't actually do it very much.
AI may not even be capable of creating new ideas. It probably isn't in its current state, and may not ever be
The red herring argument you're making here is your own. You were saying because ideas are here therefore humans must have been capable of creating them. There is no evidence of that other than saying because it's here therefore it had to come from a source of your own presupposing.
If you do not know where the original ideas came from or how they originated it is impossible to determine if AI is capable of doing the same thing. Even accepting the premise that humans did originate these ideas, there zero evidence that they did it spontaneously with no inputs. So with that premise in hand if AI is provided the same inputs it is likely they would produce the same outputs. Considering whatever those inputs where is completely unknown and unknowable at this point All of it is conjecture except for the fact that humans are effectively not creating net new ideas therefore there should be no expectation that AI is able to do that either.
> If you do not know where the original ideas came from or how they originated
What are you talking about
We do know where ideas came from! They came from humans! We sent humans to the moon using ideas about rocketry that a human invented, we know that because some of those people were alive when we were alive!
Look at the current AI LLM boom? The people responsible for that technology is alive today, you can talk to them!
Have you been reading the prior comments or did you just come to spout something off about space travel that was totally irrelevant?
How are you suggesting these ideas are here if humans didn't create them?
I don't need to make a suggestion on how something came to be, It would be up to the person who says it happened in a certain way to provide the evidence to support that. It is not evidence in any meaningful way to say because it's here now therefore it must have been done by humans. It likely was but that doesn't mean it was entirely done by humans absent any external input.
Claiming that these ideas were the sole creation of humans with zero external input seems to be the same argument that intelligent design people make when they look at the complexity of biological systems. Are you a proponent of intelligent design? Because they draw the same conclusions that something of such complexity must have had an initial creator.
If you bother to read up a few comments you will see the entire point of this thread is that there is a base assumption that AI cannot create anything absent the supplied input that they've been given. So refocus on that and my response that humans have not been shown to create anything absent the supplied input that they have been given. To make an assumption that humans created something absent inputs would require an extraordinary amount of proof demonstrating that that's possible.
I have no idea how you would demonstrate that's possible because humans continually receive input from their surroundings all of the time.
The problem of originality has been of intense debate since at least the romantic era.
There are people who have argued that human creativity is nothing more than outputs based on existing data (experience).
A cynical but imo largely correct take on this phenomenon: "People prefer A.I. art because people prefer bad art" https://maxread.substack.com/p/people-prefer-ai-art-because-...
(TL;DR - AI art is by design an attempt to reach the lowest common denominator, which makes it easy to digest, and a lot of people like stuff that's easy to digest.)
Someone on HN had a great summation in one of the art discussions that was essentially "Great human art rewards an interested viewer, AI art punishes them". Great art is full of interesting details, connections, and themes that the audience can get more from as their knowledge and attention increase. AI art is opposite. It's full of random details that make no coherent sense. A piece might look good at first glance but the more you pay attention the worse it becomes.
I think that concept applies equally well for writing. For random low value writing or an uninterested audience sure the AI writing will be fine, maybe even good. But as the desire for quality goes up the value of the AI product will go down.
This matches my own experience. Asking an AI to write a very brief story generally goes pretty well, but once you ask for more than a few simplistic paragraphs the quality falls off a cliff.
"If you can’t tell, does it matter?"
Favorite quote from Westworld ;)
[flagged]