I've always found this interesting. Think+transmit seems more likely to be the bottleneck vs receive, given that we can easily parse most podcasts etc at 3-4X speed. If being understood by everyone wasn't required, I wonder if one could learn to boost both send and receive rates?
But what you said made me curious. I listened to this podcast at 3x. I was able to understand all the words, but my conceptual understanding decreased. I also have to listen actively -- not passively.
At 4x, I could only understand the shape of the sentences but could no longer make out the words. But I turned on the captions and found I could keep up. Turns out reading at 4x works, listening at 4x doesn't.
English is also spoken with different prosodies and cadences. For instance, I can understand Singaporean English perfectly, but it's less amenable to being sped up. I tried listening to this lecture in Singaporean English in 3x and found that I could barely understand it.
This suggest that maybe human minds are able to form ideas at a certain speed, and language has evolved to convey these ideas and not be a bottleneck, but there is no reason to improve language beyond this.
Hm, I’ve always struggled with the other bottleneck - speaking. I tend to slur my words because my brain is forming the thoughts faster than I can speak. I could perhaps speed up my speech to match my brain but I’d sound like a maniac.
Don’t get me started on writing. My letters will often transition halfway into a letter thats 3 words ahead.
Likely so. I've noticed that those whose first language is one of those with fewer vowels - and thus must have more syllables per second to convey the same information rate- sometimes speak English much faster than I (as a monolingual native speaker) find normal.
3-4x! Are you the flash? I usually run things about 1.5x when I am commuting. 3x would require laser focus without distractions. Or you mean more a “can technically absorb the language being spoken” sense?
Not OP but listen to podcasts at highly accelerated settings:
The information density of ‘two dudes talking’ or any unscripted format is very low, so it time-compresses well. Specific podcasts, typically scripted monologues with technical content, such as Causality [0] (recommended!), I need to listen to much slower. Ditto if it is in an accent which isn’t mine, which slows my comprehension. I also slow the speed if I’m driving. So, yes, it takes mental overhead, but is doable. Go one click at a time and it will feel natural.
I suppose the format is a huge differentiator. I exclusively listen to highly produced content which has essentially no dead time. The content is already a compressed transmission of information.
I would say that the ability and speed to receive and process information in a timely manner also greatly depends on the density and complexity of the material.
I have always wondered if transmission bitrate remains the same for the same language, spoken at seemingly different speeds.
There is that stereotype that french-speaking Swiss people speak slower than French ones. In my experience I find it valid, but maybe I am wrong.
If this is accurate, I am wondering if Swiss people transmit information at the same rate as French people.
It could be that they use more precise words on average, that convey more information, even if spoken more slowly, and keeping transmission rate identical. Or that body language, intonations are richer (non verbal).
Or the spoken transmission rate may actually be slower, but as the article describes, bottleneck is about structuring the ideas, and Swiss speakers, on average, may be more efficient/deliberate at that, instinctively/culturally.
I don't have enough experience speaking with Swiss nationals to verify my anecdotal theories... if anyone can chime in...
I'm sorry, but this is essentially racism prettied up. The research is about language bitrate not about regional speaking rate variations within a language.
Tangentially, I'm relatively confident that what you're experience has provided you is simply confirmation bias. Unless French is not their first language.
I agree that the research is about various languages bitrates. My point is a bit different, yes, it is about total information bitrate.
Maybe I am racist, but I am not sure how the speed at which one speaks is a racist trope. I sure do not look down on the Swiss, they're a pretty successful nation.
Question for you: what is your relative confidence based on?
On the positive, your comment invited me to check Wikipedia and I was surprised to see there are actually studies about some of this. They seem to confirm the stereotype. Belgian and Parisian seem to have a higher syllable/ms rate than several Swiss counties, with some caveats. [1]
But no, it does not talk about total information bitrate, that is much more subjective. Maybe if you are a "frontalier" and have some anecdotal experience with it, I'm curious.
"Après examen des différentes études, le stéréotype des Suisses qui articulent plus lentement que les Parisiens est confirmé, à quelques exceptions près"
The paper does not seem to support that human speech has a universal transmission rate or that every spoken language has a universal information rate. They showed that information rate varied by individual and by language, just that it varied less than the syllable rate.
This was also bounded by a reading task, so the performance shown per language could be influenced by the average reading skills of people who speak those languages. They also asked them to pronounce differently than they normally would.
If you take the 17 languages they tested and get the average between them, you get 39bits/s. For English and French, the information rate they recorded was higher, with an average closer to ~45bits/s (just eyeballing their chart). Their results also showed Thai at ~35bits/s. A 10bits/s swing from median to median is pretty huge.
From the paper:
"We collected recordings of 170 native adult speakers of the aforementioned 17 languages, each reading at their normal rate a standardized set of 15 semantically similar texts across the languages (for a total amount of approximately 240,000 syllables). Speakers became familiar with the texts, by reading them several times before being recorded, so that they understand the described situation and minimize reading errors"
"Together, our findings show that while there is wide interspeaker variation in speech and IRs (information rates), this variation is also structured by language. This means that an individual’s speech behavior is not entirely due to individual characteristics but is further constrained by the language being spoken."
"However, languages seem to stably inhabit an optimal range of IRs, away from the extremes that can still be available to individual speakers. Languages achieve this balance through a trade-off between ID (information density) and SR (syllable rate), resulting in a narrower distribution of IRs compared to SRs. In the introduction, we rhetorically asked whether too low or too high an IR would impede communicative and/or cognitive efficiency. Our results here suggest that the answer to both questions is positive and that human communication seems to avoid two extreme sociolinguistic profiles: on the one hand, high ID languages spoken fast by their speakers (“high-fast”), and, on the other, low ID languages spoken slowly by their speakers (“low-slow”)."
This definitely fits with my experience as a language learner studying French and Japanese.
Japanese is definitely a faster spoken language than French, but French words tend to be a lot more verbose and packed with meaning. For Japanese speakers to communicate as much meaning as French speakers, they would need to speak faster.
I wonder, does this account for compression? Are the bits counting a concrete representation of ideas transmitted? Or are bits counting a simple abstraction over syllables, similar to a token? Language is compression. And while humans may transmit at a universal rate, maybe some languages/cultures have a more dense compression of ideas than others? Like in Turkish, how there's a word for "moonlight on the water" https://ihearthesamewinds.wordpress.com/2011/11/01/142/.
>Each participant read aloud 15 identical passages that had been translated into their mother tongue. After noting how long the speakers took to get through their readings, the researchers calculated an average speech rate per language, measured in syllables/second.
Maybe more interesting: what’s the average reading speed per language, and what’s the variation? I know that reading speed varies a lot ( also depends on topic, a math textbook reads slower than an Ian Fleming book).
Are simultaneous translator’s brains different? They need to process two languages at once, and I never could do that even though I’m fluent in more than one language.
I suspect that for someone who reads a decent amount reading speed is also bottlenecked by the rate by which we comprehend the ideas being communicated and not the rate at which we recognize the words. I recently did a bunch of reading speed tests in all the languages I understand. I have like two orders of magnitude more experience in reading German and English than in Dutch, French and Spanish. In the former group word recognition is automatic while in the later I have to concentrate and sometimes even sound out words in my mind yet my actual reading speed for my "strong" languages is only about twice that of my "weak" languages.
Translating is a completely different skill that you have to train on top of being fluent in more than one language. The way I translate is by sort of forgetting something in one language and then remembering it in another. It's a slow and awkward process but I suspect if I did this for like a thousand hours hearing something and then repeating it in another language would be as easy as switching the language in which I'm thinking. I think the real difficulty of simultaneous translation comes from having to speak while you're listening. Consider recording your response to an audio message while listening to it, that would also be very difficult but there is only one language involved.
Yeah. Reading rate varies greatly based on the complexity of the material and how familiar one is with it. It's not just the raw information density, but how much information must be retained and what that entails.
Professionals in a field very often communicate much faster with each other because so much stuff is a standard state plus deviations from that, whereas the person who doesn't know the field can't compress it like that. It's been studied with chess--good chess players are better at memorizing *sensible* board positions, but lose most of that advantage when confronted with nonsense.
Maybe not just spoken language. I would suggest sign language has a similar bit rate as spoken. The evidence is that a sign interpreter conveys about the same information in a conversation in the same time.
The bits are defined in another paper by the authors using "syntagmatic density of information ratio (SDIR)" which seems to be related to the number of syllables required to convey the same information in different languages, using Vietnamese as the baseline.
this study has a grand total sample size of 170 (that's 10 speakers per language), it also measures syllable rate (i.e. syllables/second times diversity of syllables) as a proxy for information. this is pretty far from how people will intuitively conceptualize "information".
The belief that that these 170 speakers, each speaking many long passages in 17 languages would converge to 39bps through dumb luck is unlikely enough to be almost mystical.
edit: If I hear a case that is not insane as to how the numbers could somehow determine themselves (through bad math) or be p-hacked, I'd happily consider it. Instead people are acting like taking sentences of equivalent meaning and counting the syllables to determine information density is somehow laughably naïve.
I would expect you can only transmit so fast in morse code given the need for the dots and dashes to be clearly distinguished from each other and identifiable by the recipient.
Of course, if you know both ends are computers you can just transmit in some other encoding at a much higher rate.
The point is, Morse code is as many "bits per second" as you want. You could send Morse code at 10 gigabits per second if you wanted. It is not meaningful to say that Morse code implies a particular data rate.
Historically the metric for Morse code is words per minute. Morse is similar to a Huffman code where common letters are allocated fewer elements, so it's not very meaningful to talk about "bits per second" with respect to Morse even if you do specify the number of words per minute. The number of "bits" will vary based on the letters being transmitted.
Oh, come on, this is just being coy for no reason. Given the context, it is abundantly clear they mean "normal, human operated morse code".
A skilled operator is around 30 WPM. The average English word is 5 (rounded up) characters. Add one character for the space. That's 180 characters per minute, or 3 character per second. With 37 characters available in morse code, that's log_2(37) or 5.2 bits per charater.
So 15.6 bits per second. Just under half of the 39 bits they got for speech, like they said.
I mean if you want that level of pedantry "I" can't send Morse code at 10 gigabits per second. I can make a computer transmit it at that speed, but I am not personally sending it that rate. And, because one generally needs a machine to transmit Morse code, one cold argue that "I" never send Morse code ever.
As Morse speeds up, you stop relying on individual dots and dashes and begin recognizing common combinations of letters. Faster still and you are mainly hearing word stems and suffixes.
The faster the information comes at you, the less important any particular bit is, because you have more context with which to autocorrect.
There is also a lot of nonverbal data. Imagine the horror of discovering your conversational ML build could hold a plausible verbal conversation only guessing 58% of spoken words accurately... then realizing humans likely fair much worse. =3
its not the language that communicated its the people. You recruit mediocres in different languages they all communicated in the same mid , 39 bits or whatever. Let two geniuses have a chat or even two teenagers - its a different ball game than 39 bits
I've always found this interesting. Think+transmit seems more likely to be the bottleneck vs receive, given that we can easily parse most podcasts etc at 3-4X speed. If being understood by everyone wasn't required, I wonder if one could learn to boost both send and receive rates?
I usually listen at 1.5x passively.
But what you said made me curious. I listened to this podcast at 3x. I was able to understand all the words, but my conceptual understanding decreased. I also have to listen actively -- not passively.
https://www.youtube.com/watch?v=9SFkwdm0PP0
At 4x, I could only understand the shape of the sentences but could no longer make out the words. But I turned on the captions and found I could keep up. Turns out reading at 4x works, listening at 4x doesn't.
English is also spoken with different prosodies and cadences. For instance, I can understand Singaporean English perfectly, but it's less amenable to being sped up. I tried listening to this lecture in Singaporean English in 3x and found that I could barely understand it.
https://www.youtube.com/watch?v=LA1dnUYzRWU
Blind people often go upto 600 words per minute and more with text-to-speech, which is, I think, would be an equivalent to 5x and more.
In fairness, they cannibalize their visual cortex to do that:)
There's the cue for my biennial reread of Peter Watts' Blindsight.
Went into my list. Appreciated :)
Did you read his followup Echopraxia? How would you say it compared to Blindsight?
I'm pretty sure you can train yourself to do the same, but the effort probably doesn't seem worth it for most people.
But its robotic and predictable, human speech is not.
> I usually listen at 1.5x passively.
Eminem's song "Rap God" has a segment where he goes at about 6 words per second (@4m23s):
* https://www.youtube.com/watch?v=XbGs_qK2PQA
Racine sings the english translation of the Disappearance of Hatsune Miku much faster:
https://www.youtube.com/watch?v=tohDycgc-H8
However, there is a difference between a rehearsed and practiced performance and communication that makes these things more a feat of acrobatics.
For a time in the 1980s John Moschitta was well-known for making fast-talking commercials:
* FedEx: https://www.youtube.com/watch?v=NeK5ZjtpO-M
* Micro Machines: https://www.youtube.com/watch?v=Gzd11GMBONg
* https://en.wikipedia.org/wiki/John_Moschitta_Jr.
i can catch a single fastball. sometimes. i cant catch 1 per second
1.7x here as a matter of course.-
PS. I wonder if, even peripheralally, having one of 'em newfangled AI glasses teleprompting you at the same time, could get one up to 2.0x or higher.-
This suggest that maybe human minds are able to form ideas at a certain speed, and language has evolved to convey these ideas and not be a bottleneck, but there is no reason to improve language beyond this.
Hm, I’ve always struggled with the other bottleneck - speaking. I tend to slur my words because my brain is forming the thoughts faster than I can speak. I could perhaps speed up my speech to match my brain but I’d sound like a maniac.
Don’t get me started on writing. My letters will often transition halfway into a letter thats 3 words ahead.
Typing speed is a good thing to develop.
Likely so. I've noticed that those whose first language is one of those with fewer vowels - and thus must have more syllables per second to convey the same information rate- sometimes speak English much faster than I (as a monolingual native speaker) find normal.
3-4x! Are you the flash? I usually run things about 1.5x when I am commuting. 3x would require laser focus without distractions. Or you mean more a “can technically absorb the language being spoken” sense?
Not OP but listen to podcasts at highly accelerated settings:
The information density of ‘two dudes talking’ or any unscripted format is very low, so it time-compresses well. Specific podcasts, typically scripted monologues with technical content, such as Causality [0] (recommended!), I need to listen to much slower. Ditto if it is in an accent which isn’t mine, which slows my comprehension. I also slow the speed if I’m driving. So, yes, it takes mental overhead, but is doable. Go one click at a time and it will feel natural.
[0]: https://engineered.network/causality/
I suppose the format is a huge differentiator. I exclusively listen to highly produced content which has essentially no dead time. The content is already a compressed transmission of information.
I would say that the ability and speed to receive and process information in a timely manner also greatly depends on the density and complexity of the material.
I have always wondered if transmission bitrate remains the same for the same language, spoken at seemingly different speeds.
There is that stereotype that french-speaking Swiss people speak slower than French ones. In my experience I find it valid, but maybe I am wrong.
If this is accurate, I am wondering if Swiss people transmit information at the same rate as French people.
It could be that they use more precise words on average, that convey more information, even if spoken more slowly, and keeping transmission rate identical. Or that body language, intonations are richer (non verbal).
Or the spoken transmission rate may actually be slower, but as the article describes, bottleneck is about structuring the ideas, and Swiss speakers, on average, may be more efficient/deliberate at that, instinctively/culturally.
I don't have enough experience speaking with Swiss nationals to verify my anecdotal theories... if anyone can chime in...
I'm sorry, but this is essentially racism prettied up. The research is about language bitrate not about regional speaking rate variations within a language.
Tangentially, I'm relatively confident that what you're experience has provided you is simply confirmation bias. Unless French is not their first language.
I agree that the research is about various languages bitrates. My point is a bit different, yes, it is about total information bitrate.
Maybe I am racist, but I am not sure how the speed at which one speaks is a racist trope. I sure do not look down on the Swiss, they're a pretty successful nation.
Question for you: what is your relative confidence based on?
On the positive, your comment invited me to check Wikipedia and I was surprised to see there are actually studies about some of this. They seem to confirm the stereotype. Belgian and Parisian seem to have a higher syllable/ms rate than several Swiss counties, with some caveats. [1]
But no, it does not talk about total information bitrate, that is much more subjective. Maybe if you are a "frontalier" and have some anecdotal experience with it, I'm curious.
[1] https://fr.m.wikipedia.org/wiki/Fran%C3%A7ais_de_Suisse
"Après examen des différentes études, le stéréotype des Suisses qui articulent plus lentement que les Parisiens est confirmé, à quelques exceptions près"
The paper does not seem to support that human speech has a universal transmission rate or that every spoken language has a universal information rate. They showed that information rate varied by individual and by language, just that it varied less than the syllable rate.
This was also bounded by a reading task, so the performance shown per language could be influenced by the average reading skills of people who speak those languages. They also asked them to pronounce differently than they normally would.
If you take the 17 languages they tested and get the average between them, you get 39bits/s. For English and French, the information rate they recorded was higher, with an average closer to ~45bits/s (just eyeballing their chart). Their results also showed Thai at ~35bits/s. A 10bits/s swing from median to median is pretty huge.
From the paper:
"We collected recordings of 170 native adult speakers of the aforementioned 17 languages, each reading at their normal rate a standardized set of 15 semantically similar texts across the languages (for a total amount of approximately 240,000 syllables). Speakers became familiar with the texts, by reading them several times before being recorded, so that they understand the described situation and minimize reading errors"
"Together, our findings show that while there is wide interspeaker variation in speech and IRs (information rates), this variation is also structured by language. This means that an individual’s speech behavior is not entirely due to individual characteristics but is further constrained by the language being spoken."
"However, languages seem to stably inhabit an optimal range of IRs, away from the extremes that can still be available to individual speakers. Languages achieve this balance through a trade-off between ID (information density) and SR (syllable rate), resulting in a narrower distribution of IRs compared to SRs. In the introduction, we rhetorically asked whether too low or too high an IR would impede communicative and/or cognitive efficiency. Our results here suggest that the answer to both questions is positive and that human communication seems to avoid two extreme sociolinguistic profiles: on the one hand, high ID languages spoken fast by their speakers (“high-fast”), and, on the other, low ID languages spoken slowly by their speakers (“low-slow”)."
https://www.science.org/doi/10.1126/sciadv.aaw2594
This definitely fits with my experience as a language learner studying French and Japanese.
Japanese is definitely a faster spoken language than French, but French words tend to be a lot more verbose and packed with meaning. For Japanese speakers to communicate as much meaning as French speakers, they would need to speak faster.
I wonder, does this account for compression? Are the bits counting a concrete representation of ideas transmitted? Or are bits counting a simple abstraction over syllables, similar to a token? Language is compression. And while humans may transmit at a universal rate, maybe some languages/cultures have a more dense compression of ideas than others? Like in Turkish, how there's a word for "moonlight on the water" https://ihearthesamewinds.wordpress.com/2011/11/01/142/.
Yes:
>Each participant read aloud 15 identical passages that had been translated into their mother tongue. After noting how long the speakers took to get through their readings, the researchers calculated an average speech rate per language, measured in syllables/second.
Maybe more interesting: what’s the average reading speed per language, and what’s the variation? I know that reading speed varies a lot ( also depends on topic, a math textbook reads slower than an Ian Fleming book).
Are simultaneous translator’s brains different? They need to process two languages at once, and I never could do that even though I’m fluent in more than one language.
I suspect that for someone who reads a decent amount reading speed is also bottlenecked by the rate by which we comprehend the ideas being communicated and not the rate at which we recognize the words. I recently did a bunch of reading speed tests in all the languages I understand. I have like two orders of magnitude more experience in reading German and English than in Dutch, French and Spanish. In the former group word recognition is automatic while in the later I have to concentrate and sometimes even sound out words in my mind yet my actual reading speed for my "strong" languages is only about twice that of my "weak" languages.
Translating is a completely different skill that you have to train on top of being fluent in more than one language. The way I translate is by sort of forgetting something in one language and then remembering it in another. It's a slow and awkward process but I suspect if I did this for like a thousand hours hearing something and then repeating it in another language would be as easy as switching the language in which I'm thinking. I think the real difficulty of simultaneous translation comes from having to speak while you're listening. Consider recording your response to an audio message while listening to it, that would also be very difficult but there is only one language involved.
Yeah. Reading rate varies greatly based on the complexity of the material and how familiar one is with it. It's not just the raw information density, but how much information must be retained and what that entails.
Professionals in a field very often communicate much faster with each other because so much stuff is a standard state plus deviations from that, whereas the person who doesn't know the field can't compress it like that. It's been studied with chess--good chess players are better at memorizing *sensible* board positions, but lose most of that advantage when confronted with nonsense.
https://www.science.org/content/article/human-speech-may-hav...
Maybe not just spoken language. I would suggest sign language has a similar bit rate as spoken. The evidence is that a sign interpreter conveys about the same information in a conversation in the same time.
The bits are defined in another paper by the authors using "syntagmatic density of information ratio (SDIR)" which seems to be related to the number of syllables required to convey the same information in different languages, using Vietnamese as the baseline.
I cannot wait for independent replication!
https://www.researchgate.net/publication/235971274_A_cross-L...
this study has a grand total sample size of 170 (that's 10 speakers per language), it also measures syllable rate (i.e. syllables/second times diversity of syllables) as a proxy for information. this is pretty far from how people will intuitively conceptualize "information".
i would take the results too seriously.
The belief that that these 170 speakers, each speaking many long passages in 17 languages would converge to 39bps through dumb luck is unlikely enough to be almost mystical.
edit: If I hear a case that is not insane as to how the numbers could somehow determine themselves (through bad math) or be p-hacked, I'd happily consider it. Instead people are acting like taking sentences of equivalent meaning and counting the syllables to determine information density is somehow laughably naïve.
check out the graph in the original paper, personally i won't call that "converging".
(2019)
Some discussion then: https://news.ycombinator.com/item?id=20880789
39 bits per second, about twice the speed of Morse code
Guessing there's something very fundamental that the author misunderstands about Morse code.
I would expect you can only transmit so fast in morse code given the need for the dots and dashes to be clearly distinguished from each other and identifiable by the recipient.
Of course, if you know both ends are computers you can just transmit in some other encoding at a much higher rate.
The point is, Morse code is as many "bits per second" as you want. You could send Morse code at 10 gigabits per second if you wanted. It is not meaningful to say that Morse code implies a particular data rate.
Historically the metric for Morse code is words per minute. Morse is similar to a Huffman code where common letters are allocated fewer elements, so it's not very meaningful to talk about "bits per second" with respect to Morse even if you do specify the number of words per minute. The number of "bits" will vary based on the letters being transmitted.
Oh, come on, this is just being coy for no reason. Given the context, it is abundantly clear they mean "normal, human operated morse code".
A skilled operator is around 30 WPM. The average English word is 5 (rounded up) characters. Add one character for the space. That's 180 characters per minute, or 3 character per second. With 37 characters available in morse code, that's log_2(37) or 5.2 bits per charater.
So 15.6 bits per second. Just under half of the 39 bits they got for speech, like they said.
Yes, computers can transmit any language at any speed (if we include parallelism).
That wasn't contested.
I mean if you want that level of pedantry "I" can't send Morse code at 10 gigabits per second. I can make a computer transmit it at that speed, but I am not personally sending it that rate. And, because one generally needs a machine to transmit Morse code, one cold argue that "I" never send Morse code ever.
As Morse speeds up, you stop relying on individual dots and dashes and begin recognizing common combinations of letters. Faster still and you are mainly hearing word stems and suffixes.
The faster the information comes at you, the less important any particular bit is, because you have more context with which to autocorrect.
> the dots and dashes to be clearly distinguished from each other
Yes, spaces are part of the morse code spec. It looks like a binary encoding but in fact it's ternary.
We can invent a 5-bit (or 6-bit, to include numbers and punctuation) morse-like code to avoid needing spaces.
Universal between languages maybe, but certainly not universal between individual speakers.
There is also a lot of nonverbal data. Imagine the horror of discovering your conversational ML build could hold a plausible verbal conversation only guessing 58% of spoken words accurately... then realizing humans likely fair much worse. =3
"Prisencolinensinainciusol" (Adriano Celentano)
https://www.youtube.com/watch?v=Ax9c8f82QZA
Does that mean people with a minor vocabulary probably speak faster?
Is the constraint the speaker or the listener?
Adam says to not listen at 2X.
Who is Adam, and why would that be?
[2019]
its not the language that communicated its the people. You recruit mediocres in different languages they all communicated in the same mid , 39 bits or whatever. Let two geniuses have a chat or even two teenagers - its a different ball game than 39 bits
[dead]
Unforgivable that it is not 42 bits per second.