Content warning: This is a baker’s dozen not a regular dozen, in case anyone clicks through expecting to find twelve and is mildly and briefly perturbed.
It's clearly hard, but there are tools for doing exploratory visualization of high-dim data. GGobi http://ggobi.org/ and all the ones that arrange points but try to get local neighborhoods correct (t-sne, umap, et al.).
Yeah, but still "scary" because you have to be really careful to not fool yourself and pay attention even with those algorithms. For example, a good demonstration with tsne
https://distill.pub/2016/misread-tsne/?hl=cs
Often there is little or no substitute for plotting the data to see how it is distributed. A scatter plot, histogram, density plot, etc. is almost always going to tell you a "story" about the data that the summary stats will have compressed.
But sometimes you are at the mercy of the data and your visualization of choice. Box plots, for example, are great at showing more than just how the data is centered, but it is possible to encounter situations where the box plots of the data remain static while the underlying data is clearly changing [0].
As always it is good to know about these things and continue to add to the arsenal (violin plots, in the example above) of tools and intuition needed to tease out the story behind the data.
That's an amazing addition! Once I read about Simpson's paradox[0], couldn't help but seeing it or suspecting it everywhere. Luckily, it is not a true paradox, and it can resolved if underlying data is available and not just summary statistics.
I recommend putting together the Quintet in one image, so that the original 4 charts, plus the new one are all visible and interpretable together. It will be learning aid for decades to come.
Yes, not saying the data dinosaur isn't cool. But for real-world applications, the quartet with the addition of this fifth dataset is more useful for pedagogical purposes.
During my statistics degree, Anscombe’s Quartet was used as an example of why you should always try to visualize your dataset and not just run your calculations blindly. I’m a bit odd in that I don’t care much for data viz, but Anscombe’s Quartet really shows how important it is in practice.
This dataset is definitely a treasure, and I love visualizing data. That said, i think what's missed when this is used as an argument for visual analysis is the idea of quantitatively identified outliers. If you take the descriptive statistics of p99, they most definitely will not be the same across these four sets. Visual analysis is a valuable dimension for data exploration, but it's a bit of a strawman to infer that "quantitative analysis could go no further, only visual analysis could figure this out"
I know this is against the main point of Anscombe’s Quartet but just curious: Could skewness or other summary statistics differentiate the four distributions?
And was just thinking about it the other day. I had a bug aggregating sleep-data from an iPhone, which comes in the form of sleep-samples.
I was trying to fix it, both by prodding Claude Code to fix the problem, and looking at debug logs of the sleep-samples, but we weren't getting anywhere. I asked Claude Code to graph the samples, and BAM, saw it right away. (the problem was that HealthKit returns you sleep-samples from ALL devices, not just the priority one)
Maybe not exactly the same thing as Anscombe/Tufte were getting at, but I was reminded of it, and the value of visualising data.
The example shows that the usual stats aren't enough to pin down the true data. But in practice I imagine / wonder if these stats really are reasonable "sufficient stats" because the probability of seeing data with strong structure is unlikely in most contexts. In other words...
and p(data) is only strong for a "blob / cloud" of points, so when there's some correlation the observed stats tell you that you likely have a blob having some degree of correlation.
>But in practice I imagine / wonder if these stats really are reasonable "sufficient stats" because the probability of seeing data with strong structure is unlikely in most contexts.
We just spent five years since COVID appeared to argue about statistics, with tons of bad analysis of very complicated data fuelling political rage up to this day.
The US health secretary is currently using data with "strong structure" to deny vaccines and to falsely pin down convenient targets for everything from cancer to autism.
Always visualize first. Human 'eyballing' is a good pattern detector.
Linear correlation is just one pattern the data can have.
Unfortunately many social science publications have reviewers who know only the basics and can't judge or accept statistically valid analysis that is outside their competence. Fit it into line or nothing.
And check this one, which is a generalization of the Datasaurus where you can define your own shapes :D
https://github.com/stefmolin/data-morph
From now on I won’t trust any statistic unless I can transform it into a panda.
A classic.
See also:
https://en.wikipedia.org/wiki/Datasaurus_dozen
Content warning: This is a baker’s dozen not a regular dozen, in case anyone clicks through expecting to find twelve and is mildly and briefly perturbed.
The scary thing is that yea we can see these in 2D and maybe 3D. But ...
usually there are more than 2 or 3 columns in our data :(
It's clearly hard, but there are tools for doing exploratory visualization of high-dim data. GGobi http://ggobi.org/ and all the ones that arrange points but try to get local neighborhoods correct (t-sne, umap, et al.).
Yeah, but still "scary" because you have to be really careful to not fool yourself and pay attention even with those algorithms. For example, a good demonstration with tsne https://distill.pub/2016/misread-tsne/?hl=cs
Often there is little or no substitute for plotting the data to see how it is distributed. A scatter plot, histogram, density plot, etc. is almost always going to tell you a "story" about the data that the summary stats will have compressed.
But sometimes you are at the mercy of the data and your visualization of choice. Box plots, for example, are great at showing more than just how the data is centered, but it is possible to encounter situations where the box plots of the data remain static while the underlying data is clearly changing [0].
As always it is good to know about these things and continue to add to the arsenal (violin plots, in the example above) of tools and intuition needed to tease out the story behind the data.
0: https://www.research.autodesk.com/publications/same-stats-di...
I teach curve fitting with this dataset and recently added the fifth dataset. It illustrates Simpsons paradox.
https://www.linkedin.com/posts/panela_loved-adding-ancombes-...
That's an amazing addition! Once I read about Simpson's paradox[0], couldn't help but seeing it or suspecting it everywhere. Luckily, it is not a true paradox, and it can resolved if underlying data is available and not just summary statistics.
I recommend putting together the Quintet in one image, so that the original 4 charts, plus the new one are all visible and interpretable together. It will be learning aid for decades to come.
[0] https://en.wikipedia.org/wiki/Simpson's_paradox
Yes, not saying the data dinosaur isn't cool. But for real-world applications, the quartet with the addition of this fifth dataset is more useful for pedagogical purposes.
During my statistics degree, Anscombe’s Quartet was used as an example of why you should always try to visualize your dataset and not just run your calculations blindly. I’m a bit odd in that I don’t care much for data viz, but Anscombe’s Quartet really shows how important it is in practice.
This dataset is definitely a treasure, and I love visualizing data. That said, i think what's missed when this is used as an argument for visual analysis is the idea of quantitatively identified outliers. If you take the descriptive statistics of p99, they most definitely will not be the same across these four sets. Visual analysis is a valuable dimension for data exploration, but it's a bit of a strawman to infer that "quantitative analysis could go no further, only visual analysis could figure this out"
I know this is against the main point of Anscombe’s Quartet but just curious: Could skewness or other summary statistics differentiate the four distributions?
Take enough moments and you'll be able to differentiate any distributions.
I love it. I was introduced to it by Edward Tufte's book, 'https://www.amazon.co.uk/Visual-Display-Quantitative-Informa...'.
And was just thinking about it the other day. I had a bug aggregating sleep-data from an iPhone, which comes in the form of sleep-samples.
I was trying to fix it, both by prodding Claude Code to fix the problem, and looking at debug logs of the sleep-samples, but we weren't getting anywhere. I asked Claude Code to graph the samples, and BAM, saw it right away. (the problem was that HealthKit returns you sleep-samples from ALL devices, not just the priority one)
Maybe not exactly the same thing as Anscombe/Tufte were getting at, but I was reminded of it, and the value of visualising data.
Relevant: Simpson's paradox. https://en.wikipedia.org/wiki/Simpson%27s_paradox
“The Datasaurus Dozen”:
https://blog.revolutionanalytics.com/2017/05/the-datasaurus-...
The example shows that the usual stats aren't enough to pin down the true data. But in practice I imagine / wonder if these stats really are reasonable "sufficient stats" because the probability of seeing data with strong structure is unlikely in most contexts. In other words...
p(data | stats) = p(stats | data) * p(data) / p(stats).
and p(data) is only strong for a "blob / cloud" of points, so when there's some correlation the observed stats tell you that you likely have a blob having some degree of correlation.
>But in practice I imagine / wonder if these stats really are reasonable "sufficient stats" because the probability of seeing data with strong structure is unlikely in most contexts.
We just spent five years since COVID appeared to argue about statistics, with tons of bad analysis of very complicated data fuelling political rage up to this day.
The US health secretary is currently using data with "strong structure" to deny vaccines and to falsely pin down convenient targets for everything from cancer to autism.
Thought this would be about the 'other' Anscombe:
* https://en.wikipedia.org/wiki/G._E._M._Anscombe
:)
Or:
https://en.wikipedia.org/wiki/Gareth_Anscombe
:-)
I did a talk on Cognitive Biases in performance measurement and included Anscombe's Quartet (among other things) in the section on developer bias: https://speakerdeck.com/bluesmoon/we-love-speed-understandin...
Always visualize first. Human 'eyballing' is a good pattern detector.
Linear correlation is just one pattern the data can have.
Unfortunately many social science publications have reviewers who know only the basics and can't judge or accept statistically valid analysis that is outside their competence. Fit it into line or nothing.
I do STEM mentoring for high school kids. Bookmarking this, because it'll be a great teaching aid at some point.
This reminds that “visualize while thinking” will probably become an important part of reasoning as we move closer to AGI models.
This will require improvements to vision models, RL frameworks, etc, but will be interesting to see how much it can broaden current abilities.
[dead]