Last month 2,400 University of California faculty asked for admissions to resume using the SAT "to ensure foundational fluency." Of course many employers want to ensure that too, especially when college degrees don't anymore.
The widening abilities gap followed the 2020 elimination of the SAT/ACT, a temporary measure that has now become a permanent vulnerability. This outcome was explicitly predicted by the Academic Senate’s 2020 Standardized Testing Task Force (STTF) report, which warned that removing these tests would eliminate a vital predictor of college success and obscure the impact of severe high-school grade inflation. Unfortunately, the outcomes cautioned against in that report have now materialized in the data across our campuses. All other leading STEM institutions, including the UC’s primary peers, have resumed using SAT/ACT in their admissions to ensure foundational fluency. For the University of California to remain a global leader in STEM, it is essential to restore these objective benchmarks. -- https://ucstudentsuccess.org/
The UC faculty opposed the SAT requirement being discarded in the first place. They were overruled by the UC Regents, and that may happen again. And even if the SAT is brought back, I'm sure it will be given much less weight and subjected to the "in a local context" process in the name of equity.
The SAT is actually a lot easier today than it was when most of us took it.
It is an hour or more shorter in length, the long reading passages have been replaced with short paragraphs, calculators are allowed, and vocabulary has been removed.
I never took the SAT, but I did take the GRE. The year I took it, it was 3 sections: Quantitative, Verbal and Writing. The year I took it, the writing section replaced an older Logic section. Quantitative and verbal were scored out of 800 points (like the SAT), but the writing section was scored 0-6 in half-point increments. I think the writing section has also since been removed.
I need the entire paragraph above just to _explain_ my GRE (decent) score because the test has changed yet again in the interim 20 years or so and I suspect the SAT is similar.
I flipped through a book that coached on SAT vocabulary. I knew all the words. Oh well. I never learned vocabulary as an explicit task. I simply read a lot.
I remember one question on the SAT verbal because it irked me. It asked an analogy question which required knowledge of mixed alcoholic drinks. Since I was far from drinking age, I had no idea.
The fact that it's not all multiple choice now makes it somewhat harder. Multiple choice questions make it much faster to solve some questions because you can simply plug in possible answers. They also make it easier to know that you're right, if you solve a problem and the answer you got is one of the choices (though they do sometimes include common mistaken answers to fool students).
No idea what the SAT is like now a days (based on the comments in this thread) but 'back in the day' if you knew enough about a problem to understand the math needed to solve it, which is needed to plug in the answer, then it'd generally be faster to just solve than working backwards since the number of steps would typically be less than repeatedly plugging in. And most/all questions also had a 'None of the above' option that was the answer a fair chunk of the time.
Another practical thing is that tests seem to trend substantially higher in difficulty when multiple choice. I was part of the first class at my university that took a calculus program which was multiple choice and we thought it was going to be a cake walk. But suddenly like every single test problem was using obscure trigonometric tricks on top of the basic calculus itself. And of course no partial credit for getting everything 95% right and missing one really disguised trig trick at the end. Grades for the class were significantly lower than prior years, because those tests were just nuts - and I'm a very much a math guy.
Graphing calculators can be used to quickly solve certain problems, like simultaneous equations or quadratics. They can also be used to plug in multiple-choice answers to see which one is correct, without knowing how to solve a problem the normal way (or not taking the time to, at any rate).
The new adaptive digital SAT complicates things a bit, in that some questions are not multiple choice.
Fair enough. I'm just pointing out that calculators are not useless on the SAT. Knowing how to use the provided graphing calculator, or having your own and knowing how to use it, is very important if you want a top score.
"Equity" is still something of a euphemism - the specific reason that the UC Regents got rid of the SAT requirement is because the non-profit law firm Public Counsel, along with a consortium of other non-profits and activist organizations, sued the UC system ostensibly on behalf of several students; and the UC Regents ended up agreeing to a settlement where they would not use SAT or ACT scores for admissions until after 2025. Public Counsel brags about the details of the lawsuit on their website (https://publiccounsel.org/our-cases/smith-v-regents-of-unive...):
> For decades, the University of California’s use of discriminatory SAT and ACT scores deprived hundreds of thousands of well-qualified students of color, students from low-income families, and students with disabilities of the opportunity to pursue higher education in the nation’s preeminent public university system. Rather than provide meaningful information about a student’s ability to succeed in college, SAT and ACT scores act as stand-ins for students’ wealth and race, and thus advantage more privileged applicants. Even University leaders admit that the tests are “racist” and “correlated to wealth and privilege.”
> Public Counsel and co-counsel brought this lawsuit on behalf of students and community organizations—Chinese for Affirmative Action, College Access Plan, College Seekers, Community Coalition, Dolores Huerta Foundation, and Little Manila Rising—challenging the University of California’s use of the SAT and ACT as discriminatory on the bases of race, wealth, and disability. In August 2020, Plaintiffs obtained a preliminary injunction requiring the University to immediately stop using the tests for undergraduate admissions and scholarship determinations. Plaintiffs then defeated the University’s attempts to prevent the injunction from taking effect. In the admissions cycle following the injunction, UC saw record gains in numbers of Black and Latinx students applying to and gaining admission to its campus.
Public Counsel is pretty clear that the legal theory they were originally operating under is that requiring SAT and ACT scores for university admission is racially discriminatory, and the specific races it's discriminatory against are the groups they characterize as "students of color", which we can take to mean primarily black and Latino students because they specifically mention the detail "In the admissions cycle following the injunction, UC saw record gains in numbers of Black and Latinx students applying to and gaining admission to its campus.".
Interestingly, the lawsuit was originally brought forward in late 2019, before the start of the COVID pandemic. The temporary injunction against using the SAT/ACT was imposed in August 2020, well into the pandemic, on the grounds that the pandemic conditions made it more difficult for applicants with disabilities to take standardized tests, in a way that was plausibly legally discriminatory - but of course this couldn't have been the primary legal justification that Public Counsel used when the brought the lawsuit in 2019, unless they were prescient enough to have predicted the course of the pandemic at that time (in my memory, the number of people in the Anglosphere who were paying attention to COVID-related news in China before the turn of 2020 and thought that it might develop into a concerning pandemic was incredibly small).
The settlement that the UC Regents reached in May 2021 lasted until Spring 2025, so it's only now that it's legally possible for the UC Regents to reconsider the ban on using the SAT/ACT for admissions. Presumably, Public Counsel and the other activist groups - Chinese for Affirmative Action, College Access Plan, College Seekers, Community Coalition, Dolores Huerta Foundation, and Little Manila Rising - haven't changed their opinion that the use of the SAT/ACT is racially discriminatory towards blacks and latinos. But they don't seem to have raised another lawsuit about this, perhaps because the political environment in the US has changed since 2020 in ways that make them less optimistic about their chances of success.
> lasted until Spring 2025, so it's only now that it's legally possible for the UC Regents to reconsider the ban on using the SAT/ACT for admissions
They could have reinstated the requirement last year, and they could have undertaken their recently-determined plan to engage in study regarding reinstatement anytime before that. They just couldn't remove the requirement until 2025.
Appreciate your detailed description of the lawsuit and settlement. This is what happens when two parties are settling a lawsuit but do not actually have adverse interests. They were aligned on wanting to get rid of it and signed an agreement to do so. That said, the Regents decided to get rid of it permanently, with no plan to bring it back or create any replacement test (as they had previously said they would do).
It's not a claim, there is psychological data supporting the negative impact stress can have on recall and that for some students a test is not the best way to identify their retained knowledge.
Your single example identifies a situation where we would want someone resilient to stress to pass the testing process.
However if you are evaluating someone's ability to identify characters in Shakespeare who are most closely representative of the Bard himself, a proctored exam may not be the only environment where that could be demonstrated.
Anyone in software knows as well that for a test to be effective it has to be written properly. In the history of academic and standardized testing however there has been little rigor in the construction of tests, and those who pass are ones who give the "right" answers regardless of whether those answers are true proofs of knowledge.
I've been around tests all my life. I don't know anyone who bombed the test but mastered the material. And the ones who aced the tests - turns out that they actually had mastered the material.
As for people who cannot perform under the stress of a test, how are they going to perform otherwise? Anyhow, the solution to test anxiety is to keep taking tests - the anxiety will recede.
For example, the first time I tried public speaking I was paralyzed. But I kept trying it, and the anxiety went away, and now I do a fair amount of public speaking and enjoy it very much.
That still doesn’t seem like a reasonable reason to not do tests. In the software analogy you have a unit test that passes but the actual software fails when used in the real world.
What is the point of “knowledge” that can’t be demonstrated. How will that person demonstrate their knowledge of Shakespeare?
My understanding is you're equating `failing a test` to `lacking the relevant skills and knowledge to do a certain task competently`.
The reality is sometimes tests in academia are just not very well made and don't really test what they are supposed to be testing, and that's usually due to multiple reasons like misaligned incentives, staffing shortages and maybe lack of resources / funding.
I don't think the comparison to flight school is relevant enough in this context because it's a too different of a world to traditional academia.
I don't buy the notion that tests do not test relevant skills.
In my long career I've noticed a strong correlation between SAT scores and academic performance as well as job performance.
> I don't think the comparison to flight school is relevant enough in this context because it's a too different of a world to traditional academia.
My dad kept his flight school tests for flying all sorts of airplanes. They bear a lot of similarities with the SATs. There's a lot of math in there for things like fuel consumption, wind, maximum landing weight, glide distance, and so on.
For example, one day he was cruising along in his F-86 when the engine failed. he radioed the tower, and they told him to bail out. But he calculated his speed, altitude, distance, wind, sink rate, air templeratur, etc., and figured he could make the field after configuring the airplane for maximum glide. He made a perfect landing, but still got reprimanded for risking his life bringing the airplane back. But he had worked the math and disagreed that it was more risky to bring it in than bail out.
> I don't buy the notion that tests do not test relevant skills. In my long career I've noticed a strong correlation between SAT scores and academic performance as well as job performance.
SAT tests intelligence (aptitude), not skills. Which is why it correlates with job performance, where intelligence can (over some time) matter as much or more than a starting point of relevant skills.
I just checked, and the SAT math section covers algebra, trigonometry and statistics.
Look at this list:
Quadratic equations and functions (vertex form, roots, discriminant)
Polynomial operations and factoring
Exponential functions and growth/decay
Radical and rational expressions
Function notation, composite and inverse functions
Nonlinear graphs and their transformations
A genius student who had never been taught those subjects wouldn't even know what the symbols meant. A mediocre student who had studied SAT-style questions for weeks leading up to the test would likely outperform a high IQ student who last solved those types of problems over a year prior.
Standardized tests can be a great resource for assessing students, but they're not just testing for intelligence. Test-prep courses average increasing SAT scores by about 200 points. That's not because they're increasing the intelligence of the people taking them.
Somebody who goes to take a test on something they know that they know nothing about could be called many things, but genius is not one, even moreso when they're paying for the privilege of taking that test. What is on the SAT is no secret, so people are free to prepare as little or as much as they might like. If somebody can't be assed to prepare for such a critical test, then they're probably going to be the sort of person who can't be assed to do much of anything in life. And the internet has also largely relegated the inequities in access to training quite obsolete. You can get free high quality training materials on everything for free.
Or they're a student who has to work a part-time job after school. Or they have a long commute. Or they have hours of practice for a competitive sport or extracurricular. Or they have to take care of their siblings, grandparent or sick parent. Or they were told by their parents and teachers that because they're smart and do well in school that they don't need to worry about studying. Or they know their parents can't afford an elite school anyway, so they know they just have to score well enough to get into a school with in-state tuition and noncompetitive admissions.
Two kids who both went to the same school, were told the same things about the importance of the SAT and dedicated equal time to study and preparation can probably say that the difference in their scores is indicative of differences in intelligence.
But any two random students? There are so many factors that could cause students to underperform their hypothetical max score if they had perfect conditions in the months leading up to the test and day of.
We're talking about the SAT, not the gaokao [1]. It's a relatively short test that you can take it whenever you want, you can even retake it if you want, however many times you want. There's minimal memorization required and it's mostly testing basic aptitude. But the overwhelming majority of people are of average aptitude, so the overwhelming majority of people will do "poorly" on the SAT when comparing themselves to outliers.
I don't understand the effort to try to eliminate the reality that innate ability exists. I think we should accept this and work to cultivate it. Just because somebody of average aptitude doesn't mean he might not excel in other ways. The world needs all sorts of people, and I think the ideal system would work to figure out and cultivate these talents in everybody.
There's plenty of time during school hours to study up on the math. My high school was deficient in math, everyone knew that, and I took math classes at the local community college to make up for it.
I never studied for tests in school, not for the SATs, either. I tried that in college, that did not work. The stress of the tests was pretty bad. I realized I had to study hard, and I did, and it was effective at reducing test anxiety.
There's a difference between not studying and just hand-waving away the SAT, which plays a huge role in things like college admittance. And the SAT isn't particularly hard, even back in the days when it was apparently significantly more challenging, for a clever kid. My prep was picking up some old book on it at Half Price Books and leaving it in the bathroom. Worked excellently!
> A genius student who had never been taught those subjects wouldn't even know what the symbols meant.
This is pedantic. Take it to the logical next step. Words and letters are symbols too. How would someone who can’t read do on the SAT?
> A mediocre student who had studied SAT-style questions for weeks leading up to the test would likely outperform a high IQ student who last solved those types of problems over a year prior.
Would make an interesting experiment, if it hasn’t already been studied. I would put my money on the lapsed high IQ student though.
"A mediocre student who had studied SAT-style questions for weeks leading up to the test would likely outperform a high IQ student who last solved those types of problems over a year prior."
Do you also think LLM leaderboards accurately reflect the capabilities of the models being tested? If you do, then I can easily point you to numerous academic papers pointing out the various flaws in many leaderboards (from poorly designed benchmarks like bABI and the original SQuAD, to data contamination, and more).
In that same way, any test, including the SAT and GRE have flaws. They can be gamed in ways similar to LLM leadeboards: test prep makes you better at them. That's one of the main reasons universities moved away from SAT; they were afraid that it disenfranchised lower socioeconomic status students (and it does to some degree). The issue is that the test is positively correlated with success in an undergraduate program, so they threw out the baby with the bathwster. The real issue is that the SAT is not able to distinguish the capabilities among students to the degree it purports to.
And if you want an anecdote to match all yours, the first time I took a GRE practice test, I got a 3 on the writing. Not because I'm poor at writing, but because I didn't really know what they were looking for. After reading a test prep book, I got a 4.5 on my next practice test and a 5 on my final practice test. When I finally took the actual GRE, I got 6 on the analytical writing. Trust me, nothing changed in my writing ability over that time. In fact, I didn't even practice the skill except through those three practice tests. Clearly the test was not capable of determining my real ability to make an argument; it merely tested my ability to adapt my writing to what was supposedly being tested.
Interestingly, the vast majority of universities that got rid of the GRE requirements for PhD programs are not going back on that. Turns out that the students with the highest GRE scores are the ones most likely to drop out of their STEM PhD. [1]
There are three major parts of the modern GRE: Verbal, Quantitative, and Analytical Writing. You could easily look that up, or ask if you didn't know.
Responding off the cuff without any reflection on the comment you're responding to doesn't move the conversation forward in any meaningful way. It just comes across as disrespectful.
Do you think that LLM leaderboards don’t? Do you think a Llama 3 is going to beat an Opus 4.7 on any leaderboard?
The real issue is that standardized tests disenfranchise lower SES students less than any other metric.
Everyone who takes the SAT has to sit in the same room for the same amount of time answering the same questions. You can’t just pay someone else to take it for you (like essays) or select which difficulty level you take (like going to a prep school with grade inflation), or luck out in who your parents know (like recommendation letters).
Some may have better opportunities to learn the material, but, at the end of the day, you have to actually learn the material. There’s no getting around that.
As your own GRE anecdote shows: A little studying with some inexpensive books makes all the difference. Unless things have radically changed, a couple SAT or GRE test prep books are significantly less expensive than just one college textbook.
Bluntly, the reason SATs are better correlated to college performance than other measures are because of the reasons I mentioned. They strip away most of the privilege of coming from a high-SES family.
In my country, we have public exams to get into Uni, with the ones for high demand majors being very competitive, but performance in that exam is not a good predictor of academic performance.
The guy who got into my uni class as #2 in that exam dropped out after a few semesters because he couldn't beat calculus. The #4 took several extra semesters to graduate despite not working/not interning. Several others in the top third struggled through. We had _maybe_ 2 or 3 guys who straight-A'd the entire major.
I myself got in as #17 and still failed a few courses. Thankfully no one cared throughout my professional career.
> In my long career I've noticed a strong correlation between SAT scores and academic performance as well as job performance
On average. However, I've also had the experience that some of the most competent people I've known had rather difficult teens and twenties.
Hiring someone who flunked out of highschool, worked odd jobs for 10 years, then got a diploma and a degree is higher risk, higher reward. They are often times harder workers, unusual thinkers and more grateful for what they have.
My wife and I are perfect poster children for demonstrating the danger in making the assumption that tests are an accurate assessment of someone's mastery of material.
My wife has anxiety, and the time-limited, high-stakes, pass-or-fail nature of a test makes it much, much harder for her to perform well. Outside that context, she's blazingly intelligent, performs very well under real-world high-pressure situations, is extremely diligent at getting any other assigned work done well on time, and has repeatedly written the guidelines for processes and procedures in her jobs.
I'm the opposite in many ways. I generally do somewhat poorly in a classic classroom environment, primarily due to ADHD hampering my ability to get homework done regularly and on time. However, I test amazingly well. I consistently finish in 50-70% of the time of other students, with scores in the high 90% range. (In my jobs, I think I do pretty well.)
Naturally the plural of "anecdote" is not "data", but I've known enough other people who fit both of our molds that I think it's fair to say that basing even a plurality of your assessment of someone purely on tests, especially standardised tests, is going to mislead you at least as often as not, in both directions.
Whether or not tests are bad the fact is that nobody has even come close with something objective that can replace it at scale. Teachers have 30 kids in their class- they actually in reality have no idea what their students can or cannot do.
I agree with you. The real world is stressful or less than ideal for many reasons and we have to apply our knowledge within that context.
In addition to that, this claim is like the dragon in my garage thought experiment. It an unfalsifiable claim that they have “knowledge” but can’t demonstrate it.
Not really comparable... the overwhelming majority of flight tests involve flying an aircraft. There is no meaningful way someone can be excellent at flying an aircraft but can't pass a test which involves flying an aircraft.
The same can't be said for many other tests. If the test involves the practical application of the very skill being tested, then that test has direct relevance to he competency of said skill.
But many other tests are not like that. A teacher can be brilliant in the classroom yet stumble on a standardized certification exam full of pedagogical jargon. A chef can cook a variety of excellent dishes but fail a written culinary theory exam testing the French names of techniques they perform by instinct. And perhaps more relevant to this audience, a coding interview that relies on whiteboarding algorithms from memory can easily fail an excellent engineer who builds great software every day but doesn't recall the optimal solution to some puzzle on the spot.
Given that I'm not responding to any claim about the efficacy of math tests... it's actually your statement which is wholly irrelevant to this discussion.
The submission isn't about academic tests involving academics... it's about the relevance of using such academic tests for the purpose of job screening.
Which is why almost no employers ask for SAT scores as a way of screening candidates, and why this article was written to showcase the specific reasons doing so is ineffective.
Of course you'd have refrained from making such a vacuous reply if you bothered to read the article.
The problem that colleges and the SWE profession in general face are identifying “bullshitters”. We need a filter and a fact based exam seems like a good place to start.
They are never going to let you into a cockpit until you pass ground school, which involves a lot of math.
> A teacher can be brilliant in the classroom yet stumble on a standardized certification exam full of pedagogical jargon.
A teacher that cannot explain how calculus works cannot teach it to anybody.
> a coding interview that relies on whiteboarding algorithms from memory can easily fail an excellent engineer who builds great software every day but doesn't recall the optimal solution to some puzzle on the spot.
I've seen too many coders using bubble sort because they don't know enough to look for a better algorithm.
In any case, the purpose of leet coding tests is to quickly filter out the utter frauds. I have a programmer friend who wanted a job at a major software corp. He knew he'd have to pass the leetcode in an early stage of the interviewing. He figured it would take 6 weeks or so to study that material. I suggested that, since he was applying for a $250K job, that would be the most productive studying he'd ever done. He agreed, did the 6 weeks of studying, aced the leetcode test, and got the $250K.
So ya, there is a point to those tests, in filtering out the frauds and the ones who aren't willing to do what it takes to get those jobs.
So you yourself has just given an example how easy it is to temporarily get good at "test-filling", by simply preparing for that.
The question is whether a given test measures anything relevant - did your friend become a better programmer for doing 6 weeks of leetcodes? E.g. what kind of experience did he gain about large code bases and how to handle those? Continuing your analogy, would you fly with a pilot who drilled on taking off a bunch of time, but never practised flying in a storm?
I'm not saying leetcodes or exams are useless, but Goodhart's Law apply.
Regardless of the content, a test filters for people willing to put in the work to pass. All else being equal, given two candidates, where one has passed a test and another one who didn't bother, the first one sends a stronger signal to the potential employer.
It's still worth discussing if the particular test is ideal, but the answer IMO is not to say let's remove tests in general.
Is that what the sat is supposed to measure, in your opinion? Do you worry that would heavily skew toward people who didn't have to work over people who didn't have as much available time to train to a particular test? Does that even matter?
Ground school most certainly does not involve a lot of math, it's not like there's any calculus or algebra involved... it's basic arithmetic. Furthermore it's categorically false that you need to pass ground school before you're allowed to fly.
Are you just making things up?
>A teacher that cannot explain how calculus works cannot teach it to anybody.
This is a strawman argument, I never made anything that could even remotely be interpreted as this.
>I've seen too many coders using bubble sort because they don't know enough to look for a better algorithm.
This is committing a very basic logical fallacy. The fact that someone who is incompetent likely can't pass a test is not the same claim as someone who can't pass a test is likely incompetent.
Hopefully you are able to identify this logical mistake that you're committing and revise your position accordingly.
Google sez: "The U.S. Air Force strictly requires you to complete and pass formal academic ground training before you ever touch the controls of an aircraft"
They're not going to risk an aircraft on an incompetent student.
> A teacher can be brilliant in the classroom yet stumble on a standardized certification exam full of pedagogical jargon
I stand by my statement.
> logical fallacy
A implies B meaning B implies A is indeed a logical fallacy. But that does not rule out B implies A. A and B can be strongly related to each other.
So you are making things up... thanks for confirming that. While I appreciate that you reviewed what Google "sez"... you have misunderstood the relevant context which is that the U.S. Air Force also requires that you complete Initial Flight Training (IFT) before you start the Air Force's own formal training program (UPT). In IFT you will not be required to pass ground school before you get to fly.
Furthermore, even if the Air Force did not require IFT before UPT (the Air Force's own training program), you've completely changed the nature of your argument. I have no dispute about whether the Air Force may or may not have stricter requirements for their pilots, but that wasn't your argument.
>I stand by my statement.
You've proudly planted your flag on a point nobody was contesting, which is a strange hill to celebrate on but you do you.
>But that does not rule out B implies A. A and B can be strongly related to each other.
Discussing a topic with someone who not only uses logically fallacies as justification for their argument but brazenly doubles down on said fallacy is a good sign that this is probably not a good discussion to continue spending time on. Like am I supposed to simply accept your logical fallacy and take on the burden of disproving every claim you can dream up simply because you've asserted it isn't logically impossible? The person making the claim carries the burden of supporting it, and "they're strongly related" is something you have to actually show, not something I'm obligated to refute on your behalf.
There is no singular "IFT"... you happened to find one IFT among hundreds across the U.S. that has such a syllabus, great... but it does not come first as requirement mandated either by law/regulation or convention. Here is the syllabus for a different FAA Part 61 and 141 approved IFT program that uses an integrated approach with the following quote:
"Each Module contains both a flight and ground lesson. This presents an integrated flight training process and will promote easier learning and a more efficient flight training program"
The USAF is not subject to (most) FAA regulations. As far as I know this also extends to training: the USAF does not follow either of the FAA’s Part 61 or 141 flight training syllabi. So bringing up parts 61 and 141 doesn’t really refute the parent’s point.
Regardless, as a civilian, you do have to pass a written multiple-choice test on flying theory before you can solo. Any time you spend in the cockpit before passing that test will be under the supervision of a Certified Flight Instructor (CFI), in a plane with dual controls, and you are not acting a pilot-in-command.
So, although the parent may be slightly over-focused on the USAF way, I think it is fair to say that for any type of pilot training in the USA, you do have to complete academic ground training (including passing a formal written test) before you are truly “given the controls”.
When I was a boy, as a present my dad bought a ride in a Cessna for me. I sat in the copilot's seat, and the pilot let me try out the controls. I also was offered a ride by my lead engineer at Boeing and he let me work the controls a bit.
In addition to affirmation of the consequent he's also employing attacking a strawman, petitio principii, faulty analogy, and goalpost shifting, at least. His followup example "Note that a tree implies it is made of wood. If you find a stick of wood, odds are it came from a tree." is hilarious. No doubt there are numerous other examples completely unrelated to coders and whiteboard tests where A implies B and B is highly correlated to A, but their existence tells us nothing about coders and whiteboard tests and doesn't justify a blatant fallacy of affirmation of the consequent.
Here's something to consider: just because someone is good at writing compilers or designing a language, that doesn't entail anything about the quality of their arguments.
I’m very curious where you get the idea that flying does not involve algebra or trigonometry or calculus.
How would you calculate a crosswind component from the runway heading and reported wind speed and direction without trig? How would you think pilots measured their distance to non-directional beacons before GPS and DMEs existed?
How would you solve for fuel remaining without algebra? How would you estimate the best speed to fly with a given headwind to maximize fuel onboard at the intended destination without calculus?
A very basic principle of glider flying involves finding the tangent to a curve. Is calculus not applicable there?
Fuel consumption is often estimated by numerical integration of fuel flow rate. That doesn’t require an analytic solution of the integral, but I think most pilots have at least a passing familiarity with the concept.
> Furthermore it's categorically false that you need to pass ground school before you're allowed to fly.
I interpreted the parent’s statement to mean “before you fly solo.”
> it's not like there's any calculus or algebra involved
There certainly is when you're navigating.
Some of the more advanced math is boiled down to specialized slide rules, though these days they'd use a computers.
For example, the fuel consumption rate vs range is not a linear relationship, because burning fuel lightens the airplane and so it can go faster/further.
but to get to the city we had to take it was a 2 hour long drive through twisted roads that made me carsick. I lived in a small town far away from the city. By the time I got there my breakfast I had quickly eaten gave me a stomach ache, I had woken up far earlier than usual and not gotten my 7 hours too. I certainly would have done a lot better if I had lived in the big city.
Another factor: if you wanted to pay the fee you could just take the test over and over again until you got a great score. So kids with poor parents obviously had a huge disadvantage. Also kids who had the time and money could study for it with prep books - I did, while some of my friends were flipping burgers while still in highschool. Its not surprising I got a higher score than them, but it said nothing about my intellegence or understanding compaired to theirs.
Fee waivers are great. A lot of poor parents are too busy / don't have their shit together enough to learn about them & use them. A lot of poor parents won't even have their kids take the tests at all even if they are free, especially if it means taking an entire day to drive them to the test facility. They might not even know what an SAT test is.
Hm that's true, but the alternative would also be flawed, and possibly worse. If you could only take the SAT once and you were stuck with that score, then people would complain that rich kids could take all sorts of prep courses to get ready and to figure out what they needed to work on before taking the test, whereas poor kids would be figuring this all out on test day. There is no ideal way to do this, and the current system may be better than anything else.
Also, in a decent number of states, it's mandatory for all HS students to take (and I believe free to students as a result).
I would be much more ok with someone who failed the test but knew how to pilot a plane vs someone who aced the test but can’t figure out how to get the engine started.
The ironic thing is that feel good policy can be more racist and prejudiced than any test.
They experimented replacing tests in my country with giving teachers the final say- it was a fucking disaster. Human beings are quite terrible at objectivity.
Why is a lossy testing filter better than just failing out those who can't make it? Maybe allow for larger freshmen classes and smaller latter classes or adopt community colleges and have all students start there and advance into the UC system sophomore year on. Instead they bring back what is basically an IQ test for admission.
I personally strongly disagree. I think it's much better to be given the opportunity to do the actual work, rather than to be required to do the pre-assessment song and dance. And if there are actual prerequisites that a person hasn't previously passed, they should be allowed to be tested on these specifically.
It depends on a lot of things. If they've also applied to another college that's better suited to their ability, admitting then to a school where they'll likely fail is not really doing them a favor.
Agreed. I've seen some interesting data that kids who could've been successful premeds or engineers at their state flagship but instead eke their way into Harvard or Princeton (sports scholarship, legacy etc) will instead graduate with a flaky studies major because they couldn't cut it in their intended major and the switching costs of transferring were too high.
There are downsides if you end up a small fish in a big pond.
Is the actual premed or engineering coursework at Harvard or Princeton really that much more rigorous than that at a flagship state school? I'm doubtful.
I'm no expert on that particular situation, but I compared my syllabi and projects from a state flagship (not Georgia Tech, Berkeley, or UIUC) with my brother's from Carnegie Mellon, and the expectations of first/second year CS majors were extremely different. Sometimes we used the same textbook but CMU covered more chapters and their projects were more involved. Some courses that typically waited until senior year at the state flagship were common to take spring sophomore year at CMU. There were a lot of courses that were numbered as undergraduate at CMU but covered content that was only covered in graduate courses at the state flagship.
Longer answer - in the other reply to your doubtfulness.
This is true across the entire US system, some state flagship universities curricula are so deficient that graduate level at better schools wont even consider the bachelor level diplomas from those schools as eligible unless the applicant is top n% of the graduating class, where n is a low single digit.
The admissions committee may never publish or say it directly, but for MANY state flagship universities the B.S. level maths and science courses are simply insufficient fo higher level studies at leading schools.
Thus, companies with hiring and leadership that is aware of these conditions will also simply pass over applicants with degrees from flagship state universities, much the same as they do with online diploma mill "Graduates."
My take on this situation is that as primary education outcomes worsened in the US, state universities modulated the coursework to match the readiness of incoming students in order to keep enrollment 'available' to everyone and extract revenue from the student loans system.
The "Princeton and Harvard(s)" were differently motivated, in that they never had a goal of admitting the majority of High School graduates, and thus were not required to lower levels of educational rigour to meet eroding conditions in primary education.
> The "Princeton and Harvard(s)" were differently motivated, in that they never had a goal of admitting the majority of High School graduates, and thus were not required to lower levels of educational rigour to meet eroding conditions in primary education.
It's easy to find recent reporting on claimed grade inflation, reductions in rigor, and students who seem unprepared or unwilling to do the work at Harvard and Princeton too.
Speaking as a professor: the filter is really helpful. Having students struggle for two years in a program they're going to fail out of is terrible. I've seen it happen - and I saw more of it happen for the years CMU also stopped requiring the SAT.
The SAT is a very imperfect measure but it turns out a lot of the others are even worse.
Honest question: why is it terrible? I would assume that if someone is capable enough to struggle for 2 years, they stand a real chance to struggle for 4 years and graduate. And seeing how the job market rewards a mediocre diploma from a good college significantly more than a good diploma from a mediocre college, it seems like a very rational decision from the student's side. And if they do fail out, they should still be able to enroll in another program, and presumably get a decent grade there, no?
>Having students struggle for two years in a program they're going to fail out of is terrible.
Putting a body every a seat for those profitable 500-person lectures taught by an adjunct and graded by TAs who are all paid peanuts and then having them wash out before those expensive 50-person high level classes and esoteric electives taught by tenured profs and accompanied by lots of expensive lab time is great for the university's bottom line though.
Good point, and that was for me the promise of the original MOOCs 15 years ago - for students to be able to take on these MIT/Stanford courses, with the exact same workload and see if they could handle them, without any additional cost to the school and without the student having to upend their life yet. And if they proved to themselves that they could and are willing to, then enroll, and use that as proof.
There are still some online courses that try to do this sort of thing, with a particular example being the University of Helsinky's Full Stack Open [0] which offers post-hoc university credits for those who complete the course, but it seems that the dream didn't quite materialize.
If you completely ignore costs, and the negative affects of having 50+% of a class be unprepared and taking time from the other students, sure, that would work great. Seems like a bad alternative to standardized tests considering students will then have to pass tests once in college...
My wife has a civil engineering degree. There were a number of courses where partial credit was not permitted and the final exam was 2 questions. It was common for students to take those courses 3 or 4 times before passing. Giving a failing grade to only 1/3 of the class might get a professor investigated for making the class too easy.
Unless, somehow this is part of an advanced certification process? I really can’t imagine professionals not knowing how to study for an exam like this. Medical school shows what motivated people can do.
In America, dude. Pick your favorite engineering school and read syllabi for core classes such as statics. Read department policies on number of tries you get on core classes before you’re kicked out of the civil engineering program. They take certain things very seriously.
The incentives change when the school becomes a mechanism to sell a chance to increase probability of immigration rights at tens of thousands of dollars per year, especially when the pool of domestic kids is shrinking.
Failing 1/3 of a class if that cohort is genuinely deemed not qualified enough to pass shouldn't be a problem by itself.
But then it raises questions like "are they really unqualified or is the testing methodology inadequate?" and "why was the system unable to provide the necessary growth to such a high slice of the class?". And then the easy way out is to just cherry-pick which students enter the system at all.
> High graduation rates are an important metric to administrators. If a professor gave a failing grade to 1/3 of the class they would be in hot water.
I remember practically every single instructor/professor on the first day of class during my freshman year of my undergraduate study said something along the lines of "I have no curves. Your grades depend on you and nobody else. If the whole class does well, everyone can get an A. If nobody does well, everybody can fail."
So I guess this was more motivational to get us to study rather than stating facts?
It really depends on what kind of class it is, but at most schools:
* If 1/3 of calculus physics for engineers fail, they take it next semester.
* If 1/3 of gen-ed physics for poets fail, the professor better have a good explanation for the provost.
* If 1/3 of physics for pre-meds fail, the professor better have a pretty good home security system and might want to think about having the family stay in a hotel for a few weeks.
I remember a first lecture when I started my CS studies, where the professor said something like "look at the people to your left and to your right, it's likely that at least one of you will drop out by the end of this year; it's ok, this is not for everyone; if you truly believe this is for you, put in the effort and you'll make it"
1/3 isn't that bad in the late 80s/early 90s at the UT Austin CS department. Only ~30% graduated at the time. The orientation was literally "look to your left and looked to your right only one of you will graduate." They weren't joking!
That depends. Some schools actually cap the number of students permitted to continue. They fail a certain fixed number or percentage of students below a threshold, even if the raw score is good.
Disagree. Hiring and firing is better than a bad interview process. The reason we don't have that is due to regulations and litigiousness (and the laws that facilitate it).
IMO failing to get the opportunity is worse than getting the opportunity and failing at it.
SAT score is known to be predictive of college grades. Is it also predictive for whether a mid-to-late career candidate will pass a phone screen? It is used for early career candidate filtering in finance, but I have not heard of anybody caring beyond that because of the availability of signal on the actual tasks they will be performing.
I feel like it's a common misconception that IQ only tests some narrow intellectual abilities like math and logic, likely because online tests and fake IQ tests tend to focus on those kinds of puzzles. Actual IQ tests actually do place heavy emphasis on language skills like vocabulary, verbal abstraction and comprehension.
> Is it also predictive for whether a mid-to-late career candidate will pass a phone screen?
I’m sure having access to your own SAT scores (or even remembering what they were) is highly predictive of not being someone who it would be illegal to fire because they are too old… which is probably the point… and why I’d expect most HR departments to shy away from this requirement.
how many elected leaders are in STEM? would high SAT scores and grades exclude many US presidents and congressmen? (yes) winning elections seems kind of important to me. so if you were just like, selecting for leadership - and many of our leaders are brilliant people, just not in the sense of being good at taking tests - would that be good or bad? or... what is your real opinion? what are you actually mad about?
obviously the UC system should give spots to the kids who will use those spots the best. but it is very hard to define what "using spots the best" means.
One would say the tests (and job interviews) should have been designed with the original intent of testing candidates AS IS, i.e. preparing specifically for such tests should have been considered as cheating... But at some point it turned into prep gymnastics, and measuring how desperate the candidates are.
That's not how it plays out in practice. There is overwhelming evidence that students who otherwise excel academically score fairly mediocre SAT scores on their first attempt and then jump substantially after weeks of targeted practice and/or tutoring, even though they didn't learn anything new in the classroom.
If attention in class were all it took then that improvement couldn't happen. What changed was familiarity with the test, not classroom focus.
I guess times have changed. When I took the SAT, I did zero prep work. Nobody else I knew did prep work, either.
Paul Graham recently posted SAT advice along the form of "when you finish the test and have more time, go back over the test and check for mistakes."
I was kinda astonished at this advice, isn't it obvious? A strategy I also employed was to do the easy problems first, so I don't miss a question that would have been easy. Apparently this has to be explained to people?
I suppose prep work would be fine for the students who didn't pay attention in class.
Yes, if you took your SAT among a cohort of people where none of you practiced for the SAT, then what you're saying holds true.
That's not really the case anymore. Top tier students nowadays prepare for the SAT, they don't go into it blind and haven't done so for the better part of 20 years.
That report also references another study showing that each hour of tutoring was associated with an increase of 2.34 points on the SAT score which unfortunately is behind a paywall:
I have seen company descriptions in job ads that list college achievements of founders. They are invariably young Asian men. I understand that it's a cultural signifier and don't judge them. But, I also understand that I will never hear back from them because I don't share that background. So, I never apply to any job listing that references college experience of either side, other than wanting a degree in general.
Someone once told me that American work culture used to be based more on intern-ship / apprentice type hiring but now obsess with formal degrees. I wonder how much of this shift in culture is influenced by the Korean, Chinese and Indian immigrants, as a formal education is a prerequisite to compete in these countries' job market? For example, it is quite common in India for employers to ask for our 10th and 12th standard marks / grade (because these are national exams) along with college grades - to apparently gauge "Consistency". Fluctuating performance, a break or dropout years all negatively impact you and can be nerve wracking for many freshers, until they manage to get some work experience. It is somewhat disappointing to see this culture permeate to America too, even though I feel quite conflicted about it - after all, everyone does want to hire the best / most competent / reliable candidate; but the other approach - a vocational kind of training - also has its merits and seems to have served American companies well too. (Zoho in India is experimenting with this kind of hiring in India where they are hiring high-school students, mainly from rural areas, and offering them a work cum study program. They don't get any formal diploma or degree though - https://www.zohoschools.com/ ).
> I wonder how much of this shift in culture is influenced by the Korean, Chinese and Indian immigrants
I think there’s an influence, but it’s amplifying a pre-existing trend. Bureaucratic societies favor formal credentials. The U.S. has become much more bureaucratic since the mid-20th century, and credentialism has grown. Reliance on degrees and other formal credentials also enables the universities to achieve political goals through admissions and grading policies. Asian immigrants in the U.S. have readily adapted to that system.
how many uncredentialed people's families go to private school with what you pay them?
it's one thing to hire some people for some roles with this sort of, diamond in the rough mentality. obviously that can be a good idea. but in my experience, if you try to take leadership in that way, you are spending most of your time persuading other people that it's a good idea, which they will reject, and consequently, it's of little influence.
then you look at people who become bosses who lack credentials (or whatever), and you find out it's only because they drop out of their competitive colleges to be fabulously successful. the true weirdos out there - whatever held them back from "credentials" doesn't stop them from becoming fabulously wealthy, but rarely do they go and hire anyone else. like they do not create enterprises, teams or even families. do you get it?
What you’re describing is the culture of credentialism. You can’t change it by yourself and it’s hard to fight against. But that’s my point.
The problem with credentialism is that the credential becomes the end, not a means to an end. There is a huge problem in India that there are far more people with credentials (often of dubious worth) than jobs for those people. The culture is very focused on “the track,” where you get the credential then go to the job unlocked by the credential. But the problem is that there’s very few people actually starting the businesses and creating jobs that would hire degree holders.
I have suspected the influence is real. For a reference point, the majority of students at top tier US universities are Asian at this point, broadly. Not every top tier university but there's a trend to have about 30-40% Asian American students and then roughly 1/3 international which is heavily weighted toward China and India. This constitutes the largest group usually. So it's quite likely that universities adapt to this and hiring practices begin to reflect an intense interest in exam taking and credentials.
The thing about it is I view it similarly to how in the past "well-roundedness" and "leadership" was part of hiring and admissions. We laugh at that now but my understanding is the SAT score can be improved with long term studying. So intensive SAT studying seems like a new thing that isn't evenly practiced among people in the US. So at worst SAT score usage seems like a way for an elite group to preserve and replicate itself. I have no SAT score so I feel somewhat outside of this debate and have no experience with it.
I don't think other cultures are driving much of the trend for educational check marks. I remember my dad and uncle talking about the awkwardness of being asked where they did their MBA's by colleagues/clients in the late 90s/early 00's and them trying to figure out how to navigate that as they didn't even have bachelors degrees. And I doubt whoever replaced them when they retired had less than an MBA.
Between increased regulation and greater competition for jobs, the degree requirements keep going up in a lot of/most industries. I also think there is a tendency for those that have reached a level of educational attainment to push back on others without equal numbers of checkmarks. Once a role is populated by MBA, PhD, MS or even BS, individuals don't like to see others doing the same work with less credentials. Maybe it's a 'I had to do this, so you do too' mentality or a sense that it devalues their own credentials.
I think it is a positive for an employer to ask for an SAT because it tells me right away I don't want to work for them. Once (a long time ago) I tried to upload my resume to apply for a job. The web page started asking me very basic questions, like a basic aptitude test. I was out. Tell me you do not know how to find and evaluate talent!
I think this is highly age-dependent. I took the SAT well over a decade ago and have significant work experience since then. It would be odd to require me to put down my SAT scores, which I don't even precisely remember.
But if I were < 5 years out of college, and especially if I had gone to school during COVID times (when SATs were not required by many colleges), I would completely understand why an employer might ask.
Basically, colleges used to act as a filter for SAT and other attributes. During the 2020-2025 period, they admitted students under fairly different standards, due in part to testing challenges and social movements.
It makes sense for an employer to want to do a little more diligence to ensure that students who were admitted during this period are similar to students admitted during the prior several decades.
It was required at my first job in 1989, for entry level actuaries.
In fairness, part of job performance was passing the actuarial exams, the first two of which were calculus and statistics. I imagine testing well on the SATs for a math or EE degree (what they hired) was a good indicator of passing tests.
No only sat scores but specifically they ask for the percentile band of your high school maths and hard sciences scores.
Not even kidding. I’ve been in a staff level+ role at 3 of the 5 faang. Applied to canonical because their products are interesting. I’m ~30 years past high school and i get hit with ‘what are your high school maths scores’. I answered the online form honestly and got a rejection email immediately on send. Phew!
Not at all kidding on that and there’s screenshots of the literally insane questions they ask online.
lol I know, I applied for them around 4-5 years ago and thought it was a joke... I did the first part which was some gimmic IQ tests and then I had to write an essay (which I didn't) !
> similar-to-me bias (I like you because you're me!).
My first boss in the 90s eventually told me why he hired me.
"I assume that everybody at their first job with a CS degree have more or less the same level of technical competence [which is not much IMHO] so I ask which are the last books they have read. You told me a few, I usually get none, so I hired you because I hoped that talking with you would be interesting."
At least a similar-to-me bias builds a pleasing work environment because of homogeneity.
You cannot use the SAT as a metric to compare different cohorts. SAT scoring has been revised many times over the years. When I took it the highest possible score was 1600. From 2004 through 2016 the highest score was 2400. Now it is back to 1600 again. Plus, both the content and the format of the exam has changed many times over the years. At times, there was no essay requirement, at times the essay was required, and at times it was optional. Hence, each year the examination produces a different distribution/histogram of scores even if you normalize the 1600 vs 2400 difference.
Even the scaled score is not that informative (and perhaps crosses the line on age discrimination) because for older workers the population of people taking the SAT was much smaller as a percentage of high school grads (and presumably weighted towards higher IQs). It's also why there were so many fewer perfect SAT scores -- smaller population in the bell curve.
Yeah, test prep was considered more for people who were worried about low scores. 1500 vs 1600 wouldn't make much difference in college admissions at that point.
I somehow doubt that the people that would ask for SAT scores would actually be the sort to think about how those numbers should most effectively be used.
Paid test prep is generally considered to be more effective on the current SAT than it was several years ago which also makes it harder to compare across years.
Which requires them to explicitly ask your age outside the bounds of qualification for a job (over 18 etc). Which ends up opening them to age discrimination lawsuits.
It does not require them to ask about your age, just the year in which you took the SAT. As other commenters have pointed out, this can range from 12 to 17.
Also, they could just ask for your SAT score and any relevant info (if you took it during COVID from your car, etc.) and then you could disclose whatever context you wanted.
I don't kid myself into thinking that employers can't tell roughly how old applicants are. If they're asking for SAT they could also ask for college transcripts/graduation info. That's going to reveal the approximate age of many candidates right there. Finding out what year you took the SAT will add 0 info in most cases.
While I generally agree with you that employers should absolutely not be asking for this sort of information, they're still going to be able to make a very good guess at your general age by the dates of your education and employment.
Unfortunately, there isn't really a viable way to prevent employers from finding out your age from your resume/CV.
When I took the digital SAT a couple years ago, we had access to the Desmos Graphing Calculator during the whole math section.
The entire point of the exam was to test whether you can read a math question, input it into the calculator and select the option that matches the result within 60 seconds. If you get a couple questions wrong, you drop hundreds of points. I don’t think it was a valuable test whatsoever (and of course, it biases to students who can afford time/money for thousands of practice questions to improve this “skill” through repetition)
The English reading/writing section was much more interesting, but again, the time limitations make it a skimming test more than anything else.
Many universities allow you to ‘superscore’ multiple attempts, to combine a math and RW score from different SATs. So again, scores bias towards students who can afford to take one test dedicated to math, and another dedicated to English.
> When I took it the highest possible score was 1600. From 2004 through 2016 the highest score was 2400. Now it is back to 1600 again.
that's because the 2400 score included an 800 point writing component; the math and reading components remained scored at 800 points (and still do now)
As a manager, there are several qualities that I value highly in an engineer, and they all happen to begin with the letter C: Competent, Consistent, Curious, Caring, and Clear Communicators.
While SAT scores might act as a proxy for competency and possibly curiosity, they're not going to tell you much about whether the person is consistently reliable, whether they care about others and cooperate well, or whether their vocabulary or literary analysis skills have any correlation with their ability to read the room and tailor their communication to their audience.
If I were giving these job posters the benefit of the doubt, I would guess they're including this requirement for the same reason that musicians request particular colors of M&Ms in their riders. They want to weed out people (or bots) who aren't paying attention. Nevertheless, there are better ways to do that than demanding (and presumably filtering by) teenage performance metrics.
I met an HR manager who had worked for a local but well known company with a reputation for caring about things like GPA and SAT scores. She told me that remembering your SAT scores after college was a sign of a competitive attitude.
In the same sense that the high school quarterback continues to talk about the Big Game now that he’s an overweight retail employee making minimum wage years later.
I've seen what happens in engineering with those with low SAT math scores. They need others to do the math for them, or they just wing it.
I remember one who was trying to reduce the noise in an electronic amplifier. He spent days trying random things. Another engineer asked what he was doing, did a quick calculation, and put in an RC circuit that solved the problem.
Off-topic, but can an RC circuit really reduce noise? I can see how it would reduce distortion (which is not the same as noise), but adding passives is surely going to increase thermal noise?
You are correct, but an RC circuit is part of the amplifier circuit and you can do a lot to improve the signal to noise ratio (SNR) depending on frequencies/signals are trying to amplify versus the ones you want to filter out.
I see, so you want to band-limit the noise, and are willing to accept potentially raising the noise floor in the band-limited range, the idea being that if the original noise is sufficiently broad-band, this will still increase SNR?
Fair enough: I was reading the original comment as referring to (thermal) noise intrinsic to the amplifier design itself (as opposed to noise due to imperfections in DC supply or due to coupling with some external transmitter), but this makes more sense, and we should consider the whole system anyway.
The problem here seems to be that the person was unwilling or unable to ask for help when they needed it, not that they don't know math per se.
I don't know how to do that either, but "winging it" is not something that would occur to me. First I'd Google it and try to figure it out. If it turns out to be nontrivial, I would just ask for help.
And I wouldn't feel the least bit bad about it. After all, those same highly educated folks need my help with e.g. git a lot more often than most software needs serious math :)
You forgot that the SAT requirement is not exclusive but an additional data point. While I agree that it could narrow the path for truly good employees, I’d argue that an additional data point like SAT (+ GPA) could tell the employer a lot about consistency of the applicants. Or at least an interesting talking point (“I see you got a very high SAT score but your GPA was lower, what happened?”), if they care.
I think it could serve the purposes of hiring fresh/young graduates. However, it’s still weird if they requested it for people already 5-10 years or more in the industry.
I had a friend with an MS show up for first-day of work for a job that asked for SAT scores on the application. HR said "we never got documentation for your SAT scores, can you provide that?" He was on the phone with his mother, having her go through a filing cabinet when he realized that he didn't want to work for a company that was this serious about SAT scores when hiring someone with a post-graduate degree.
That does seem wild. Out of curiosity, did he also have to submit GRE scores, which would be closer in time and more representative of his current knowledge/skills?
Worker w/PhD was asked by Quebec during residency interview to produce High School grades. Bureaucrats will be bureaucrats independent of language I guess.
I would never ask for them since its so cringe. But SAT scores correlate to IQ at .81, and IQ is one of the few things that strongly correlates to knowledge work performance positively. There is probably a lot of alpha from knowing candidates SAT scores. Its more useful than knowing the college they went to.
That's r = 0.81 right? so R2 would then be ~.66. Meaning that 66% of the variation in IQ scores can be statistically explained by variation in SAT scores.
It's really high for psych stuff. If you even get r=0.5, you've got a great result there.
But it is important to note, I feel, that SAT maps to only about 2/3 parts of the IQ score, and IQ score is also a quite fuzzy measure here for things like knowledge work job performance.
I do agree though, you get quite a bang for your buck just reporting these numbers.
But, if you explicitly tie money and compensation to the SAT score, man, that is setting up some very perverse incentives around it. If it adopts widely to do so, then you're gonna get some really strange interaction effects there.
> and IQ is one of the few things that strongly correlates to knowledge work performance positively
i think you mean that it correlates to pay. nobody knows what you mean by "knowledge work performance." reviews of your peers also correlated with pay. often it is not the smartest person who is the most popular. so... do you see how you said something kind of meaningless?
> nobody knows what you mean by "knowledge work performance"
I actually was pretty easily able to deduce what they meant by "knowledge work performance".
It's understandable to be frustrated by not knowing something, but to claim "I don't understand that and therefore no one does and you're being nonsensical" is a bad look.
Consider responding with curiosity rather than defensiveness.
A very good metastudy is "The Validity and Utility of Selection Methods in Personnel Psychology" (by Frank L. Schmidt and John E. Hunter). It summarizes 100 years of research on predicting job and training performance. It makes a very strong case that General Mental Ability (GMA, their word for IQ) is the single most valid predictor of employee success on the job, not just income.
That's not what that paper says. Work sample tests are more valid than GMAs; the paper just presumes they're too expensive. Meanwhile, we don't have to axiomatically derive any of this: we know that relying on general cognitive assessments to prospective software developers wouldn't work. That's why almost no firms use them.
If you exclude work sample testing from your analysis, all this paper is really saying is that active examination of candidates beats subjective interviews and resume scans. Well, obviously.
The job description touts investment from some members of the "PayPal Mafia." For some odd reason that fact that and the SAT requirement combine to make this whole thing feel kind of normal.
> Low ego: There can be significant economic upside to this role as things progress, but the nature of this role day-to-day may involve things that can be less exciting / more mundane (it's a job, after all!)
> Please note that we will also rely significantly on both solicited references (where you introduce us) as well as unsolicited or "back-end" references (where we do our own). For the latter, please rest assured that we will never contact a current employer without first getting your permission.
References already give me goosebumps. Having them reach out to people who haven't given you permission to be a reference sounds like a recipe for disaster.
If they do sensitive work for the government, it'd make sense that they'd do those back-end references. Also if they are in high end finance where you want to weed out people who have demonstrated moral flexibility was well as total lack of it for certain things.
> If they do sensitive work for the government, it'd make sense that they'd do those back-end references.
If they do government work that requires clearances, the clearance process already covers this sort of investigation on its own.
In any case, they are free to do whatever background checks they want within legal limits, but I'd never apply to a company with such ridiculous hiring processes.
in my experience it (announcing to people in your network that you are job hunting) is precisely what you should be doing. Some might even point you to the next opportunity. I understand your concern though and have personally only shared it with close contacts (guess it is an ego thing).
They are obviously using the SAT as a safer alternative to the legally dubious practice of IQ testing which can lead to running afoul of the ADA and EEOC. I'm not sure it's much safer, but I am positive it's less safe than doing timed leetcode. At least leetcode problems can be painted as relating to the job.
Additionally, the SAT is a shitty IQ test that is constantly crammed for and cheated on. I remember my SAT test. I was the only person in the room not openly cheating. The teacher proctor didn't care. Higher scores mean better students, more funds, higher home prices, bonuses, and a litany of secondary effects. That's not even including people that pay professional test-takers to do it for them.
The software industry needs to let go of their obsession with finding 10X ROCKSTAR L33T programmers. They never will though. It has gotten worse every few years for decades, and the problems are almost entirely managerial.
IQ testing isn't legally dubious. The idea that it is is an Internet myth. There are a couple household-name corporations that administer general cognitive tests for candidates for some roles.
More companies don't do it because it doesn't work well.
This by default includes a whole bunch of people who didn't take any kind of standardized tests (most notably, immigrants).
The (albeit small) country I'm from doesn't do any. Reasoning was that standardized tests create an environment where teaching is merely done to create good test scores, not to actually teach.
It's weird that people think AI breaks the concept of work sample testing. Work sample testing isn't "about" programming, and predates the profession of programming. You can (and some companies do) work-sample test sales account managers, customer support, accountants, whatever.
AI changes the underlying job you're testing for. So, obviously, the tests you might have been using pre-AI won't work anymore; they're testing something that isn't really the job anymore. Update your tests so they're about the real work again, that's all. For coding, that probably means assuming (or requiring) candidates use AI to do your assessment.
What AI really does mess with is conversational/interactive interviewing. We do all our interactive scripted interview on Slack, but I can imagine us having to end that practice and return to face-to-face.
I've seen you comment along those lines before, and I think you're both right, and terribly wrong.
By all measures, the way you describe any hiring processes you've designed and had input on designing the metric and the work sample, probably is both high in signal, and not very arbitrary in scoring. They sound genuinely and refreshingly good. Maybe one day I'll interact with one :)
But man, that is very much not how work sample tests work in _a lot_ of places out there.
Places with actual, formalized rubrics for what constitutes a "pass" or a "good" score are very rare, it's almost always just based on vibes of whatever person is reading your code. If there _are_ formalized rubrics, then they have "suggested allowed time" that is incongruous with what the requirements for a "pass" actually are.
All of that is downstream, fundamentally, from the design of the tests not being taken very seriously by the people doing that. And because they're not actually thinking about it very deeply (or they're not allowed to because of time constraints or whatever); "sit down and redesign this task from first principles thinking about what AI enables now" is just not something that happens, and you just get people increasing the scopes of the project blindly.
I agree that there aren't many places that are serious about work-sample testing and far too many that using them as a hiring hazing ritual, but that doesn't make me "terribly wrong", just more distinctively right. We've hired resume-blind at Fly.io for 6 years using work samples with fixed rubrics.
Between 2005 and 2015 the maximum score was 2400 instead of 1600. Assuming anyone who got <1600 during that period wouldn't admit it, you now have three well defined buckets.
But of course this is a lot of unnecessary steps compared to the usual method: length of work and education history +18 years.
The extra 800 points didn't count for a few years, I didn't practice for it and as expected did bad. Don't remeber the score except I did well enough on math
Hah, you did the one at 12 also? I too remember both, and it was also 30 years ago. I don't know why, probably because they were really important numbers to teenagers, and they say you remember things that happened to you as a teen more than any other part of your life.
Sneaky age filter? You must be young enough to remember your SAT scores.
I can remember mine just fine.
If you're really looking for smart people, use "Answer this word problem in two or more paragraphs. Write your answer on the sheet of paper provided. In cursive."
The signal being that the smart people will refuse to jump through this hoop in your inane process because they have a lot of other opportunities to choose from.
Are people who were forced to learn cursive smarter? They're just older.
Doing this may well expose you to age discrimination lawsuits, since it's just sneaky indirect age filtering.
Another example would be if you required a minimum SAT score of 1601. Sure, someone could have gone off and taken the SAT as an adult or a young child but in reality it is mainly an age filter.
Are people who were forced to learn cursive smarter?
By definition, people who know more things are smarter than people who know fewer things. That's just how it works.
For centuries, people have striven to improve themselves through the acquisition of knowledge and skills. It is a quirk of recent generations that so many members take pride in their lack of knowledge.
I'm repeatedly bewildered by my Millennial colleagues who proudly say "I don't know what that means," or boast "I don't know what that is" with no sense of shame.
Imagine, you have two people. Person A knows cursive, person B does not. Person B knows the ins and outs of Newtonian physics, person A does not.
Which person is smarter? Which person would the cursive test say is smarter?
What you seem to have mistook for people not knowing things without shame, is people valuing knowledge not by the preponderance of its quantity but by its total when multiplied by its utility.
Otherwise I do not envy the shame you must feel at lacking the knowledge of which plants are edible, how to.clean a carcass, how to fashion a needle from bone and an axe from stone, the mixture of clays to use to make your bricks, and all manner of other once-necessary tidbits whose usefulness has lapsed for the general population.
Once upon a time, I thought I wanted to learn cursive handwriting. Except this version of me was already in his 20s, would be out of school in a matter of months, and quickly realized the skill would be of such marginal utility in the future that it wasn't worth the hours spent tracing out giant letters like a kindergartener every day.
One could learn this skill in their 20s or beyond, but there's an opportunity cost – why not something else that would actually improve work performance, or that you enjoy doing?
I still wish I'd been taught in elementary school, though, because it would've been really useful as a student. Some of our teachers discreetly handed out practice booklets to students who'd "expressed interest" (their parents taught them the basics and teacher noticed); most of us were not so lucky.
I learned in Brazil. Here in the California I asked my son's kindergarten teacher if she would teach cursive, and she said they don't teach calligraphy and I've never seen it described this way, but she's right.
Aside from the fact that they're judging you based on who you were as a minor, it's interesting how it was a startup that asked for this and not a large corporation, given that modern US public schooling is more optimized towards getting you to work in an large institution than a startup.
Same schedule every single day, you get a lunch break, exams are designed in a way that there's only one single correct answer, outside of the box thinking is wrong, and you'll get at least a B+ if you just do your homework everyday.
>Each applicant took the test under unknown conditions. You don't know [...] whether they walked in cold or worked with a tutor for months, or whether they came from a perfectly stable home life or found out the night before that their dog died/parents are getting divorced/worked the night shift at a grocery store to help pay rent. Maybe they struggled with addiction and rebellion in their youth.
I'm pretty sure this is (part of) what's being filtered for quite intentionally.
They made me do an online, timed IQ test, stressing that I should take it in my native language to not waste precious seconds understanding it in english.
It was horribly translated, every sentence was written like something this: "A and B are two broters/sisters. A gives B 3 apples and he/she/them eats one and returns one to he/she/them…" at some point one section had the instructions wrong so I did all the questions wrong. There was no way to change the language or re-read the instructions to try to understand what the original text might have actually been.
They're known for asking odd questions like "how did you perform in math in high school" and "please justify your performance in math in high school". It was actually my guess as well before I read the post.
I like it. I also like it when companies ask for 10 years of [5 year old technology] experience, or say "there's more to working here than the salary!", or other red flags that make it easy to move to the next listing.
If you think my decades old SAT score is relevant, then I know all I need to know about your company.
I only had one company (D. E. Shaw & Co.) ask for my SAT scores. I was late-20s and had to have the recruiter repeat herself two more times before I understood what she was asking.
It was also the single highest density of talent I’ve ever worked, by a long shot. Crazy talented coworkers.
I would think it an inappropriate question if you're asking a PhD (or MS). You could just ask for a copy of the paper(s) or the dissertation/thesis. Some people improve over the time in the uni environment. Some people don't test well when they're bored, etc. Some just grow up.
I took both (since I could just use whichever I scored better on for applications), and the ACT is a much less miserable format. The test is broken up into much larger time chunks, meaning I could take a more useful nap after finishing a 50 minute section in 20 minutes than I could finishing a 20 minute section in 8.
Difficulty wise, the ACT was easier, but not by a lot. They seemed to have pretty similar predictive strengths, based on percentiles of people I knew lining up between the two, but obviously the ACT loses precision because of how coarsely it's scored.
A lot of responses pointing out various flaws with this question, including the fact that it can be used as a proxy for ageism, the fact that the grading scale has not been consistent over time, or that most foreigners will not have gone through the US education system. However, is it really that uncommon for Americans to never have had reason to take the SAT/ACT, such as, simply not going to uni, or going straight to work after graduating high school?
You only need to SAT/ACT for going to get into an university. And these days it seems to matter less. I looked it up and less than 50% of high school students take the SAT/ACT. Personally I never had any employer ask for my SAT/ACT scores since the 90s.
I just applied to Epic, the EHR company from Wisconsin, and I can confirm that they also ask for SAT scores. Thankfully I have my collegeboard credentials saved
Out of curiosity, did they require SATs, or just ask? And how many years out of HS are you? Seems like it would be crazy to ask for them once someone has finished college and has a UGPA to report.
Their pre-screen test was awful. Brain teasers, including some infamous ones like “if you have two coins that make 15 cents and one is not a nickel, how is this possible,” and moderately involved programming questions like “parse phone numbers from a file and record those with any of these area codes OR every other digit is a 2 OR they have multiple pairs of consecutive digits” and you’re given a blank text box with no formatting, IDE, or even non-word processing style indentation help.
Thankfully, the coding assessment does have syntax highlighting for plenty of languages, but in general I feel like face-to-face assessments are more productive. TFA seems to make a similar point
> Edison also subjected candidates to the 'salt test'. He'd serve candidates soup, and if they salted it before tasting it, he'd allegedly disqualify them. His theory here was that this proved they operated on assumptions.
I really don’t want to eat soup in front of someone I’m interviewing with.
> Take-home projects or a trial period of some kind. This makes the most intuitive sense by far: having candidates do a representative slice of the job gives you a solid idea of whether they'd be any good at it. Combining this with structured interviews was (before AI) considered a gold standard; you'd get a sense of who they are and how they work by talking, have a way to compare them pretty objectively to other candidates because of the structured and consistent nature of the interview process, and then you'd get a sense of how they apply their attributes practically to the job via the work exercise.
Unfortunately a lot of companies have over the last several years been using this to get candidates to do a project for free for them. If it's going to take more than a few hours of my time, I don't take project style interviews seriously unless compensation is added (which some companies do offer and is a big green flag).
Definitely been tricked into working for free a time or two.
Best middle ground I ever had was an interview where they impose a strict 1 hour (maybe it was 90 minutes, idk) time limit between when I got the prompt and when I emailed them back. Then they spent some time looking over it, then I had an interview with an engineer who had read my code and we chatted about it. Why I had made certain decisions, what corners were cut because of the time limit, etc.
Felt very fair. Not enough time to assign a valuable task, enough time and privacy that I wasn't under the gun like you are in a whiteboard interview, and it was pretty applicable to what I would be doing at the company. Solid interview. Didn't get the job, but respected the process.
This style works for me too. I don’t necessarily think the live coding interview is always the best presentation of what I know and some types of interviewers try to apply pressure in those situations I don’t feel is entirely fair, plus you’re often forced to use a third party webapp where it’s entirely out of the normal workstation flow I’d be more comfortable in (I use mostly text editors with plugins when allowed).
there has been a proliferation of “take home” exercises that are them just trying to pick your brain for problems they’re struggling with. I have had a “take home” project that led to a trial contract employment that required me doing a ton of work for free I knew they had used later, because I saw the trial API key I had set up for the project used much later on down the line. Never got paid, clearly work was stolen, not much you can do about it and there’s a real power imbalance there that’s difficult to really do much about. Was a tough lesson. When done right I think it’s the best way to screen for good candidates that may not be good live coding interviewers.
I do take the position that like, I get why you may have a culture where live pair programming in high pressure situations may be valued. But in most orgs the async nature of -> get requirement -> implement asynchronously -> seek feedback and adjustments -> iterate is far more realistic, but when you deliver something, never hear back, get hit with the whole “we’re going with another candidate” feedback feels like you just got ripped off.
While it does not come up in interviews, it does seem to come up during resume filtering. (Plenty of jobs I'm otherwise easily qualified for that I didn't get contacted for, even back when basically having a pulse and knowing code could get you any job you wanted.)
Didn't even read the article, first thing I did was go to comments and Ctrl+F "Canonical". Seeing several results light up was really amusing and validating. What a truly awful hiring process
I couldn’t even tell you my SAT scores. I took it and did reasonably well, but after college it just ceased to be information my brain retained. I don’t even know where my results are, so it’s completely lost information at this point.
If I may rant. My oldest son will be a junior in HS next year at a STEM magnet (Townview SEM in Dallas). I think even one of his electives is an AP, It’s kind of ridiculous. APs in HS seem pointless if you can’t apply the credit to your college degree plan. As if a University is going to let a STEM major out of calculus because they took the AP in HS…
At my highschol, "Honors" classes were the ones that didn't get college credit and "AP" classes allowed you to take a placement test at the end to get college credit.
Same here, but selective universities will often not let students out of requirements with AP scores. They may let you get credit toward graduation, but if you have to start with all the base level prerequisites then you've not made much progress.
Basically, universities are aware that APs have gotten easier and are not a substitute for college-level classes that selective universities offer.
It's always odd to me when filters are applied, that arguably neither influence, nor impact outcomes. From what I can see, SAT scores have effectively changed over time, such that reading comprehension has dropped dramatically, with maths abilities increasing (https://www.pewresearch.org/short-reads/2026/06/11/for-the-1...).
One (me) might argue that now, more then ever, reading matters. The age of AI, only increases the value of critical thinking and reading comprehension skills. Combine that with a (notable) 5-10% change in SAT scores, and it would appear that this practice is unlikely to unearth what factor is being filtered for.
Mind you, no one has ever not accused our industry of tunnel vision, and confirmation bias.
> Why would you ask for a self-reported, unverifiable test score that could be decades old at this point?
Because many colleges that used to reliably filter for them no longer do (or didn't during a several-year period).
It's true that self-reported scores are not the most accurate, but if I were applying for a job I would report honestly, on the assumption that they could easily request for the scores to be sent by the College Board. The risk/reward of lying does not make sense, at least in my case.
> > Why would you ask for a self-reported, unverifiable test score that could be decades old at this point?
> It's true that self-reported scores are not the most accurate, but if I were applying for a job I would report honestly, on the assumption that they could easily request for the scores to be sent by the College Board.
No, they couldn't, except by going through you (the College Board doesn't take third-party score requests.) You might be able to request that if they are recent enough, but not if they are literally decades old (well, not if they are ~21 years old or older.)
I am aware that third parties can't request scores. I was referring to the employer asking to have the scores sent, which the applicant would be compelled to do (or look like they fudged their original score reporting).
I'm also aware that the College Board doesn't hang onto scores forever. I doubt there are any employers who require SAT scores for applicants who took it prior to 200% (the cutoff indicated in your linked article).
Age and socioeconomic discrimination. They want only younger people who took the SAT from yuppy societal bastions. Kind of like how Handshake was full of jobs where the firms wanted only recent graduates.
Clearly they will pay a lot of money but be badly run. They want to avoid wokeism beneficiary kids and old folks. An actual job related aptitude test would be better. Half the nation didn't even use the SAT at that age. They used the ACT.
They want young coastal yuppies who were not wokeism beneficiaries at the DEI universities. Those universities broke signaling so making all their alumni show SAT scores instead of relying on the brand will mitigate the fallout.
ACT takers were from lame red states anyway. They want the young blue state brains without the old blue state baggage.
Tech companies more and more delusional every time, thinking they are special and that their product is so critical for society, and possibly more complex than operating a particle accelerator. So, you basically have to be an outstanding genius in order to have the "honor" to work for them. Then you go in and it's just a glorified vibecoded spaghetti CRUD. Not surprised.
Another thing making a comeback -- reference checks. I had to supply references for my first job in 1999. Then I wasn't asked for them again until 2024, and then for every job after that.
As I mentioned elsewhere on HN [0], younger generations are much more competitive now.
Visit and talk with undergrads at a top CS program like Stanford, Cal, UIUC, MIT, etc. The culture is different because this is a much more competitive generation. When the acceptance rate into a top CS program is in the 1-5% range and laurels like being a Valedictorian, NHS member, JV or Varsity sports team member in HS, getting a 2100/1500+ on the SAT, and taking 6-7 APs are now table stakes, you get a degree of viciousness, competitiveness, and steel-eyed execution that a lot of older Americans just aren't used to.
This mindset is the norm across Asia though - from the Gaokao to the JEE to SKY-or-bust. Honestly, I'm glad that younger generations are much more competitive now - pressure makes diamonds.
And honestly, the top 40-50 STEM programs nationally graduate around 30-40k new grads a year. Add to that respected regional programs and Veteran-to-Employment pipelines and you have a self-sustaining talent pipeline.
I disagree with the increase in competitiveness being a good thing. Excessive filtering at all levels has meant that eccentrics or absent minded professor types are not making it into research roles, and creatives or mad geniuses are filtered out before they have the chance to make an impact. There are a lot of people who are extremely bright and creative, but just don't have it all together the whole time from ages 14-25, and these days they have no chance of making it into research positions.
The system is rewarding conscientiousness and consistency over creativity.
> The system is rewarding conscientiousness and consistency over creativity
This assumes that you can get to the top via rote skills alone. Rote learning only gets you so far and most of those kinds flame out.
It's hard to describe, but once you meet actually talented people what you end up seeing is that they're just extremely diligent and deeply passionate about a topic and will continuously execute.
For example, when I was in HS I wrestled. Yes there were physical differences that could impact a sparring round, but technique and preparation was almost always able to outcompete base innate talent. Later, I ended up learning ballet the Russian style and it was the same - the truly creative types who were at Vaganova or Paris had already built strong fundamental and technical skills which allowed them to mix and match and create.
You cannot be creative without also being diligent and understanding fundamentals.
The "eccentrics" and "mad geniuses" are few and far between, and to find people with talent, you do need to use exclusionary tactics like scores and interview performance.
Not really. Outliers of that sort get dealt with as an exceptional case and it works. I've seen amazingly bright batchmates get into all sorts of programs without most of the qualifications, because they were a genuine math wizard (the kind that submitted errata to a standard textbook on some weird number theory stuff at age 15), but they didn't always score the best on many exams, especially when aggregated across subjects. IIRC he's a researcher now. All trends point to him being an eccentric old hag at age 50 ;)
The filtering system is meant for the majority case and there it works. The outliers get dealt with as outliers, which also works. In this case, he later asked the author of that textbook who he emailed with the errata, to connect him with the group he wanted to work in. Needless to say it was a very strong referral.
The US education has always been competitive. In sports.
Its school system has always been a state-sponsored daycare.
SAT/ACT tests reflect this. I can get a perfect score in SAT math easily. And I likely could do that as a kid (I never took standardized tests at school). I wouldn't have been able to get the perfect scores in the Chinese gaokao or Korean/Japanese tests.
> There are a lot of people who are extremely bright and creative, but just don't have it all together the whole time from ages 14-25, and these days they have no chance of making it into research positions.
This is just nonsense. Are you saying that we should kick out smart kids with high test scores to let in absent-minded students who care about only getting drunk so that they _might_ become great researchers in their 30-s?
To the topic at hand: it's way too easy to fluff your resume with nonsense like "Coordinated a responsible team for an implementation of cross-cutting concerns improving customer retention change by 12.23% across the organization". Test scores provide at least some objective measurement.
I'm not opposed to the use of test scores, it's more the requirement that kids have the whole package of grades+activities+honors societies+test scores consistently through a phase of life that is widely understood as tumultuous for many. We're selecting for robots.
There's a reason the west was so productive in terms of new scientific and technological discoveries in the twentieth century, and it's not that our scientists were the most consistent conscientious students who prepared extensively for exams and padded their resumes in just the right way.
And to add to this, learning itself is a _skill_. Working on a complex problem, looking at it from different angles, spending time memorizing facts, working on learning to paint fine lines - these are all skills that you need to master.
By not motivating children to do that during their formative years, you set them up for failure later in life.
Yes, there will always be exceptions, humans are extremely variable. But for the general case just letting children float along without any goals or competition is not a great general strategy.
I didn't really state my whole view, but hard exams like Oxford and Cambridge use make sense for elite colleges to use (rather than unrelated extracurriculars), but reforming the whole education system to be oriented around a single high stakes test like in China or Korea has its own severe costs. I do not want high schoolers to spend 20 hours a week at hagwons, and the current resume filler system is also terrible. I do not think declining admissions rates at elite universities reflect that the students are any smarter or more prepared than they were 20 years ago, but rather they are much more cutthroat about many things that are orthogonal to being successful adults.
Ideally we'd follow a more exam focused system more like the UK, though I wouldn't want to require all students to only study 3-4 subjects towards the end of high school. But something in the european model of IB/Abitur/A Levels, where there are serious exams in various subjects at the end of high school for all uni bound students, plus some special higher level exams for the most elite unis (in the vein of Cambridge's Sixth Term Examination Paper). We could probably repurpose AP exams to fill a similar admissions role to A Levels, and possibly use the AMC/AIME/USAMO more explicitly for admission.
Edit: this doesn't let me reply again, I think the chain got too deep. But the point is that we're not just using AP exams and USAMO scores, we're also using a pile of other metrics around extracurriculars, GPA, and honors societies, and the end result is stressed out children and not actually having better outcomes than we did two decades ago. Declining admissions rates at elite colleges do not reflect smarter and more productive incoming students.
Yeah, Chinese-style gaokao would be impossible in the US. And I think it goes _too_ far into the "competitive high pressure" direction.
There are many ways to make a more competitive and objective system. I honestly don't have a lot of professional experience with any particular one, so I don't have strong opinions on a particular form it should take. A European model is good, some kind of mix of Chinese+European would also be great. And ultimately, these systems would be more fair for applicants.
And the current topic just highlights the ridiculousness of the status quo. For most people in the US, SAT is the _only_ objective test score that they have.
> We could probably repurpose AP exams to fill a similar admissions role to A Levels
They are already being used like that in college admissions today.
> possibly use the AMC/AIME/USAMO more explicitly for admission
Already in use explicitly at most of the top CS programs today
> I do not think declining admissions rates at elite universities reflect that the students are any smarter or more prepared than they were 20 years ago, but rather they are much more cutthroat about many things that are orthogonal to being successful adults
The cutthroatness is what we should be optimizing for long term. Competition is what begets innovation.
Also, even at solid middle tier universities like UC Riverside, the calibre of student has increased dramatically over the past decade [0]
> The cutthroatness is what we should be optimizing for long term. Competition is what begets innovation.
The difficult pill that society refuses to swallow is that most of the simple innovations that are feasible under our constraints have already been innovated. There are no more individual 'inventors'. The source of modern technological innovation are lucky discoveries that manage to scrape from the bottom of the easy-to-find barrel, genuine one in a million genuises and (mostly) the hard work of large groups over many years. Even the brightest top 0.1% of people mostly won't be coming up with ideas that a thousand people before them didn't already have.
Yet people will not settle for this. Instead of acknowledging it, we'd rather apply the good old human model of 'if 1x of something is good, then we need 1000000x of that' and take healthy competition to transform it into a ruthless race to the bottom.
Past a certain point of competition, you stop selecting for the metric you care about (academic performance, curiosity, creativity) and start selecting for people who are willing to slave away sunrise-to-sunset to work on tasks they know are almost meaningless but that contribute to your selection criteria. The metric becomes the only goal. You're also indirectly selecting for wealth, because rich parents can afford to spend lots of time with their child, not make the child work, pay for all their extracurriculars and tutoring.
The diminishing returns are immense, and we can already see that in countries that never fought this race to the bottom. Their students are more miserable than ever, and for what? Frankly, forcing children to waste away their best days to make them clear arbitrarily rising barriers that we know are meaningless is unthinkably heartless.
> There's a reason the west was so productive in terms of new scientific and technological discoveries in the twentieth century
> it's not that our scientists were the most consistent conscientious students who prepared extensively for exams and padded their resumes in just the right way.
Instead, a large portion were immigrants or the children of immigrants who arrived in the US as part of trans-national brain drain from countries with strict education systems (eg. Hungarian Jewish Americans in WW2, Eastern Europeans in the 1980s to present, Asian Americans today).
There's a reason Asian Americans, Eastern European Americans, and immigrant African Americans are overrepresented in leadership and white collar industries despite the very real handicap of having extended periods time without US citizenship or a greencard.
Instead of optimizing for feel-happy edge cases, we should be optimizing for building the best talent where possible, and that requires being competitive.
> We're selecting for robots.
Frankly, this is insulting as well. Yes there are some late bloomers, but they are outliers. If they can truly succeed they would stil find a non-beaten path to succeed in a competitive ecosystem.
> consistently through a phase of life that is widely understood as tumultuous for many
Only to y'all "heritage" Americans. For those of us who are kids of immigrants, we learnt that life is a race, either you compete or you fall to the wayside.
Of late, US folks coming out of university optimize for money and head for things like finance - people who hire STEM people. Those more interested in the field than the bucks go for advanced STEM degrees. And if the Yanks don't go for the bucks, they go for the MD.
Yep, and we end up in a situation where the US-born people are working on "people skills", getting MBAs, and then going into management positions to manage the financialized companies.
Except that there aren't that many management positions. And once _everyone_ is doing complex financial stuff, you end up losing competitive edge against other countries.
> US folks coming out of university optimize for money and head for things like finance...
Not really, and I say this as someone who works in VC with peers in PE, Growth Equity, and other segments of high finance.
If you have the resume to get hired as an IB Analyst you will also get hired as a new grad SWE or APM at OpenAI, Google, or Roblox where they would earn the same or more than as an IB analyst with chiller work hours.
People overestimate finance salaries - it's the same as big tech with worse hours.
> And if the Yanks...
I don't think you live here in the States or Canada and as such haven't experienced our job market.
Please butt out of the convo.
---
Edit: can't reply
> Have you experienced the US job market outside of the Valley and NYC
Yes.
Before I switched to VC, I've managed teams and hiring for teams or product lines that reported to me in North Carolina, Georgia, Virgina, Texas, Washington, Massachusetts, and Colorado, and helped open my previous employer's Prague and Warsaw offices.
I also started my career outside of Bay Area or NYC tech before my stint as a staffer.
Additionally, the majority of tech hiring in the US remains consolidated in a handful of geographical locales [0].
Judging from your career path, you shifted from tech to money towards the start. I hope you were a good boss when you were starting up, as such people are rare and wondrous creatures. Forgive my choice of jargon - I divide the world into people who do the work and those who talk about the work. And the latter group tends to set the rules and collect the profits. I preferred the hacking. And tech != STEM, as biotech/pharma is rather different than selling ads and harvesting information.
> Judging from your career path, you shifted from tech to money towards the start
What the hell does that even mean?!?
I started my career working on on low latency computing at a networking company and was paid roughly the same I would have earned as an IB Analyst at JPMC. If I kept climbing the tech ladder I actually would have ended up earning more than I do today.
> I preferred the hacking...
Vast swathes of the tech industry are working on actually innovative stuff AND being paid top dollar, such as my stuff in HPC being dual use.
> And tech != STEM, as biotech/pharma is rather different than selling ads and harvesting information
I know. Not all "Big Tech" is AdTech. And salaries for scientists in the biopharma industry are comparable to big tech salaries as well (not for the SWEs in that industry though - they're cost centers not generating IP that matters).
> [...] you get a degree of viciousness, competitiveness, and steel-eyed execution
I think there's a lot of truth to that. (Aside: Many manage without the viciousness part. It's not their fault their parents lined them up with an internship and a research paper co-author in high school, and they're not jerks about it.)
Though the current generation of students didn't invent hyper-competitive. Before software engineering jobs (and startups) were high-income and high-status, you'd see that mentality among many people on track for Wall Street, for example.
Another example: Before CS was a go-to for the hyper-competitive, a mentor of mine actually switched from pre-med to CS, at an Ivy, because a percentage of pre-med students were outright sabotaging other students, and it turned him off of the field.
> that a lot of older Americans just aren't used to.
Though, there have been -- and hopefully will remain to be -- people doing it for the love of the field, who are not impressed.
Other than the genuine people being crowded out of admissions slots and fratbro interviews by Wall Street types...
If a Palo Alto helicopter-parented overachiever McDojo black belt tries to pick a fight... with a humble rope-belted person in Asia, who's studied martial arts for the love of it... the latter will chuckle good-naturedly, and help the Californian up off the ground.
> a Valedictorian, NHS member, JV or Varsity sports team member in HS, getting a 2100/1500+ on the SAT, and taking 6-7 APs are now table stakes
This is very true in my experience, except I subbed out Valedictorian with multiple varsity sports/student government and the SAT with ACT and I didn’t even get waitlisted at top schools.
My problem is it's self reported, so it ends up being a "are they smart or did they lie?" game. You can't easily verify it for anybody over 23 or so.
Canonical has a job application where you are supposed to rank yourself on a percentile (up to like 1 in 10,000) on how good you were at math in high school. It's a very easy way to incentivize lying, and also to hire people with an excessively high appraisal of themselves. There are a lot of people who are reasonably good at math, and have avoided humbling environments like the Putnam, and have convinced themselves they are God's gift to math, when in reality they were just the brightest kid in a class of 100 high school students.
If you want the job, can’t you just lie? Or are SAT scores something that cannot be faked? I dunno, I also say I know Kotlin when I have more experience in Java (and honestly I couldn’t care less about specific tech stacks), or that I know about tcp/udp when all I have is read a couple of (good) books about it.
I don’t feel bad lying about some stupid requirement
> If you want the job, can’t you just lie? Or are SAT scores something that cannot be faked?
I would assume that if you progress to the point of an offer, they would ask you to have the official scores sent by the College Board. Apparently they hang onto scores back to 2005 and can send them for a fee.
I remember the Cisco (!!) table was the most popular table at the campus job fair, and they turned away anyone without a 3.8+ GPA. I had a 2.42 (2.40 was minimum degree requirement).
Dodged a bullet there. I've worked happily at a FAANG for many years now and somehow I've avoided living in a cardboard box by a dumpster.
Many years ago I was walking around a college job fair when a recruiter was yelling at a college student. His crime: "wasting everybody's time" by presenting a Resume with a 3.4 GPA when the company clearly listed a 3.5 GPA as the minimum they would accept.
If this is how they treat people that don't yet work for them, it doesn't bode well for how they will treat people that work for them.
> One of the least effective predictors was unstructured interviews or 'chats’
Yeah those are the worst, one time I had an “interview” with a company that I really liked, the founder is also an awesome guy and we chatted few times and all is well. Then I got invited to their facility, great place and team, some of them were structured on how they evaluate, but most of them were an absolute mess, and some of them were hostile as if I would get hired it will get them fired the day after (the passive aggressive of trying to belittle your projects or work and not trying to understand your approach it but to attack it instead) and when I would ask them in a good faith about something they did, you would get a fake halo effect with “oh I can’t tell it’s secret! NDA bla bla” as if they did a patented work.. it was horrible method to hire people despite the great founder I knew.
In my opinion, the best way is what I usually do, after initial screening, I give them an assignment that they can do in few days and then return the work, the quality of the output will determine that, and it’s exactly how you will do in real work anyway, and you get to measure their critical thinking and problem solving rather than how would they sell or articulate something on the spot (maybe they are overwhelmed and their head went blank), as I am looking for an engineer not a sales dude, and they would tale some time to build and solve it.
I've been talking to a few companies lately, and one just keeps stringing it along, having me talk to manager after manager. It's been 6 weeks, and still no end in sight
If it's been 10+ years and an employer wants your SAT scores, 1600 is as good an answer as any. Anyone asking for that data point doesn't actually care about the accuracy, they just want to see if you'll compliantly jump through a pointless hoop.
(Save the "but that's fraud!" replies. It's not material to the job, so it isn't).
That cost-benefit analysis makes no sense if you had a pretty good scores. What's the point in fudging a few extra points if it means that diligence reveals you to be a liar?
It would also be somewhat suspicious if you went to a so-so college but allegedly had a perfect SAT. It would only make sense to lie if your score was well under 1600, you went to a college that makes sense for someone with a perfect SAT, and you didn't think it was likely they would follow up with a request for the official score report.
In service to the point, if you're _really_ smart—like definitely going to grad school smart—you apply to universities with the best faculty for your intended area of study. Often that is a sole professor at a university you've never heard of, but the researchers in that area of study regard as a top school.
SAT isn’t an IQ test, and probably all sorts of people took it before they had cultural awareness or a diagnosis that would have lead to different testing conditions had it been taken after diagnosis, let alone the fact that test scores are not comparable.
GPAs similarly not comparable over large time ranges, schools, or degrees without normalization you can’t get.
SAT isn't a perfect IQ test but it's not bad for a first-pass filter. Nearly all who scored 1600 will be bright and nearly all who score 800 won't be.
It's a bit like BMI. Yes, if you're Peter Dinklage or Arnold Schwarzenegger it will be pretty meaningless. But most people aren't and BMI works pretty well for them.
And the meaning of the score changes over the years based on the test itself changing. Same goes for the company's GPA requirements where there have clearly been shifts across schools on the amount of grade inflation allowed or even encouraged.
As an aside, I'm not sure if I or the College Board can prove my score at this point.
Not only that, but the SAT is not an IQ test and you can definitely study for it. Students with wealthy or motivated parents can get study books or tutors which makes a huge difference in score.
The Princeton Review promises a 200 point score improvement with some of their packages. And they can fairly-reliably achieve it too.
And I guarantee you that claim is based on an intentionally flawed experiment where they take students who have never seen the test before vs after completing the program. The actual control should be against students who have taken a couple of cheaply available practice tests.
I never done an iq test, but just curious, can’t one simply “rehearse/study” for such tests? I dunno, let’s say you do a few past IQ tests (with answers available), I guess one could get a higher score just by doing that
I dislike this argument. I think in some dimensions these types of tests can work, but I’ve never been the type of person who’s been able to score well, and I don’t test particularly well in general, yet I did my PhD work at <IVY LEAGUE> and have had a great career despite this. I think that testing is good for people who can be adequately evaluated, but for people like me it just leads to a lifetime of feeling like something’s wrong with you.
Test-taking may only roughly correlate to intelligence, and is just one dimension of a human... but they likely care less about false negatives (like you) than they do about false positives in alternative assessments.
I can't speak for others here and we don't have SATs in this country, but these things can be very unfair. When I was seventeen, I had a lot of things to deal with which were not of my own making, such as caring for a terminally ill relative and wondering if I would even have a roof over my head a few months down the line. That kind of thing tends to take your mind off school work. Several years later and I was in a much better place.
I think it's fair enough to say teenagers in general have more instability in their life even without this.
We don't have SATs in this country (like I said). Yes, we can retake the equivalent here, but retaken exams are valued less by both employers and universities.
I spent much of that period visiting hospices and hospitals, and eventually ending up homeless for a while as a result of all this. I was more interested in being able to support myself financially for the next few years, and recovering psychologically.
My qualifications after this consisted of diplomas and degrees.
Last month 2,400 University of California faculty asked for admissions to resume using the SAT "to ensure foundational fluency." Of course many employers want to ensure that too, especially when college degrees don't anymore.
https://www.insidehighered.com/news/faculty/learning-assessm...The UC faculty opposed the SAT requirement being discarded in the first place. They were overruled by the UC Regents, and that may happen again. And even if the SAT is brought back, I'm sure it will be given much less weight and subjected to the "in a local context" process in the name of equity.
I took the SAT 50 some years ago. I kinda doubt I would do so well on it today without doing remedial prep work on the math.
The SAT is actually a lot easier today than it was when most of us took it.
It is an hour or more shorter in length, the long reading passages have been replaced with short paragraphs, calculators are allowed, and vocabulary has been removed.
I never took the SAT, but I did take the GRE. The year I took it, it was 3 sections: Quantitative, Verbal and Writing. The year I took it, the writing section replaced an older Logic section. Quantitative and verbal were scored out of 800 points (like the SAT), but the writing section was scored 0-6 in half-point increments. I think the writing section has also since been removed.
I need the entire paragraph above just to _explain_ my GRE (decent) score because the test has changed yet again in the interim 20 years or so and I suspect the SAT is similar.
> calculators are allowed
OMG. Calculators are useless on the SAT anyway.
> vocabulary has been removed.
I flipped through a book that coached on SAT vocabulary. I knew all the words. Oh well. I never learned vocabulary as an explicit task. I simply read a lot.
I remember one question on the SAT verbal because it irked me. It asked an analogy question which required knowledge of mixed alcoholic drinks. Since I was far from drinking age, I had no idea.
> OMG. Calculators are useless on the SAT anyway.
No, they removed all the non-calculator “thinking” and “logic” math questions. It’s calculator stuff now.
They really nerfed the crap out of the SAT. It’s so soft
The fact that it's not all multiple choice now makes it somewhat harder. Multiple choice questions make it much faster to solve some questions because you can simply plug in possible answers. They also make it easier to know that you're right, if you solve a problem and the answer you got is one of the choices (though they do sometimes include common mistaken answers to fool students).
No idea what the SAT is like now a days (based on the comments in this thread) but 'back in the day' if you knew enough about a problem to understand the math needed to solve it, which is needed to plug in the answer, then it'd generally be faster to just solve than working backwards since the number of steps would typically be less than repeatedly plugging in. And most/all questions also had a 'None of the above' option that was the answer a fair chunk of the time.
Another practical thing is that tests seem to trend substantially higher in difficulty when multiple choice. I was part of the first class at my university that took a calculus program which was multiple choice and we thought it was going to be a cake walk. But suddenly like every single test problem was using obscure trigonometric tricks on top of the basic calculus itself. And of course no partial credit for getting everything 95% right and missing one really disguised trig trick at the end. Grades for the class were significantly lower than prior years, because those tests were just nuts - and I'm a very much a math guy.
> though they do sometimes include common mistaken answers to fool students
Ya, I quickly noticed that, and so didn't at all take for granted that a matching result was correct.
I’m going to have to ask for a source for this. How long ago is your most recent experience with the SAT?
Here’s a source that contradicts you (first hit for Desmos SAT). One of the allowed calculators is the Desmos app built in to the testing program.
https://www.strategictestprep.com/post/is-desmos-dead-on-the...
too bad everything is dumbed down.
> Calculators are useless on the SAT anyway.
Graphing calculators can be used to quickly solve certain problems, like simultaneous equations or quadratics. They can also be used to plug in multiple-choice answers to see which one is correct, without knowing how to solve a problem the normal way (or not taking the time to, at any rate).
The new adaptive digital SAT complicates things a bit, in that some questions are not multiple choice.
When I took the SATs, it was still the slide rule daze. Graphing calculators didn't appear until many years later.
Fair enough. I'm just pointing out that calculators are not useless on the SAT. Knowing how to use the provided graphing calculator, or having your own and knowing how to use it, is very important if you want a top score.
My new HP Prime calculator (that I'm good at) and the HP you used when dinosauers walked the earth are not the same thing.
We're all really proud of you.
I couldn't find a version of this video on YouTube without commentary. The reading comprehension level in the US is completely unacceptable.
https://www.youtube.com/watch?v=4FdGIRBUds4
Apologies, but which part of my comment was this a reply to?
It was in the context of asking people for their SAT scores long after they had taken the tests. Perhaps I replied to the wrong person. Apologies.
"Equity" is still something of a euphemism - the specific reason that the UC Regents got rid of the SAT requirement is because the non-profit law firm Public Counsel, along with a consortium of other non-profits and activist organizations, sued the UC system ostensibly on behalf of several students; and the UC Regents ended up agreeing to a settlement where they would not use SAT or ACT scores for admissions until after 2025. Public Counsel brags about the details of the lawsuit on their website (https://publiccounsel.org/our-cases/smith-v-regents-of-unive...):
> For decades, the University of California’s use of discriminatory SAT and ACT scores deprived hundreds of thousands of well-qualified students of color, students from low-income families, and students with disabilities of the opportunity to pursue higher education in the nation’s preeminent public university system. Rather than provide meaningful information about a student’s ability to succeed in college, SAT and ACT scores act as stand-ins for students’ wealth and race, and thus advantage more privileged applicants. Even University leaders admit that the tests are “racist” and “correlated to wealth and privilege.”
> Public Counsel and co-counsel brought this lawsuit on behalf of students and community organizations—Chinese for Affirmative Action, College Access Plan, College Seekers, Community Coalition, Dolores Huerta Foundation, and Little Manila Rising—challenging the University of California’s use of the SAT and ACT as discriminatory on the bases of race, wealth, and disability. In August 2020, Plaintiffs obtained a preliminary injunction requiring the University to immediately stop using the tests for undergraduate admissions and scholarship determinations. Plaintiffs then defeated the University’s attempts to prevent the injunction from taking effect. In the admissions cycle following the injunction, UC saw record gains in numbers of Black and Latinx students applying to and gaining admission to its campus.
Public Counsel is pretty clear that the legal theory they were originally operating under is that requiring SAT and ACT scores for university admission is racially discriminatory, and the specific races it's discriminatory against are the groups they characterize as "students of color", which we can take to mean primarily black and Latino students because they specifically mention the detail "In the admissions cycle following the injunction, UC saw record gains in numbers of Black and Latinx students applying to and gaining admission to its campus.".
Interestingly, the lawsuit was originally brought forward in late 2019, before the start of the COVID pandemic. The temporary injunction against using the SAT/ACT was imposed in August 2020, well into the pandemic, on the grounds that the pandemic conditions made it more difficult for applicants with disabilities to take standardized tests, in a way that was plausibly legally discriminatory - but of course this couldn't have been the primary legal justification that Public Counsel used when the brought the lawsuit in 2019, unless they were prescient enough to have predicted the course of the pandemic at that time (in my memory, the number of people in the Anglosphere who were paying attention to COVID-related news in China before the turn of 2020 and thought that it might develop into a concerning pandemic was incredibly small).
The settlement that the UC Regents reached in May 2021 lasted until Spring 2025, so it's only now that it's legally possible for the UC Regents to reconsider the ban on using the SAT/ACT for admissions. Presumably, Public Counsel and the other activist groups - Chinese for Affirmative Action, College Access Plan, College Seekers, Community Coalition, Dolores Huerta Foundation, and Little Manila Rising - haven't changed their opinion that the use of the SAT/ACT is racially discriminatory towards blacks and latinos. But they don't seem to have raised another lawsuit about this, perhaps because the political environment in the US has changed since 2020 in ways that make them less optimistic about their chances of success.
> lasted until Spring 2025, so it's only now that it's legally possible for the UC Regents to reconsider the ban on using the SAT/ACT for admissions
They could have reinstated the requirement last year, and they could have undertaken their recently-determined plan to engage in study regarding reinstatement anytime before that. They just couldn't remove the requirement until 2025.
Appreciate your detailed description of the lawsuit and settlement. This is what happens when two parties are settling a lawsuit but do not actually have adverse interests. They were aligned on wanting to get rid of it and signed an agreement to do so. That said, the Regents decided to get rid of it permanently, with no plan to bring it back or create any replacement test (as they had previously said they would do).
Regents are selected by Sacramento. At bottom, it's a political failure.
I always enjoy the advocates who claim that students have mastered their subjects, but "don't test well".
Would you want a pilot on your flight who flunked flying school exams, but somehow "really knew how to fly!"?
It's not a claim, there is psychological data supporting the negative impact stress can have on recall and that for some students a test is not the best way to identify their retained knowledge.
Your single example identifies a situation where we would want someone resilient to stress to pass the testing process.
However if you are evaluating someone's ability to identify characters in Shakespeare who are most closely representative of the Bard himself, a proctored exam may not be the only environment where that could be demonstrated.
Anyone in software knows as well that for a test to be effective it has to be written properly. In the history of academic and standardized testing however there has been little rigor in the construction of tests, and those who pass are ones who give the "right" answers regardless of whether those answers are true proofs of knowledge.
I've been around tests all my life. I don't know anyone who bombed the test but mastered the material. And the ones who aced the tests - turns out that they actually had mastered the material.
As for people who cannot perform under the stress of a test, how are they going to perform otherwise? Anyhow, the solution to test anxiety is to keep taking tests - the anxiety will recede.
For example, the first time I tried public speaking I was paralyzed. But I kept trying it, and the anxiety went away, and now I do a fair amount of public speaking and enjoy it very much.
That still doesn’t seem like a reasonable reason to not do tests. In the software analogy you have a unit test that passes but the actual software fails when used in the real world.
What is the point of “knowledge” that can’t be demonstrated. How will that person demonstrate their knowledge of Shakespeare?
My understanding is you're equating `failing a test` to `lacking the relevant skills and knowledge to do a certain task competently`.
The reality is sometimes tests in academia are just not very well made and don't really test what they are supposed to be testing, and that's usually due to multiple reasons like misaligned incentives, staffing shortages and maybe lack of resources / funding.
I don't think the comparison to flight school is relevant enough in this context because it's a too different of a world to traditional academia.
I don't buy the notion that tests do not test relevant skills.
In my long career I've noticed a strong correlation between SAT scores and academic performance as well as job performance.
> I don't think the comparison to flight school is relevant enough in this context because it's a too different of a world to traditional academia.
My dad kept his flight school tests for flying all sorts of airplanes. They bear a lot of similarities with the SATs. There's a lot of math in there for things like fuel consumption, wind, maximum landing weight, glide distance, and so on.
For example, one day he was cruising along in his F-86 when the engine failed. he radioed the tower, and they told him to bail out. But he calculated his speed, altitude, distance, wind, sink rate, air templeratur, etc., and figured he could make the field after configuring the airplane for maximum glide. He made a perfect landing, but still got reprimanded for risking his life bringing the airplane back. But he had worked the math and disagreed that it was more risky to bring it in than bail out.
> I don't buy the notion that tests do not test relevant skills.
> In my long career I've noticed a strong correlation between SAT scores and academic performance as well as job performance.
A test doesn't need to test the relevant skills for that, it just needs to test _something_ that correlates with academic performance and job success.
> I don't buy the notion that tests do not test relevant skills. In my long career I've noticed a strong correlation between SAT scores and academic performance as well as job performance.
SAT tests intelligence (aptitude), not skills. Which is why it correlates with job performance, where intelligence can (over some time) matter as much or more than a starting point of relevant skills.
I just checked, and the SAT math section covers algebra, trigonometry and statistics.
Look at this list:
A genius student who had never been taught those subjects wouldn't even know what the symbols meant. A mediocre student who had studied SAT-style questions for weeks leading up to the test would likely outperform a high IQ student who last solved those types of problems over a year prior.Standardized tests can be a great resource for assessing students, but they're not just testing for intelligence. Test-prep courses average increasing SAT scores by about 200 points. That's not because they're increasing the intelligence of the people taking them.
Somebody who goes to take a test on something they know that they know nothing about could be called many things, but genius is not one, even moreso when they're paying for the privilege of taking that test. What is on the SAT is no secret, so people are free to prepare as little or as much as they might like. If somebody can't be assed to prepare for such a critical test, then they're probably going to be the sort of person who can't be assed to do much of anything in life. And the internet has also largely relegated the inequities in access to training quite obsolete. You can get free high quality training materials on everything for free.
Or they're a student who has to work a part-time job after school. Or they have a long commute. Or they have hours of practice for a competitive sport or extracurricular. Or they have to take care of their siblings, grandparent or sick parent. Or they were told by their parents and teachers that because they're smart and do well in school that they don't need to worry about studying. Or they know their parents can't afford an elite school anyway, so they know they just have to score well enough to get into a school with in-state tuition and noncompetitive admissions.
Two kids who both went to the same school, were told the same things about the importance of the SAT and dedicated equal time to study and preparation can probably say that the difference in their scores is indicative of differences in intelligence.
But any two random students? There are so many factors that could cause students to underperform their hypothetical max score if they had perfect conditions in the months leading up to the test and day of.
We're talking about the SAT, not the gaokao [1]. It's a relatively short test that you can take it whenever you want, you can even retake it if you want, however many times you want. There's minimal memorization required and it's mostly testing basic aptitude. But the overwhelming majority of people are of average aptitude, so the overwhelming majority of people will do "poorly" on the SAT when comparing themselves to outliers.
I don't understand the effort to try to eliminate the reality that innate ability exists. I think we should accept this and work to cultivate it. Just because somebody of average aptitude doesn't mean he might not excel in other ways. The world needs all sorts of people, and I think the ideal system would work to figure out and cultivate these talents in everybody.
[1] - https://en.wikipedia.org/wiki/Gaokao
I know a person who has a pretty modest ability, shall we say. But he still managed to become a multimillionaire as a young man.
There's plenty of time during school hours to study up on the math. My high school was deficient in math, everyone knew that, and I took math classes at the local community college to make up for it.
I never studied for tests in school, not for the SATs, either. I tried that in college, that did not work. The stress of the tests was pretty bad. I realized I had to study hard, and I did, and it was effective at reducing test anxiety.
You've never met a smart high school student that didn't study? You've also never met someone who became more disciplined later in life?
There's a difference between not studying and just hand-waving away the SAT, which plays a huge role in things like college admittance. And the SAT isn't particularly hard, even back in the days when it was apparently significantly more challenging, for a clever kid. My prep was picking up some old book on it at Half Price Books and leaving it in the bathroom. Worked excellently!
> Test-prep courses average increasing SAT scores by about 200 points.
Perhaps because they finally bothered to learn the material. I am skeptical that "strategy" makes much of any difference.
> A genius student who had never been taught those subjects wouldn't even know what the symbols meant.
This is pedantic. Take it to the logical next step. Words and letters are symbols too. How would someone who can’t read do on the SAT?
> A mediocre student who had studied SAT-style questions for weeks leading up to the test would likely outperform a high IQ student who last solved those types of problems over a year prior.
Would make an interesting experiment, if it hasn’t already been studied. I would put my money on the lapsed high IQ student though.
Hal Finney was famous at Caltech for skipping class, flipping through the textbook a couple hours before finals, and acing them.
But he was one in a million. I always felt like a moron around him. But Hal was also charismatic and everyone liked him, including myself.
I suspect Hal was a "skin job" alien.
"A mediocre student who had studied SAT-style questions for weeks leading up to the test would likely outperform a high IQ student who last solved those types of problems over a year prior."
That's why hard questions exist.
Nothing's perfect, but the SAT tests do an adequate job.
I guess the question is: would you rather hire someone with poor SATs and god-tier Leetcoding skills or vice-versa?
I’d for sure take the high SAT scores. I think the SAT has more correlation with/is more predictive of work success than leetcode.
Do you also think LLM leaderboards accurately reflect the capabilities of the models being tested? If you do, then I can easily point you to numerous academic papers pointing out the various flaws in many leaderboards (from poorly designed benchmarks like bABI and the original SQuAD, to data contamination, and more).
In that same way, any test, including the SAT and GRE have flaws. They can be gamed in ways similar to LLM leadeboards: test prep makes you better at them. That's one of the main reasons universities moved away from SAT; they were afraid that it disenfranchised lower socioeconomic status students (and it does to some degree). The issue is that the test is positively correlated with success in an undergraduate program, so they threw out the baby with the bathwster. The real issue is that the SAT is not able to distinguish the capabilities among students to the degree it purports to.
And if you want an anecdote to match all yours, the first time I took a GRE practice test, I got a 3 on the writing. Not because I'm poor at writing, but because I didn't really know what they were looking for. After reading a test prep book, I got a 4.5 on my next practice test and a 5 on my final practice test. When I finally took the actual GRE, I got 6 on the analytical writing. Trust me, nothing changed in my writing ability over that time. In fact, I didn't even practice the skill except through those three practice tests. Clearly the test was not capable of determining my real ability to make an argument; it merely tested my ability to adapt my writing to what was supposedly being tested.
Interestingly, the vast majority of universities that got rid of the GRE requirements for PhD programs are not going back on that. Turns out that the students with the highest GRE scores are the ones most likely to drop out of their STEM PhD. [1]
[1]: https://journals.plos.org/plosone/article?id=10.1371/journal...
I took the GREs, I don't recall a writing section.
Anyhow, the questions were all about freshman engineering knowledge.
There are three major parts of the modern GRE: Verbal, Quantitative, and Analytical Writing. You could easily look that up, or ask if you didn't know.
Responding off the cuff without any reflection on the comment you're responding to doesn't move the conversation forward in any meaningful way. It just comes across as disrespectful.
Do you think that LLM leaderboards don’t? Do you think a Llama 3 is going to beat an Opus 4.7 on any leaderboard?
The real issue is that standardized tests disenfranchise lower SES students less than any other metric.
Everyone who takes the SAT has to sit in the same room for the same amount of time answering the same questions. You can’t just pay someone else to take it for you (like essays) or select which difficulty level you take (like going to a prep school with grade inflation), or luck out in who your parents know (like recommendation letters).
Some may have better opportunities to learn the material, but, at the end of the day, you have to actually learn the material. There’s no getting around that.
As your own GRE anecdote shows: A little studying with some inexpensive books makes all the difference. Unless things have radically changed, a couple SAT or GRE test prep books are significantly less expensive than just one college textbook.
Bluntly, the reason SATs are better correlated to college performance than other measures are because of the reasons I mentioned. They strip away most of the privilege of coming from a high-SES family.
In my country, we have public exams to get into Uni, with the ones for high demand majors being very competitive, but performance in that exam is not a good predictor of academic performance.
The guy who got into my uni class as #2 in that exam dropped out after a few semesters because he couldn't beat calculus. The #4 took several extra semesters to graduate despite not working/not interning. Several others in the top third struggled through. We had _maybe_ 2 or 3 guys who straight-A'd the entire major.
I myself got in as #17 and still failed a few courses. Thankfully no one cared throughout my professional career.
> In my long career I've noticed a strong correlation between SAT scores and academic performance as well as job performance
On average. However, I've also had the experience that some of the most competent people I've known had rather difficult teens and twenties.
Hiring someone who flunked out of highschool, worked odd jobs for 10 years, then got a diploma and a degree is higher risk, higher reward. They are often times harder workers, unusual thinkers and more grateful for what they have.
My wife and I are perfect poster children for demonstrating the danger in making the assumption that tests are an accurate assessment of someone's mastery of material.
My wife has anxiety, and the time-limited, high-stakes, pass-or-fail nature of a test makes it much, much harder for her to perform well. Outside that context, she's blazingly intelligent, performs very well under real-world high-pressure situations, is extremely diligent at getting any other assigned work done well on time, and has repeatedly written the guidelines for processes and procedures in her jobs.
I'm the opposite in many ways. I generally do somewhat poorly in a classic classroom environment, primarily due to ADHD hampering my ability to get homework done regularly and on time. However, I test amazingly well. I consistently finish in 50-70% of the time of other students, with scores in the high 90% range. (In my jobs, I think I do pretty well.)
Naturally the plural of "anecdote" is not "data", but I've known enough other people who fit both of our molds that I think it's fair to say that basing even a plurality of your assessment of someone purely on tests, especially standardised tests, is going to mislead you at least as often as not, in both directions.
Correlation isn’t causation. Students with test aptitude and ambition will score hire regardless of the test content.
Whether or not tests are bad the fact is that nobody has even come close with something objective that can replace it at scale. Teachers have 30 kids in their class- they actually in reality have no idea what their students can or cannot do.
I agree with you. The real world is stressful or less than ideal for many reasons and we have to apply our knowledge within that context.
In addition to that, this claim is like the dragon in my garage thought experiment. It an unfalsifiable claim that they have “knowledge” but can’t demonstrate it.
> Would you want a pilot on your flight who flunked flying school exams, but somehow "really knew how to fly!
Sure, just not in the cockpit
Not really comparable... the overwhelming majority of flight tests involve flying an aircraft. There is no meaningful way someone can be excellent at flying an aircraft but can't pass a test which involves flying an aircraft.
The same can't be said for many other tests. If the test involves the practical application of the very skill being tested, then that test has direct relevance to he competency of said skill.
But many other tests are not like that. A teacher can be brilliant in the classroom yet stumble on a standardized certification exam full of pedagogical jargon. A chef can cook a variety of excellent dishes but fail a written culinary theory exam testing the French names of techniques they perform by instinct. And perhaps more relevant to this audience, a coding interview that relies on whiteboarding algorithms from memory can easily fail an excellent engineer who builds great software every day but doesn't recall the optimal solution to some puzzle on the spot.
The overwhelming majority of math tests involve doing math, so I'm not sure your critique is useful in this context.
Given that I'm not responding to any claim about the efficacy of math tests... it's actually your statement which is wholly irrelevant to this discussion.
"You can't compare academic tests and flight tests because flight tests involve flying"
"Academic tests involve academics"
"I'm not talking about academics!"
That about sum it up?
The submission isn't about academic tests involving academics... it's about the relevance of using such academic tests for the purpose of job screening.
A business that wastes money on ineffective screening methods for employees will lose to a business hiring better employees.
Which is why almost no employers ask for SAT scores as a way of screening candidates, and why this article was written to showcase the specific reasons doing so is ineffective.
Of course you'd have refrained from making such a vacuous reply if you bothered to read the article.
Is there a reason you left out the SAT and ACT?
Because both have been shown to have predictive power for success in college.
Because I'm not trying to make a universal claim about all tests.
For one semester.
Maybe two.
The problem that colleges and the SWE profession in general face are identifying “bullshitters”. We need a filter and a fact based exam seems like a good place to start.
They are never going to let you into a cockpit until you pass ground school, which involves a lot of math.
> A teacher can be brilliant in the classroom yet stumble on a standardized certification exam full of pedagogical jargon.
A teacher that cannot explain how calculus works cannot teach it to anybody.
> a coding interview that relies on whiteboarding algorithms from memory can easily fail an excellent engineer who builds great software every day but doesn't recall the optimal solution to some puzzle on the spot.
I've seen too many coders using bubble sort because they don't know enough to look for a better algorithm.
In any case, the purpose of leet coding tests is to quickly filter out the utter frauds. I have a programmer friend who wanted a job at a major software corp. He knew he'd have to pass the leetcode in an early stage of the interviewing. He figured it would take 6 weeks or so to study that material. I suggested that, since he was applying for a $250K job, that would be the most productive studying he'd ever done. He agreed, did the 6 weeks of studying, aced the leetcode test, and got the $250K.
So ya, there is a point to those tests, in filtering out the frauds and the ones who aren't willing to do what it takes to get those jobs.
So you yourself has just given an example how easy it is to temporarily get good at "test-filling", by simply preparing for that.
The question is whether a given test measures anything relevant - did your friend become a better programmer for doing 6 weeks of leetcodes? E.g. what kind of experience did he gain about large code bases and how to handle those? Continuing your analogy, would you fly with a pilot who drilled on taking off a bunch of time, but never practised flying in a storm?
I'm not saying leetcodes or exams are useless, but Goodhart's Law apply.
Regardless of the content, a test filters for people willing to put in the work to pass. All else being equal, given two candidates, where one has passed a test and another one who didn't bother, the first one sends a stronger signal to the potential employer.
It's still worth discussing if the particular test is ideal, but the answer IMO is not to say let's remove tests in general.
Right on. Instead of complaining about the test, he did what was necessary to ace it.
He's the kind of person I'd want to hire.
Is that what the sat is supposed to measure, in your opinion? Do you worry that would heavily skew toward people who didn't have to work over people who didn't have as much available time to train to a particular test? Does that even matter?
This has to be a joke...
Ground school most certainly does not involve a lot of math, it's not like there's any calculus or algebra involved... it's basic arithmetic. Furthermore it's categorically false that you need to pass ground school before you're allowed to fly.
Are you just making things up?
>A teacher that cannot explain how calculus works cannot teach it to anybody.
This is a strawman argument, I never made anything that could even remotely be interpreted as this.
>I've seen too many coders using bubble sort because they don't know enough to look for a better algorithm.
This is committing a very basic logical fallacy. The fact that someone who is incompetent likely can't pass a test is not the same claim as someone who can't pass a test is likely incompetent.
Hopefully you are able to identify this logical mistake that you're committing and revise your position accordingly.
> categorically false
Google sez: "The U.S. Air Force strictly requires you to complete and pass formal academic ground training before you ever touch the controls of an aircraft"
They're not going to risk an aircraft on an incompetent student.
> A teacher can be brilliant in the classroom yet stumble on a standardized certification exam full of pedagogical jargon
I stand by my statement.
> logical fallacy
A implies B meaning B implies A is indeed a logical fallacy. But that does not rule out B implies A. A and B can be strongly related to each other.
So you are making things up... thanks for confirming that. While I appreciate that you reviewed what Google "sez"... you have misunderstood the relevant context which is that the U.S. Air Force also requires that you complete Initial Flight Training (IFT) before you start the Air Force's own formal training program (UPT). In IFT you will not be required to pass ground school before you get to fly.
Furthermore, even if the Air Force did not require IFT before UPT (the Air Force's own training program), you've completely changed the nature of your argument. I have no dispute about whether the Air Force may or may not have stricter requirements for their pilots, but that wasn't your argument.
>I stand by my statement.
You've proudly planted your flag on a point nobody was contesting, which is a strange hill to celebrate on but you do you.
>But that does not rule out B implies A. A and B can be strongly related to each other.
Discussing a topic with someone who not only uses logically fallacies as justification for their argument but brazenly doubles down on said fallacy is a good sign that this is probably not a good discussion to continue spending time on. Like am I supposed to simply accept your logical fallacy and take on the burden of disproving every claim you can dream up simply because you've asserted it isn't logically impossible? The person making the claim carries the burden of supporting it, and "they're strongly related" is something you have to actually show, not something I'm obligated to refute on your behalf.
Ground school comes first at IFT https://www.baseops.net/militarypilot/usaf_ift.html
Note that a tree implies it is made of wood. If you find a stick of wood, odds are it came from a tree.
>Ground school comes first at IFT
There is no singular "IFT"... you happened to find one IFT among hundreds across the U.S. that has such a syllabus, great... but it does not come first as requirement mandated either by law/regulation or convention. Here is the syllabus for a different FAA Part 61 and 141 approved IFT program that uses an integrated approach with the following quote:
"Each Module contains both a flight and ground lesson. This presents an integrated flight training process and will promote easier learning and a more efficient flight training program"
https://static1.squarespace.com/static/5ad1e29b372b96bedc6b1...
>Note that a tree implies it is made of wood. If you find a stick of wood, odds are it came from a tree.
This is false, not all trees are made of wood (palm trees) and there are natural sources of wood that don't come from trees.
https://en.wikipedia.org/wiki/Woody_plant
But of course... instead of just admitting you were wrong to make that logical fallacy... free to continue doubling down and making things up.
The USAF is not subject to (most) FAA regulations. As far as I know this also extends to training: the USAF does not follow either of the FAA’s Part 61 or 141 flight training syllabi. So bringing up parts 61 and 141 doesn’t really refute the parent’s point.
Regardless, as a civilian, you do have to pass a written multiple-choice test on flying theory before you can solo. Any time you spend in the cockpit before passing that test will be under the supervision of a Certified Flight Instructor (CFI), in a plane with dual controls, and you are not acting a pilot-in-command.
So, although the parent may be slightly over-focused on the USAF way, I think it is fair to say that for any type of pilot training in the USA, you do have to complete academic ground training (including passing a formal written test) before you are truly “given the controls”.
When I was a boy, as a present my dad bought a ride in a Cessna for me. I sat in the copilot's seat, and the pilot let me try out the controls. I also was offered a ride by my lead engineer at Boeing and he let me work the controls a bit.
I do not consider that flight training.
> But of course...
You goofed. A palm tree is not pedantically considered a tree. You made that up.
https://texastreesurgeons.com/blog/2025/01/03/palm-trees-are...
> odds are it came from a tree >> there are natural sources of wood that don't come from trees
Confirms I was correct.
> Each Module contains both a flight and ground lesson.
And the ground lesson comes first. I read your link.
In addition to affirmation of the consequent he's also employing attacking a strawman, petitio principii, faulty analogy, and goalpost shifting, at least. His followup example "Note that a tree implies it is made of wood. If you find a stick of wood, odds are it came from a tree." is hilarious. No doubt there are numerous other examples completely unrelated to coders and whiteboard tests where A implies B and B is highly correlated to A, but their existence tells us nothing about coders and whiteboard tests and doesn't justify a blatant fallacy of affirmation of the consequent.
Here's something to consider: just because someone is good at writing compilers or designing a language, that doesn't entail anything about the quality of their arguments.
I’m very curious where you get the idea that flying does not involve algebra or trigonometry or calculus.
How would you calculate a crosswind component from the runway heading and reported wind speed and direction without trig? How would you think pilots measured their distance to non-directional beacons before GPS and DMEs existed?
How would you solve for fuel remaining without algebra? How would you estimate the best speed to fly with a given headwind to maximize fuel onboard at the intended destination without calculus?
A very basic principle of glider flying involves finding the tangent to a curve. Is calculus not applicable there?
Fuel consumption is often estimated by numerical integration of fuel flow rate. That doesn’t require an analytic solution of the integral, but I think most pilots have at least a passing familiarity with the concept.
> Furthermore it's categorically false that you need to pass ground school before you're allowed to fly.
I interpreted the parent’s statement to mean “before you fly solo.”
> it's not like there's any calculus or algebra involved
There certainly is when you're navigating.
Some of the more advanced math is boiled down to specialized slide rules, though these days they'd use a computers.
For example, the fuel consumption rate vs range is not a linear relationship, because burning fuel lightens the airplane and so it can go faster/further.
Ground school absolutely has algebra involved (fuel burn, trip/segment times, wind correction, weight and balance)
No, but it'd probably be okay if the pilot flunked their SATs.
I did very well with my SAT
but to get to the city we had to take it was a 2 hour long drive through twisted roads that made me carsick. I lived in a small town far away from the city. By the time I got there my breakfast I had quickly eaten gave me a stomach ache, I had woken up far earlier than usual and not gotten my 7 hours too. I certainly would have done a lot better if I had lived in the big city.
Another factor: if you wanted to pay the fee you could just take the test over and over again until you got a great score. So kids with poor parents obviously had a huge disadvantage. Also kids who had the time and money could study for it with prep books - I did, while some of my friends were flipping burgers while still in highschool. Its not surprising I got a higher score than them, but it said nothing about my intellegence or understanding compaired to theirs.
There are fee waivers for poor kids, and free test prep thanks to Khan Academy. These may not have existed decades ago, but they exist now.
Fee waivers are great. A lot of poor parents are too busy / don't have their shit together enough to learn about them & use them. A lot of poor parents won't even have their kids take the tests at all even if they are free, especially if it means taking an entire day to drive them to the test facility. They might not even know what an SAT test is.
Hm that's true, but the alternative would also be flawed, and possibly worse. If you could only take the SAT once and you were stuck with that score, then people would complain that rich kids could take all sorts of prep courses to get ready and to figure out what they needed to work on before taking the test, whereas poor kids would be figuring this all out on test day. There is no ideal way to do this, and the current system may be better than anything else.
Also, in a decent number of states, it's mandatory for all HS students to take (and I believe free to students as a result).
I would be much more ok with someone who failed the test but knew how to pilot a plane vs someone who aced the test but can’t figure out how to get the engine started.
One who cannot start the engine cannot kill me.
One who cannot calculate how much fuel he needs to cross the lake will kill me.
Remember JFKjr? He killed himself, his fiance and her friend because he did not pay attention to the instruments.
An acquaintance of mine died trying to fly through a thunderstorm. Another one didn't pay attention to the weather and nearly died from wing icing.
Flying is no joke.
It’s frustrating when easily predicted outcomes are ignored for the sake of feel good policy.
The ironic thing is that feel good policy can be more racist and prejudiced than any test.
They experimented replacing tests in my country with giving teachers the final say- it was a fucking disaster. Human beings are quite terrible at objectivity.
Why is a lossy testing filter better than just failing out those who can't make it? Maybe allow for larger freshmen classes and smaller latter classes or adopt community colleges and have all students start there and advance into the UC system sophomore year on. Instead they bring back what is basically an IQ test for admission.
From the students' perspective, it is better to not be allowed in than fail out midway through. One test is cheaper than years in college.
I personally strongly disagree. I think it's much better to be given the opportunity to do the actual work, rather than to be required to do the pre-assessment song and dance. And if there are actual prerequisites that a person hasn't previously passed, they should be allowed to be tested on these specifically.
It depends on a lot of things. If they've also applied to another college that's better suited to their ability, admitting then to a school where they'll likely fail is not really doing them a favor.
Agreed. I've seen some interesting data that kids who could've been successful premeds or engineers at their state flagship but instead eke their way into Harvard or Princeton (sports scholarship, legacy etc) will instead graduate with a flaky studies major because they couldn't cut it in their intended major and the switching costs of transferring were too high.
There are downsides if you end up a small fish in a big pond.
Is the actual premed or engineering coursework at Harvard or Princeton really that much more rigorous than that at a flagship state school? I'm doubtful.
I'm no expert on that particular situation, but I compared my syllabi and projects from a state flagship (not Georgia Tech, Berkeley, or UIUC) with my brother's from Carnegie Mellon, and the expectations of first/second year CS majors were extremely different. Sometimes we used the same textbook but CMU covered more chapters and their projects were more involved. Some courses that typically waited until senior year at the state flagship were common to take spring sophomore year at CMU. There were a lot of courses that were numbered as undergraduate at CMU but covered content that was only covered in graduate courses at the state flagship.
Quick answer: YES
Longer answer - in the other reply to your doubtfulness.
This is true across the entire US system, some state flagship universities curricula are so deficient that graduate level at better schools wont even consider the bachelor level diplomas from those schools as eligible unless the applicant is top n% of the graduating class, where n is a low single digit.
The admissions committee may never publish or say it directly, but for MANY state flagship universities the B.S. level maths and science courses are simply insufficient fo higher level studies at leading schools.
Thus, companies with hiring and leadership that is aware of these conditions will also simply pass over applicants with degrees from flagship state universities, much the same as they do with online diploma mill "Graduates."
My take on this situation is that as primary education outcomes worsened in the US, state universities modulated the coursework to match the readiness of incoming students in order to keep enrollment 'available' to everyone and extract revenue from the student loans system.
The "Princeton and Harvard(s)" were differently motivated, in that they never had a goal of admitting the majority of High School graduates, and thus were not required to lower levels of educational rigour to meet eroding conditions in primary education.
It truly is a sad "state" of affairs.
> The "Princeton and Harvard(s)" were differently motivated, in that they never had a goal of admitting the majority of High School graduates, and thus were not required to lower levels of educational rigour to meet eroding conditions in primary education.
It's easy to find recent reporting on claimed grade inflation, reductions in rigor, and students who seem unprepared or unwilling to do the work at Harvard and Princeton too.
Speaking as a professor: the filter is really helpful. Having students struggle for two years in a program they're going to fail out of is terrible. I've seen it happen - and I saw more of it happen for the years CMU also stopped requiring the SAT.
The SAT is a very imperfect measure but it turns out a lot of the others are even worse.
Honest question: why is it terrible? I would assume that if someone is capable enough to struggle for 2 years, they stand a real chance to struggle for 4 years and graduate. And seeing how the job market rewards a mediocre diploma from a good college significantly more than a good diploma from a mediocre college, it seems like a very rational decision from the student's side. And if they do fail out, they should still be able to enroll in another program, and presumably get a decent grade there, no?
>Having students struggle for two years in a program they're going to fail out of is terrible.
Putting a body every a seat for those profitable 500-person lectures taught by an adjunct and graded by TAs who are all paid peanuts and then having them wash out before those expensive 50-person high level classes and esoteric electives taught by tenured profs and accompanied by lots of expensive lab time is great for the university's bottom line though.
Good point, and that was for me the promise of the original MOOCs 15 years ago - for students to be able to take on these MIT/Stanford courses, with the exact same workload and see if they could handle them, without any additional cost to the school and without the student having to upend their life yet. And if they proved to themselves that they could and are willing to, then enroll, and use that as proof.
There are still some online courses that try to do this sort of thing, with a particular example being the University of Helsinky's Full Stack Open [0] which offers post-hoc university credits for those who complete the course, but it seems that the dream didn't quite materialize.
[0] https://fullstackopen.com/en/about
If you completely ignore costs, and the negative affects of having 50+% of a class be unprepared and taking time from the other students, sure, that would work great. Seems like a bad alternative to standardized tests considering students will then have to pass tests once in college...
High graduation rates are an important metric to administrators. If a professor gave a failing grade to 1/3 of the class they would be in hot water.
My wife has a civil engineering degree. There were a number of courses where partial credit was not permitted and the final exam was 2 questions. It was common for students to take those courses 3 or 4 times before passing. Giving a failing grade to only 1/3 of the class might get a professor investigated for making the class too easy.
What decade and country?
In the US, this would be a huge outlier.
Unless, somehow this is part of an advanced certification process? I really can’t imagine professionals not knowing how to study for an exam like this. Medical school shows what motivated people can do.
In America, dude. Pick your favorite engineering school and read syllabi for core classes such as statics. Read department policies on number of tries you get on core classes before you’re kicked out of the civil engineering program. They take certain things very seriously.
The incentives change when the school becomes a mechanism to sell a chance to increase probability of immigration rights at tens of thousands of dollars per year, especially when the pool of domestic kids is shrinking.
Failing 1/3 of a class if that cohort is genuinely deemed not qualified enough to pass shouldn't be a problem by itself.
But then it raises questions like "are they really unqualified or is the testing methodology inadequate?" and "why was the system unable to provide the necessary growth to such a high slice of the class?". And then the easy way out is to just cherry-pick which students enter the system at all.
> High graduation rates are an important metric to administrators. If a professor gave a failing grade to 1/3 of the class they would be in hot water.
I remember practically every single instructor/professor on the first day of class during my freshman year of my undergraduate study said something along the lines of "I have no curves. Your grades depend on you and nobody else. If the whole class does well, everyone can get an A. If nobody does well, everybody can fail."
So I guess this was more motivational to get us to study rather than stating facts?
It really depends on what kind of class it is, but at most schools:
* If 1/3 of calculus physics for engineers fail, they take it next semester.
* If 1/3 of gen-ed physics for poets fail, the professor better have a good explanation for the provost.
* If 1/3 of physics for pre-meds fail, the professor better have a pretty good home security system and might want to think about having the family stay in a hotel for a few weeks.
I remember a first lecture when I started my CS studies, where the professor said something like "look at the people to your left and to your right, it's likely that at least one of you will drop out by the end of this year; it's ok, this is not for everyone; if you truly believe this is for you, put in the effort and you'll make it"
1/3 isn't that bad in the late 80s/early 90s at the UT Austin CS department. Only ~30% graduated at the time. The orientation was literally "look to your left and looked to your right only one of you will graduate." They weren't joking!
Caltech did not grade on a curve. I recall one class were half the class failed.
I’ve had classes like that or ones that start with 75 students and end with 5 and I went to a very easy state school (late 90s)
That depends. Some schools actually cap the number of students permitted to continue. They fail a certain fixed number or percentage of students below a threshold, even if the raw score is good.
Because it’s a complete waste of time and money for both those students and the instructors.
Apply the same question to jobs and it's easy to see: why is a lossy [interview] filter better than just [firing] those who can't make it?
This has enormous costs to the institution, the teachers/mentors, and of course to the person failing out.
And that's not even factoring in the social and psychological costs.
Disagree. Hiring and firing is better than a bad interview process. The reason we don't have that is due to regulations and litigiousness (and the laws that facilitate it).
IMO failing to get the opportunity is worse than getting the opportunity and failing at it.
Griggs v. Duke, 401 U.S. 424 (1971).
SAT score is known to be predictive of college grades. Is it also predictive for whether a mid-to-late career candidate will pass a phone screen? It is used for early career candidate filtering in finance, but I have not heard of anybody caring beyond that because of the availability of signal on the actual tasks they will be performing.
It is correlated tightly with IQ, so yes it will likely be a strong predictive signal for passing a phone screen.
Maybe Maths, but english is probably not correlated tightly with IQ as it is more affected by language background and education.
>english is probably not correlated tightly with IQ as it is more affected by language background and education.
No, see: https://en.wikipedia.org/wiki/G_factor_(psychometrics) English and math has a 0.64 correlation.
I feel like it's a common misconception that IQ only tests some narrow intellectual abilities like math and logic, likely because online tests and fake IQ tests tend to focus on those kinds of puzzles. Actual IQ tests actually do place heavy emphasis on language skills like vocabulary, verbal abstraction and comprehension.
> Is it also predictive for whether a mid-to-late career candidate will pass a phone screen?
I’m sure having access to your own SAT scores (or even remembering what they were) is highly predictive of not being someone who it would be illegal to fire because they are too old… which is probably the point… and why I’d expect most HR departments to shy away from this requirement.
> Is it also predictive for whether a mid-to-late career candidate will pass a phone screen?
The answer is probably no. I got many friend they got good marks in SAT, but they were average.
Would you include the ability to differentiate between anecdotes and peer reviewed studies in your assessment of intelligence?
how many elected leaders are in STEM? would high SAT scores and grades exclude many US presidents and congressmen? (yes) winning elections seems kind of important to me. so if you were just like, selecting for leadership - and many of our leaders are brilliant people, just not in the sense of being good at taking tests - would that be good or bad? or... what is your real opinion? what are you actually mad about?
obviously the UC system should give spots to the kids who will use those spots the best. but it is very hard to define what "using spots the best" means.
> not in the sense of being good at taking tests
The trick to doing well on the SATs is to pay attention in class.
The trick is to prepare specifically for SATs.
One would say the tests (and job interviews) should have been designed with the original intent of testing candidates AS IS, i.e. preparing specifically for such tests should have been considered as cheating... But at some point it turned into prep gymnastics, and measuring how desperate the candidates are.
Once it is a forced ranking over all the candidates - it becomes a test of preparation and execution not of underlying skill.
That's not how it plays out in practice. There is overwhelming evidence that students who otherwise excel academically score fairly mediocre SAT scores on their first attempt and then jump substantially after weeks of targeted practice and/or tutoring, even though they didn't learn anything new in the classroom.
If attention in class were all it took then that improvement couldn't happen. What changed was familiarity with the test, not classroom focus.
I guess times have changed. When I took the SAT, I did zero prep work. Nobody else I knew did prep work, either.
Paul Graham recently posted SAT advice along the form of "when you finish the test and have more time, go back over the test and check for mistakes."
I was kinda astonished at this advice, isn't it obvious? A strategy I also employed was to do the easy problems first, so I don't miss a question that would have been easy. Apparently this has to be explained to people?
I suppose prep work would be fine for the students who didn't pay attention in class.
Yes, if you took your SAT among a cohort of people where none of you practiced for the SAT, then what you're saying holds true.
That's not really the case anymore. Top tier students nowadays prepare for the SAT, they don't go into it blind and haven't done so for the better part of 20 years.
I didn't get a perfect score, but it was good enough to get what I wanted.
Well, then what if someone with a better score got the job you were going for, even though they are not smarter, just prepared more for the test?
I'd get the jobs I wanted by bringing samples of my work to the interview.
An interview ... with scores like that?!
I'm not aware that there is any method to dramatically increase SAT scores (and neither IQ test scores). could you point me to your sources?
Straight from the College Board themselves:
https://research.collegeboard.org/media/pdf/osp-technical-re...
That report also references another study showing that each hour of tutoring was associated with an increase of 2.34 points on the SAT score which unfortunately is behind a paywall:
https://www.researchgate.net/publication/282492223_Preparing...
We certainly don't want them to fail out, which is what is happening. Berkeley reported 10% failure rate in the intro CS course and 35% in the pre-intro course. https://www.dailycal.org/news/campus/academics/failing-grade...
In the US or China?
I have seen company descriptions in job ads that list college achievements of founders. They are invariably young Asian men. I understand that it's a cultural signifier and don't judge them. But, I also understand that I will never hear back from them because I don't share that background. So, I never apply to any job listing that references college experience of either side, other than wanting a degree in general.
Someone once told me that American work culture used to be based more on intern-ship / apprentice type hiring but now obsess with formal degrees. I wonder how much of this shift in culture is influenced by the Korean, Chinese and Indian immigrants, as a formal education is a prerequisite to compete in these countries' job market? For example, it is quite common in India for employers to ask for our 10th and 12th standard marks / grade (because these are national exams) along with college grades - to apparently gauge "Consistency". Fluctuating performance, a break or dropout years all negatively impact you and can be nerve wracking for many freshers, until they manage to get some work experience. It is somewhat disappointing to see this culture permeate to America too, even though I feel quite conflicted about it - after all, everyone does want to hire the best / most competent / reliable candidate; but the other approach - a vocational kind of training - also has its merits and seems to have served American companies well too. (Zoho in India is experimenting with this kind of hiring in India where they are hiring high-school students, mainly from rural areas, and offering them a work cum study program. They don't get any formal diploma or degree though - https://www.zohoschools.com/ ).
> I wonder how much of this shift in culture is influenced by the Korean, Chinese and Indian immigrants
I think there’s an influence, but it’s amplifying a pre-existing trend. Bureaucratic societies favor formal credentials. The U.S. has become much more bureaucratic since the mid-20th century, and credentialism has grown. Reliance on degrees and other formal credentials also enables the universities to achieve political goals through admissions and grading policies. Asian immigrants in the U.S. have readily adapted to that system.
how many uncredentialed people's families go to private school with what you pay them?
it's one thing to hire some people for some roles with this sort of, diamond in the rough mentality. obviously that can be a good idea. but in my experience, if you try to take leadership in that way, you are spending most of your time persuading other people that it's a good idea, which they will reject, and consequently, it's of little influence.
then you look at people who become bosses who lack credentials (or whatever), and you find out it's only because they drop out of their competitive colleges to be fabulously successful. the true weirdos out there - whatever held them back from "credentials" doesn't stop them from becoming fabulously wealthy, but rarely do they go and hire anyone else. like they do not create enterprises, teams or even families. do you get it?
What you’re describing is the culture of credentialism. You can’t change it by yourself and it’s hard to fight against. But that’s my point.
The problem with credentialism is that the credential becomes the end, not a means to an end. There is a huge problem in India that there are far more people with credentials (often of dubious worth) than jobs for those people. The culture is very focused on “the track,” where you get the credential then go to the job unlocked by the credential. But the problem is that there’s very few people actually starting the businesses and creating jobs that would hire degree holders.
I have suspected the influence is real. For a reference point, the majority of students at top tier US universities are Asian at this point, broadly. Not every top tier university but there's a trend to have about 30-40% Asian American students and then roughly 1/3 international which is heavily weighted toward China and India. This constitutes the largest group usually. So it's quite likely that universities adapt to this and hiring practices begin to reflect an intense interest in exam taking and credentials.
The thing about it is I view it similarly to how in the past "well-roundedness" and "leadership" was part of hiring and admissions. We laugh at that now but my understanding is the SAT score can be improved with long term studying. So intensive SAT studying seems like a new thing that isn't evenly practiced among people in the US. So at worst SAT score usage seems like a way for an elite group to preserve and replicate itself. I have no SAT score so I feel somewhat outside of this debate and have no experience with it.
Have university faculty become mostly Asian too?
I don't know. I won't even comment on this because I have no insight besides personal observation.
I don't think other cultures are driving much of the trend for educational check marks. I remember my dad and uncle talking about the awkwardness of being asked where they did their MBA's by colleagues/clients in the late 90s/early 00's and them trying to figure out how to navigate that as they didn't even have bachelors degrees. And I doubt whoever replaced them when they retired had less than an MBA.
Between increased regulation and greater competition for jobs, the degree requirements keep going up in a lot of/most industries. I also think there is a tendency for those that have reached a level of educational attainment to push back on others without equal numbers of checkmarks. Once a role is populated by MBA, PhD, MS or even BS, individuals don't like to see others doing the same work with less credentials. Maybe it's a 'I had to do this, so you do too' mentality or a sense that it devalues their own credentials.
I think it is a positive for an employer to ask for an SAT because it tells me right away I don't want to work for them. Once (a long time ago) I tried to upload my resume to apply for a job. The web page started asking me very basic questions, like a basic aptitude test. I was out. Tell me you do not know how to find and evaluate talent!
I think this is highly age-dependent. I took the SAT well over a decade ago and have significant work experience since then. It would be odd to require me to put down my SAT scores, which I don't even precisely remember.
But if I were < 5 years out of college, and especially if I had gone to school during COVID times (when SATs were not required by many colleges), I would completely understand why an employer might ask.
Basically, colleges used to act as a filter for SAT and other attributes. During the 2020-2025 period, they admitted students under fairly different standards, due in part to testing challenges and social movements.
It makes sense for an employer to want to do a little more diligence to ensure that students who were admitted during this period are similar to students admitted during the prior several decades.
It was required at my first job in 1989, for entry level actuaries.
In fairness, part of job performance was passing the actuarial exams, the first two of which were calculus and statistics. I imagine testing well on the SATs for a math or EE degree (what they hired) was a good indicator of passing tests.
You basically described Canonical's hiring process !
No only sat scores but specifically they ask for the percentile band of your high school maths and hard sciences scores.
Not even kidding. I’ve been in a staff level+ role at 3 of the 5 faang. Applied to canonical because their products are interesting. I’m ~30 years past high school and i get hit with ‘what are your high school maths scores’. I answered the online form honestly and got a rejection email immediately on send. Phew!
Not at all kidding on that and there’s screenshots of the literally insane questions they ask online.
lol I know, I applied for them around 4-5 years ago and thought it was a joke... I did the first part which was some gimmic IQ tests and then I had to write an essay (which I didn't) !
The filter works both ways so it makes sense. Those employers do not want to hire people like you either.
> similar-to-me bias (I like you because you're me!).
My first boss in the 90s eventually told me why he hired me.
"I assume that everybody at their first job with a CS degree have more or less the same level of technical competence [which is not much IMHO] so I ask which are the last books they have read. You told me a few, I usually get none, so I hired you because I hoped that talking with you would be interesting."
At least a similar-to-me bias builds a pleasing work environment because of homogeneity.
Isn't that bias just legitimized by having culture fitness criteria in hiring?
You cannot use the SAT as a metric to compare different cohorts. SAT scoring has been revised many times over the years. When I took it the highest possible score was 1600. From 2004 through 2016 the highest score was 2400. Now it is back to 1600 again. Plus, both the content and the format of the exam has changed many times over the years. At times, there was no essay requirement, at times the essay was required, and at times it was optional. Hence, each year the examination produces a different distribution/histogram of scores even if you normalize the 1600 vs 2400 difference.
The scores have changed, but ideally they are asking for the percentiles. Those are scaled to the current year.
Even the scaled score is not that informative (and perhaps crosses the line on age discrimination) because for older workers the population of people taking the SAT was much smaller as a percentage of high school grads (and presumably weighted towards higher IQs). It's also why there were so many fewer perfect SAT scores -- smaller population in the bell curve.
Number of perfect scores is also affected by the increase in the number of students who spend 20 hours each a week or more doing SAT prep.
Yeah, test prep was considered more for people who were worried about low scores. 1500 vs 1600 wouldn't make much difference in college admissions at that point.
I somehow doubt that the people that would ask for SAT scores would actually be the sort to think about how those numbers should most effectively be used.
Paid test prep is generally considered to be more effective on the current SAT than it was several years ago which also makes it harder to compare across years.
Don’t make stuff up to defend this practice. The original poster only said the employer asked for the score, not the percentile.
You can look at historical percentile by year and score though.
Which requires them to explicitly ask your age outside the bounds of qualification for a job (over 18 etc). Which ends up opening them to age discrimination lawsuits.
It does not require them to ask about your age, just the year in which you took the SAT. As other commenters have pointed out, this can range from 12 to 17.
Also, they could just ask for your SAT score and any relevant info (if you took it during COVID from your car, etc.) and then you could disclose whatever context you wanted.
That tells them everything they'd need to know to discriminate. If you took the SAT 40 years ago, it doesn't really matter if it was 42 or 47.
People are biased 25 vs. 55 not 33 vs. 34.
I don't kid myself into thinking that employers can't tell roughly how old applicants are. If they're asking for SAT they could also ask for college transcripts/graduation info. That's going to reveal the approximate age of many candidates right there. Finding out what year you took the SAT will add 0 info in most cases.
While I generally agree with you that employers should absolutely not be asking for this sort of information, they're still going to be able to make a very good guess at your general age by the dates of your education and employment.
Unfortunately, there isn't really a viable way to prevent employers from finding out your age from your resume/CV.
Yeah they’re much easier now.
Got a source for that, bud?
When I took the digital SAT a couple years ago, we had access to the Desmos Graphing Calculator during the whole math section.
The entire point of the exam was to test whether you can read a math question, input it into the calculator and select the option that matches the result within 60 seconds. If you get a couple questions wrong, you drop hundreds of points. I don’t think it was a valuable test whatsoever (and of course, it biases to students who can afford time/money for thousands of practice questions to improve this “skill” through repetition)
The English reading/writing section was much more interesting, but again, the time limitations make it a skimming test more than anything else.
Many universities allow you to ‘superscore’ multiple attempts, to combine a math and RW score from different SATs. So again, scores bias towards students who can afford to take one test dedicated to math, and another dedicated to English.
Everything you’re saying makes me so mad
Sure but if you also give the year they can work out the percentile for your cohort.
Although that probably also outs your company at risk for age discrimination.
> When I took it the highest possible score was 1600. From 2004 through 2016 the highest score was 2400. Now it is back to 1600 again.
that's because the 2400 score included an 800 point writing component; the math and reading components remained scored at 800 points (and still do now)
Even during that time the writing component was optional. I took it twice, once with and once without.
But numbers are so easy to compare! Don't take this joy from HRs and recruiters!
/s
As a manager, there are several qualities that I value highly in an engineer, and they all happen to begin with the letter C: Competent, Consistent, Curious, Caring, and Clear Communicators.
While SAT scores might act as a proxy for competency and possibly curiosity, they're not going to tell you much about whether the person is consistently reliable, whether they care about others and cooperate well, or whether their vocabulary or literary analysis skills have any correlation with their ability to read the room and tailor their communication to their audience.
If I were giving these job posters the benefit of the doubt, I would guess they're including this requirement for the same reason that musicians request particular colors of M&Ms in their riders. They want to weed out people (or bots) who aren't paying attention. Nevertheless, there are better ways to do that than demanding (and presumably filtering by) teenage performance metrics.
C = Competitiveness.
I met an HR manager who had worked for a local but well known company with a reputation for caring about things like GPA and SAT scores. She told me that remembering your SAT scores after college was a sign of a competitive attitude.
In the same sense that the high school quarterback continues to talk about the Big Game now that he’s an overweight retail employee making minimum wage years later.
I've seen what happens in engineering with those with low SAT math scores. They need others to do the math for them, or they just wing it.
I remember one who was trying to reduce the noise in an electronic amplifier. He spent days trying random things. Another engineer asked what he was doing, did a quick calculation, and put in an RC circuit that solved the problem.
Off-topic, but can an RC circuit really reduce noise? I can see how it would reduce distortion (which is not the same as noise), but adding passives is surely going to increase thermal noise?
You are correct, but an RC circuit is part of the amplifier circuit and you can do a lot to improve the signal to noise ratio (SNR) depending on frequencies/signals are trying to amplify versus the ones you want to filter out.
I see, so you want to band-limit the noise, and are willing to accept potentially raising the noise floor in the band-limited range, the idea being that if the original noise is sufficiently broad-band, this will still increase SNR?
Noise doesn’t have to be broadband. 60Hz is a common noise in US from the AC line frequency. Then there are harmonics of that due to rectifiers.
Fair enough: I was reading the original comment as referring to (thermal) noise intrinsic to the amplifier design itself (as opposed to noise due to imperfections in DC supply or due to coupling with some external transmitter), but this makes more sense, and we should consider the whole system anyway.
The problem here seems to be that the person was unwilling or unable to ask for help when they needed it, not that they don't know math per se.
I don't know how to do that either, but "winging it" is not something that would occur to me. First I'd Google it and try to figure it out. If it turns out to be nontrivial, I would just ask for help.
And I wouldn't feel the least bit bad about it. After all, those same highly educated folks need my help with e.g. git a lot more often than most software needs serious math :)
The problem was the "engineer" did not know how to design an RC circuit, one of the simplest electronic circuits, in Electronics 101.
Would you rather pay an engineer days to fail to solve a basic problem, or pay a real engineer 15 minutes to solve it?
You forgot that the SAT requirement is not exclusive but an additional data point. While I agree that it could narrow the path for truly good employees, I’d argue that an additional data point like SAT (+ GPA) could tell the employer a lot about consistency of the applicants. Or at least an interesting talking point (“I see you got a very high SAT score but your GPA was lower, what happened?”), if they care.
I think it could serve the purposes of hiring fresh/young graduates. However, it’s still weird if they requested it for people already 5-10 years or more in the industry.
I had a friend with an MS show up for first-day of work for a job that asked for SAT scores on the application. HR said "we never got documentation for your SAT scores, can you provide that?" He was on the phone with his mother, having her go through a filing cabinet when he realized that he didn't want to work for a company that was this serious about SAT scores when hiring someone with a post-graduate degree.
That does seem wild. Out of curiosity, did he also have to submit GRE scores, which would be closer in time and more representative of his current knowledge/skills?
Worker w/PhD was asked by Quebec during residency interview to produce High School grades. Bureaucrats will be bureaucrats independent of language I guess.
I'll ask him. This was about 20 years ago.
I would never ask for them since its so cringe. But SAT scores correlate to IQ at .81, and IQ is one of the few things that strongly correlates to knowledge work performance positively. There is probably a lot of alpha from knowing candidates SAT scores. Its more useful than knowing the college they went to.
Is it really that hard to assess someone's intelligence after talking with them for half an hour?
That would be an interesting experiment, post-interview ask each interviewer to guesstimate the candidate's SAT score.
That's r = 0.81 right? so R2 would then be ~.66. Meaning that 66% of the variation in IQ scores can be statistically explained by variation in SAT scores.
It's really high for psych stuff. If you even get r=0.5, you've got a great result there.
But it is important to note, I feel, that SAT maps to only about 2/3 parts of the IQ score, and IQ score is also a quite fuzzy measure here for things like knowledge work job performance.
I do agree though, you get quite a bang for your buck just reporting these numbers.
But, if you explicitly tie money and compensation to the SAT score, man, that is setting up some very perverse incentives around it. If it adopts widely to do so, then you're gonna get some really strange interaction effects there.
Nothing is worse than a smart bad programmer. There's no limit to the technical debt they can pile up.
>I would never ask for them since its so cringe. But SAT scores correlate to IQ at .81
The combination of these two phrases is the equivalent to "I hate Trump, but ..." in another context.
> and IQ is one of the few things that strongly correlates to knowledge work performance positively
i think you mean that it correlates to pay. nobody knows what you mean by "knowledge work performance." reviews of your peers also correlated with pay. often it is not the smartest person who is the most popular. so... do you see how you said something kind of meaningless?
> nobody knows what you mean by "knowledge work performance"
I actually was pretty easily able to deduce what they meant by "knowledge work performance".
It's understandable to be frustrated by not knowing something, but to claim "I don't understand that and therefore no one does and you're being nonsensical" is a bad look.
Consider responding with curiosity rather than defensiveness.
Nope, I meant what I said.
A very good metastudy is "The Validity and Utility of Selection Methods in Personnel Psychology" (by Frank L. Schmidt and John E. Hunter). It summarizes 100 years of research on predicting job and training performance. It makes a very strong case that General Mental Ability (GMA, their word for IQ) is the single most valid predictor of employee success on the job, not just income.
That's not what that paper says. Work sample tests are more valid than GMAs; the paper just presumes they're too expensive. Meanwhile, we don't have to axiomatically derive any of this: we know that relying on general cognitive assessments to prospective software developers wouldn't work. That's why almost no firms use them.
If you exclude work sample testing from your analysis, all this paper is really saying is that active examination of candidates beats subjective interviews and resume scans. Well, obviously.
Company is Alpha Vantage:
https://beaverhand.com/apply/alpha-vantage-gtm-team-various-...
The job description touts investment from some members of the "PayPal Mafia." For some odd reason that fact that and the SAT requirement combine to make this whole thing feel kind of normal.
> Low ego: There can be significant economic upside to this role as things progress, but the nature of this role day-to-day may involve things that can be less exciting / more mundane (it's a job, after all!)
What does this mean?
Have to make sure all coworker's coffee cups are full.
"I'm the boss, you do what I tell you when I tell you. I say jump out the window, you ask which one."
No way! These guys stole my money a while back. But it was too small (less than $100) to pursue through criminal/civil.
> Please note that we will also rely significantly on both solicited references (where you introduce us) as well as unsolicited or "back-end" references (where we do our own). For the latter, please rest assured that we will never contact a current employer without first getting your permission.
References already give me goosebumps. Having them reach out to people who haven't given you permission to be a reference sounds like a recipe for disaster.
Every job does that, whether they tell you or not.
No they don't. Source: have hired people without doing that.
Sorry, no. That might be a thing you've personally experienced, but it's far from universal.
If they do sensitive work for the government, it'd make sense that they'd do those back-end references. Also if they are in high end finance where you want to weed out people who have demonstrated moral flexibility was well as total lack of it for certain things.
> If they do sensitive work for the government, it'd make sense that they'd do those back-end references.
If they do government work that requires clearances, the clearance process already covers this sort of investigation on its own.
In any case, they are free to do whatever background checks they want within legal limits, but I'd never apply to a company with such ridiculous hiring processes.
i hate references.
all a good reference means to a potential employer is "you are on good terms with somebody from a previous job".
and as a job seeker, it's awkward reaching out to people you may not have talked to in a couple years to announce that you're job hunting.
in my experience it (announcing to people in your network that you are job hunting) is precisely what you should be doing. Some might even point you to the next opportunity. I understand your concern though and have personally only shared it with close contacts (guess it is an ego thing).
airbnb asked me why i didnt go iit in india and interviever soured after that. it was a chinese guy.
Seriously? This made me lol.
They are obviously using the SAT as a safer alternative to the legally dubious practice of IQ testing which can lead to running afoul of the ADA and EEOC. I'm not sure it's much safer, but I am positive it's less safe than doing timed leetcode. At least leetcode problems can be painted as relating to the job.
Additionally, the SAT is a shitty IQ test that is constantly crammed for and cheated on. I remember my SAT test. I was the only person in the room not openly cheating. The teacher proctor didn't care. Higher scores mean better students, more funds, higher home prices, bonuses, and a litany of secondary effects. That's not even including people that pay professional test-takers to do it for them.
The software industry needs to let go of their obsession with finding 10X ROCKSTAR L33T programmers. They never will though. It has gotten worse every few years for decades, and the problems are almost entirely managerial.
IQ testing isn't legally dubious. The idea that it is is an Internet myth. There are a couple household-name corporations that administer general cognitive tests for candidates for some roles.
More companies don't do it because it doesn't work well.
> Higher scores mean better students, more funds, higher home prices, bonuses, and a litany of secondary effects.
Sounds like an IQ test
Canonical is doing IQ testing (with very crappy tests, see my other comment). Not sure if that is because SAT doesn't exist everywhere.
I hate to be the one to tell you this, but every job interview is a derivative IQ test. That includes leetcode.
This by default includes a whole bunch of people who didn't take any kind of standardized tests (most notably, immigrants).
The (albeit small) country I'm from doesn't do any. Reasoning was that standardized tests create an environment where teaching is merely done to create good test scores, not to actually teach.
The SAT doesn’t test course material. It is literally just applying middle school math and English proficiency.
It's not just middle school math. It covers geometry and algebra 2, which very few students complete in middle school.
Starting in 2016 College Board said that they were aligning the test with Common Core, so this might not be true any more.
It's weird that people think AI breaks the concept of work sample testing. Work sample testing isn't "about" programming, and predates the profession of programming. You can (and some companies do) work-sample test sales account managers, customer support, accountants, whatever.
AI changes the underlying job you're testing for. So, obviously, the tests you might have been using pre-AI won't work anymore; they're testing something that isn't really the job anymore. Update your tests so they're about the real work again, that's all. For coding, that probably means assuming (or requiring) candidates use AI to do your assessment.
What AI really does mess with is conversational/interactive interviewing. We do all our interactive scripted interview on Slack, but I can imagine us having to end that practice and return to face-to-face.
I've seen you comment along those lines before, and I think you're both right, and terribly wrong.
By all measures, the way you describe any hiring processes you've designed and had input on designing the metric and the work sample, probably is both high in signal, and not very arbitrary in scoring. They sound genuinely and refreshingly good. Maybe one day I'll interact with one :)
But man, that is very much not how work sample tests work in _a lot_ of places out there.
Places with actual, formalized rubrics for what constitutes a "pass" or a "good" score are very rare, it's almost always just based on vibes of whatever person is reading your code. If there _are_ formalized rubrics, then they have "suggested allowed time" that is incongruous with what the requirements for a "pass" actually are.
All of that is downstream, fundamentally, from the design of the tests not being taken very seriously by the people doing that. And because they're not actually thinking about it very deeply (or they're not allowed to because of time constraints or whatever); "sit down and redesign this task from first principles thinking about what AI enables now" is just not something that happens, and you just get people increasing the scopes of the project blindly.
I agree that there aren't many places that are serious about work-sample testing and far too many that using them as a hiring hazing ritual, but that doesn't make me "terribly wrong", just more distinctively right. We've hired resume-blind at Fly.io for 6 years using work samples with fixed rubrics.
Sure, it was a rhetorical flourish that maybe didn't land :)
Sneaky age filter? You must be young enough to remember your SAT scores.
Sneaky immigration filter? Most wont have an SAT score at all
Between 2005 and 2015 the maximum score was 2400 instead of 1600. Assuming anyone who got <1600 during that period wouldn't admit it, you now have three well defined buckets.
But of course this is a lot of unnecessary steps compared to the usual method: length of work and education history +18 years.
The extra 800 points didn't count for a few years, I didn't practice for it and as expected did bad. Don't remeber the score except I did well enough on math
I remember mine (both from when I was 12 and when I was 17) close to 30 years later.
Hah, you did the one at 12 also? I too remember both, and it was also 30 years ago. I don't know why, probably because they were really important numbers to teenagers, and they say you remember things that happened to you as a teen more than any other part of your life.
Sneaky age filter? You must be young enough to remember your SAT scores.
I can remember mine just fine.
If you're really looking for smart people, use "Answer this word problem in two or more paragraphs. Write your answer on the sheet of paper provided. In cursive."
The signal being that the smart people will refuse to jump through this hoop in your inane process because they have a lot of other opportunities to choose from.
Are people who were forced to learn cursive smarter? They're just older.
Doing this may well expose you to age discrimination lawsuits, since it's just sneaky indirect age filtering.
Another example would be if you required a minimum SAT score of 1601. Sure, someone could have gone off and taken the SAT as an adult or a young child but in reality it is mainly an age filter.
> Are people who were forced to learn cursive smarter? They're just older.
My kids are learning cursive in elementary school right now, FWIW.
Fair enough but it's still an age heuristic.
Are people who were forced to learn cursive smarter?
By definition, people who know more things are smarter than people who know fewer things. That's just how it works.
For centuries, people have striven to improve themselves through the acquisition of knowledge and skills. It is a quirk of recent generations that so many members take pride in their lack of knowledge.
I'm repeatedly bewildered by my Millennial colleagues who proudly say "I don't know what that means," or boast "I don't know what that is" with no sense of shame.
I may be able to help with that bewilderment.
Imagine, you have two people. Person A knows cursive, person B does not. Person B knows the ins and outs of Newtonian physics, person A does not.
Which person is smarter? Which person would the cursive test say is smarter?
What you seem to have mistook for people not knowing things without shame, is people valuing knowledge not by the preponderance of its quantity but by its total when multiplied by its utility.
Otherwise I do not envy the shame you must feel at lacking the knowledge of which plants are edible, how to.clean a carcass, how to fashion a needle from bone and an axe from stone, the mixture of clays to use to make your bricks, and all manner of other once-necessary tidbits whose usefulness has lapsed for the general population.
The person who knows cursive can go on to learn Newtonian physics. Now he knows two things. While your supposed hero still only knows one.
The person who knows physics can go on to learn cursive, should they choose. "Knowing cursive" is not specially indicative of one's intelligence.
Once upon a time, I thought I wanted to learn cursive handwriting. Except this version of me was already in his 20s, would be out of school in a matter of months, and quickly realized the skill would be of such marginal utility in the future that it wasn't worth the hours spent tracing out giant letters like a kindergartener every day.
One could learn this skill in their 20s or beyond, but there's an opportunity cost – why not something else that would actually improve work performance, or that you enjoy doing?
I still wish I'd been taught in elementary school, though, because it would've been really useful as a student. Some of our teachers discreetly handed out practice booklets to students who'd "expressed interest" (their parents taught them the basics and teacher noticed); most of us were not so lucky.
So what did you learn instead of cursive?
Some type theory, some abstract algebra...
Communication skills, teamwork skills...
How to cook better, maintain relationships better, keep a tidier space...
Not perfect! But time well spent.
I'm not sure the definition of smart is so clear cut. If anything, that falls closer under the definition of knowledgeable.
Having to write the "I did not cheat" pledge in cursive was the most difficult part of the SAT for me.
"In cursive" is just filtering for people old enough to have been taught cursive.
sadly my kids were just recently taught cursive in elementary school for some unknown reason
Learning it is mostly useful for being capable of reading it, esp. when encountering historical documents (or when encountering old people)
I can tell you that while women appreciate romantic electronic messages, they appreciate handwritten ones 10-100x more
Or encountering California license plates. Or finding Walgreens.
Not wanting to learn cursive is like not wanting to know lower case just because caps lock exists.
I learned in Brazil. Here in the California I asked my son's kindergarten teacher if she would teach cursive, and she said they don't teach calligraphy and I've never seen it described this way, but she's right.
It’s an approximation for an IQ test
Aside from the fact that they're judging you based on who you were as a minor, it's interesting how it was a startup that asked for this and not a large corporation, given that modern US public schooling is more optimized towards getting you to work in an large institution than a startup.
Same schedule every single day, you get a lunch break, exams are designed in a way that there's only one single correct answer, outside of the box thinking is wrong, and you'll get at least a B+ if you just do your homework everyday.
>Each applicant took the test under unknown conditions. You don't know [...] whether they walked in cold or worked with a tutor for months, or whether they came from a perfectly stable home life or found out the night before that their dog died/parents are getting divorced/worked the night shift at a grocery store to help pay rent. Maybe they struggled with addiction and rebellion in their youth.
I'm pretty sure this is (part of) what's being filtered for quite intentionally.
Canonical?
Heard nothing but bad things about their hiring process.
They made me do an online, timed IQ test, stressing that I should take it in my native language to not waste precious seconds understanding it in english.
It was horribly translated, every sentence was written like something this: "A and B are two broters/sisters. A gives B 3 apples and he/she/them eats one and returns one to he/she/them…" at some point one section had the instructions wrong so I did all the questions wrong. There was no way to change the language or re-read the instructions to try to understand what the original text might have actually been.
That's when I closed the tab.
I'm a Debian Developer.
wild to call out a random org like that…
They're known for asking odd questions like "how did you perform in math in high school" and "please justify your performance in math in high school". It was actually my guess as well before I read the post.
Look at any of their job applications, they're all like this:
https://canonical.com/careers/3752633/application
I like it. I also like it when companies ask for 10 years of [5 year old technology] experience, or say "there's more to working here than the salary!", or other red flags that make it easy to move to the next listing.
If you think my decades old SAT score is relevant, then I know all I need to know about your company.
I only had one company (D. E. Shaw & Co.) ask for my SAT scores. I was late-20s and had to have the recruiter repeat herself two more times before I understood what she was asking.
It was also the single highest density of talent I’ve ever worked, by a long shot. Crazy talented coworkers.
I would think it an inappropriate question if you're asking a PhD (or MS). You could just ask for a copy of the paper(s) or the dissertation/thesis. Some people improve over the time in the uni environment. Some people don't test well when they're bored, etc. Some just grow up.
The SAT vs ACT preference map on Wikipedia is something I had not seen before.
https://en.wikipedia.org/wiki/File:SAT-ACT-Preference-Map.sv...
I took both (since I could just use whichever I scored better on for applications), and the ACT is a much less miserable format. The test is broken up into much larger time chunks, meaning I could take a more useful nap after finishing a 50 minute section in 20 minutes than I could finishing a 20 minute section in 8.
Difficulty wise, the ACT was easier, but not by a lot. They seemed to have pretty similar predictive strengths, based on percentiles of people I knew lining up between the two, but obviously the ACT loses precision because of how coarsely it's scored.
In Virginia, I had never heard of the ACT until I got to college
A lot of responses pointing out various flaws with this question, including the fact that it can be used as a proxy for ageism, the fact that the grading scale has not been consistent over time, or that most foreigners will not have gone through the US education system. However, is it really that uncommon for Americans to never have had reason to take the SAT/ACT, such as, simply not going to uni, or going straight to work after graduating high school?
You only need to SAT/ACT for going to get into an university. And these days it seems to matter less. I looked it up and less than 50% of high school students take the SAT/ACT. Personally I never had any employer ask for my SAT/ACT scores since the 90s.
I wish I could find a position that would. My SAT scores, by far, outshine my CV.
I just applied to Epic, the EHR company from Wisconsin, and I can confirm that they also ask for SAT scores. Thankfully I have my collegeboard credentials saved
Out of curiosity, did they require SATs, or just ask? And how many years out of HS are you? Seems like it would be crazy to ask for them once someone has finished college and has a UGPA to report.
Their pre-screen test was awful. Brain teasers, including some infamous ones like “if you have two coins that make 15 cents and one is not a nickel, how is this possible,” and moderately involved programming questions like “parse phone numbers from a file and record those with any of these area codes OR every other digit is a 2 OR they have multiple pairs of consecutive digits” and you’re given a blank text box with no formatting, IDE, or even non-word processing style indentation help.
Thankfully, the coding assessment does have syntax highlighting for plenty of languages, but in general I feel like face-to-face assessments are more productive. TFA seems to make a similar point
I see we (as a society) are now full Primeagen: "Epic as in health records, not Epic as in Fortnite."
> Edison also subjected candidates to the 'salt test'. He'd serve candidates soup, and if they salted it before tasting it, he'd allegedly disqualify them. His theory here was that this proved they operated on assumptions.
I really don’t want to eat soup in front of someone I’m interviewing with.
> Take-home projects or a trial period of some kind. This makes the most intuitive sense by far: having candidates do a representative slice of the job gives you a solid idea of whether they'd be any good at it. Combining this with structured interviews was (before AI) considered a gold standard; you'd get a sense of who they are and how they work by talking, have a way to compare them pretty objectively to other candidates because of the structured and consistent nature of the interview process, and then you'd get a sense of how they apply their attributes practically to the job via the work exercise.
Unfortunately a lot of companies have over the last several years been using this to get candidates to do a project for free for them. If it's going to take more than a few hours of my time, I don't take project style interviews seriously unless compensation is added (which some companies do offer and is a big green flag).
Definitely been tricked into working for free a time or two.
Best middle ground I ever had was an interview where they impose a strict 1 hour (maybe it was 90 minutes, idk) time limit between when I got the prompt and when I emailed them back. Then they spent some time looking over it, then I had an interview with an engineer who had read my code and we chatted about it. Why I had made certain decisions, what corners were cut because of the time limit, etc.
Felt very fair. Not enough time to assign a valuable task, enough time and privacy that I wasn't under the gun like you are in a whiteboard interview, and it was pretty applicable to what I would be doing at the company. Solid interview. Didn't get the job, but respected the process.
This style works for me too. I don’t necessarily think the live coding interview is always the best presentation of what I know and some types of interviewers try to apply pressure in those situations I don’t feel is entirely fair, plus you’re often forced to use a third party webapp where it’s entirely out of the normal workstation flow I’d be more comfortable in (I use mostly text editors with plugins when allowed).
there has been a proliferation of “take home” exercises that are them just trying to pick your brain for problems they’re struggling with. I have had a “take home” project that led to a trial contract employment that required me doing a ton of work for free I knew they had used later, because I saw the trial API key I had set up for the project used much later on down the line. Never got paid, clearly work was stolen, not much you can do about it and there’s a real power imbalance there that’s difficult to really do much about. Was a tough lesson. When done right I think it’s the best way to screen for good candidates that may not be good live coding interviewers.
I do take the position that like, I get why you may have a culture where live pair programming in high pressure situations may be valued. But in most orgs the async nature of -> get requirement -> implement asynchronously -> seek feedback and adjustments -> iterate is far more realistic, but when you deliver something, never hear back, get hit with the whole “we’re going with another candidate” feedback feels like you just got ripped off.
When I graduated in the 1970s, and looked for my first job, it was expected that some company might ask for such scores and, iirc, one did.
> Since 2023 or so, I've noticed more and more tech companies and hyper-growth startups dropping bachelor's degree requirements.
Those requirements were never real, anyway. I don't have any degree. The last time I was seriously questioned about it during an interview was 2004.
While it does not come up in interviews, it does seem to come up during resume filtering. (Plenty of jobs I'm otherwise easily qualified for that I didn't get contacted for, even back when basically having a pulse and knowing code could get you any job you wanted.)
Didn't even read the article, first thing I did was go to comments and Ctrl+F "Canonical". Seeing several results light up was really amusing and validating. What a truly awful hiring process
I couldn’t even tell you my SAT scores. I took it and did reasonably well, but after college it just ceased to be information my brain retained. I don’t even know where my results are, so it’s completely lost information at this point.
YC's application asks for test scores if under 25, although it doesn't specify which ones.
Will humans ever replace these tests with discussions with a super intelligence that creates a hypergraph projection of our understanding?
Interesting. Xfinity blocked this page as a "security threat." It's likely doing that solely based on the TLD.
I'm surprised they didn't at least attempt to email them to ask why.
Ignorance is always a possibility here, as it might be their first time hiring.
1600 and your GPA was 4.0. Any other answer proves you aren't trying hard enough to get the job.
4.0, what no APs? Insta-rejected. /s
If I may rant. My oldest son will be a junior in HS next year at a STEM magnet (Townview SEM in Dallas). I think even one of his electives is an AP, It’s kind of ridiculous. APs in HS seem pointless if you can’t apply the credit to your college degree plan. As if a University is going to let a STEM major out of calculus because they took the AP in HS…
At my highschol, "Honors" classes were the ones that didn't get college credit and "AP" classes allowed you to take a placement test at the end to get college credit.
Same here, but selective universities will often not let students out of requirements with AP scores. They may let you get credit toward graduation, but if you have to start with all the base level prerequisites then you've not made much progress.
Basically, universities are aware that APs have gotten easier and are not a substitute for college-level classes that selective universities offer.
Agreed. It seems like they're mostly for showcasing your ability in HS, not getting any actual credit in college.
I don’t even remember my own GPA, for me college was a lifetime ago!
It's always odd to me when filters are applied, that arguably neither influence, nor impact outcomes. From what I can see, SAT scores have effectively changed over time, such that reading comprehension has dropped dramatically, with maths abilities increasing (https://www.pewresearch.org/short-reads/2026/06/11/for-the-1...).
One (me) might argue that now, more then ever, reading matters. The age of AI, only increases the value of critical thinking and reading comprehension skills. Combine that with a (notable) 5-10% change in SAT scores, and it would appear that this practice is unlikely to unearth what factor is being filtered for.
Mind you, no one has ever not accused our industry of tunnel vision, and confirmation bias.
> Why would you ask for a self-reported, unverifiable test score that could be decades old at this point?
Because many colleges that used to reliably filter for them no longer do (or didn't during a several-year period).
It's true that self-reported scores are not the most accurate, but if I were applying for a job I would report honestly, on the assumption that they could easily request for the scores to be sent by the College Board. The risk/reward of lying does not make sense, at least in my case.
> > Why would you ask for a self-reported, unverifiable test score that could be decades old at this point?
> It's true that self-reported scores are not the most accurate, but if I were applying for a job I would report honestly, on the assumption that they could easily request for the scores to be sent by the College Board.
No, they couldn't, except by going through you (the College Board doesn't take third-party score requests.) You might be able to request that if they are recent enough, but not if they are literally decades old (well, not if they are ~21 years old or older.)
https://satsuite.collegeboard.org/scores/sending-sat-scores/...
I am aware that third parties can't request scores. I was referring to the employer asking to have the scores sent, which the applicant would be compelled to do (or look like they fudged their original score reporting).
I'm also aware that the College Board doesn't hang onto scores forever. I doubt there are any employers who require SAT scores for applicants who took it prior to 200% (the cutoff indicated in your linked article).
Age and socioeconomic discrimination. They want only younger people who took the SAT from yuppy societal bastions. Kind of like how Handshake was full of jobs where the firms wanted only recent graduates.
Clearly they will pay a lot of money but be badly run. They want to avoid wokeism beneficiary kids and old folks. An actual job related aptitude test would be better. Half the nation didn't even use the SAT at that age. They used the ACT.
They want young coastal yuppies who were not wokeism beneficiaries at the DEI universities. Those universities broke signaling so making all their alumni show SAT scores instead of relying on the brand will mitigate the fallout.
ACT takers were from lame red states anyway. They want the young blue state brains without the old blue state baggage.
Tech companies more and more delusional every time, thinking they are special and that their product is so critical for society, and possibly more complex than operating a particle accelerator. So, you basically have to be an outstanding genius in order to have the "honor" to work for them. Then you go in and it's just a glorified vibecoded spaghetti CRUD. Not surprised.
Is SAT as good a preditcor as EQ, I wonder, in LLM based work. It seems clearer articulation is more important.
Huh, I never took the SATs. What then?
Another thing making a comeback -- reference checks. I had to supply references for my first job in 1999. Then I wasn't asked for them again until 2024, and then for every job after that.
Whatchoo talkin about, Willis? I've had to supply references for every job I've had.
Somehow it seems every other hackernews was living in a much better timeline, job-wise, than I during the ZIRP era.
Soon people will be taking proctored IQ tests just to be allowed to submit a resume.
As I mentioned elsewhere on HN [0], younger generations are much more competitive now.
Visit and talk with undergrads at a top CS program like Stanford, Cal, UIUC, MIT, etc. The culture is different because this is a much more competitive generation. When the acceptance rate into a top CS program is in the 1-5% range and laurels like being a Valedictorian, NHS member, JV or Varsity sports team member in HS, getting a 2100/1500+ on the SAT, and taking 6-7 APs are now table stakes, you get a degree of viciousness, competitiveness, and steel-eyed execution that a lot of older Americans just aren't used to.
This mindset is the norm across Asia though - from the Gaokao to the JEE to SKY-or-bust. Honestly, I'm glad that younger generations are much more competitive now - pressure makes diamonds.
And honestly, the top 40-50 STEM programs nationally graduate around 30-40k new grads a year. Add to that respected regional programs and Veteran-to-Employment pipelines and you have a self-sustaining talent pipeline.
[0] - https://news.ycombinator.com/item?id=48506001
I disagree with the increase in competitiveness being a good thing. Excessive filtering at all levels has meant that eccentrics or absent minded professor types are not making it into research roles, and creatives or mad geniuses are filtered out before they have the chance to make an impact. There are a lot of people who are extremely bright and creative, but just don't have it all together the whole time from ages 14-25, and these days they have no chance of making it into research positions.
The system is rewarding conscientiousness and consistency over creativity.
> The system is rewarding conscientiousness and consistency over creativity
This assumes that you can get to the top via rote skills alone. Rote learning only gets you so far and most of those kinds flame out.
It's hard to describe, but once you meet actually talented people what you end up seeing is that they're just extremely diligent and deeply passionate about a topic and will continuously execute.
For example, when I was in HS I wrestled. Yes there were physical differences that could impact a sparring round, but technique and preparation was almost always able to outcompete base innate talent. Later, I ended up learning ballet the Russian style and it was the same - the truly creative types who were at Vaganova or Paris had already built strong fundamental and technical skills which allowed them to mix and match and create.
You cannot be creative without also being diligent and understanding fundamentals.
The "eccentrics" and "mad geniuses" are few and far between, and to find people with talent, you do need to use exclusionary tactics like scores and interview performance.
Not really. Outliers of that sort get dealt with as an exceptional case and it works. I've seen amazingly bright batchmates get into all sorts of programs without most of the qualifications, because they were a genuine math wizard (the kind that submitted errata to a standard textbook on some weird number theory stuff at age 15), but they didn't always score the best on many exams, especially when aggregated across subjects. IIRC he's a researcher now. All trends point to him being an eccentric old hag at age 50 ;)
The filtering system is meant for the majority case and there it works. The outliers get dealt with as outliers, which also works. In this case, he later asked the author of that textbook who he emailed with the errata, to connect him with the group he wanted to work in. Needless to say it was a very strong referral.
The US education has always been competitive. In sports.
Its school system has always been a state-sponsored daycare.
SAT/ACT tests reflect this. I can get a perfect score in SAT math easily. And I likely could do that as a kid (I never took standardized tests at school). I wouldn't have been able to get the perfect scores in the Chinese gaokao or Korean/Japanese tests.
> There are a lot of people who are extremely bright and creative, but just don't have it all together the whole time from ages 14-25, and these days they have no chance of making it into research positions.
This is just nonsense. Are you saying that we should kick out smart kids with high test scores to let in absent-minded students who care about only getting drunk so that they _might_ become great researchers in their 30-s?
To the topic at hand: it's way too easy to fluff your resume with nonsense like "Coordinated a responsible team for an implementation of cross-cutting concerns improving customer retention change by 12.23% across the organization". Test scores provide at least some objective measurement.
I'm not opposed to the use of test scores, it's more the requirement that kids have the whole package of grades+activities+honors societies+test scores consistently through a phase of life that is widely understood as tumultuous for many. We're selecting for robots.
There's a reason the west was so productive in terms of new scientific and technological discoveries in the twentieth century, and it's not that our scientists were the most consistent conscientious students who prepared extensively for exams and padded their resumes in just the right way.
> We're selecting for robots.
And to add to this, learning itself is a _skill_. Working on a complex problem, looking at it from different angles, spending time memorizing facts, working on learning to paint fine lines - these are all skills that you need to master.
By not motivating children to do that during their formative years, you set them up for failure later in life.
Yes, there will always be exceptions, humans are extremely variable. But for the general case just letting children float along without any goals or competition is not a great general strategy.
> the whole package of grades+activities+honors societies+test scores
This is _also_ a very US thing. Without true competition, students have to fluff their "resumes" with nonsense to get admitted into good colleges.
Other countries have tougher tests that can provide a better signal.
> We're selecting for robots.
I disagree. We're selecting for people who can set a goal and follow it.
Apply the same arguments to sports. Should we not stop all the competitions until the age of 25?
I didn't really state my whole view, but hard exams like Oxford and Cambridge use make sense for elite colleges to use (rather than unrelated extracurriculars), but reforming the whole education system to be oriented around a single high stakes test like in China or Korea has its own severe costs. I do not want high schoolers to spend 20 hours a week at hagwons, and the current resume filler system is also terrible. I do not think declining admissions rates at elite universities reflect that the students are any smarter or more prepared than they were 20 years ago, but rather they are much more cutthroat about many things that are orthogonal to being successful adults.
Ideally we'd follow a more exam focused system more like the UK, though I wouldn't want to require all students to only study 3-4 subjects towards the end of high school. But something in the european model of IB/Abitur/A Levels, where there are serious exams in various subjects at the end of high school for all uni bound students, plus some special higher level exams for the most elite unis (in the vein of Cambridge's Sixth Term Examination Paper). We could probably repurpose AP exams to fill a similar admissions role to A Levels, and possibly use the AMC/AIME/USAMO more explicitly for admission.
Edit: this doesn't let me reply again, I think the chain got too deep. But the point is that we're not just using AP exams and USAMO scores, we're also using a pile of other metrics around extracurriculars, GPA, and honors societies, and the end result is stressed out children and not actually having better outcomes than we did two decades ago. Declining admissions rates at elite colleges do not reflect smarter and more productive incoming students.
Yeah, Chinese-style gaokao would be impossible in the US. And I think it goes _too_ far into the "competitive high pressure" direction.
There are many ways to make a more competitive and objective system. I honestly don't have a lot of professional experience with any particular one, so I don't have strong opinions on a particular form it should take. A European model is good, some kind of mix of Chinese+European would also be great. And ultimately, these systems would be more fair for applicants.
And the current topic just highlights the ridiculousness of the status quo. For most people in the US, SAT is the _only_ objective test score that they have.
> We could probably repurpose AP exams to fill a similar admissions role to A Levels
They are already being used like that in college admissions today.
> possibly use the AMC/AIME/USAMO more explicitly for admission
Already in use explicitly at most of the top CS programs today
> I do not think declining admissions rates at elite universities reflect that the students are any smarter or more prepared than they were 20 years ago, but rather they are much more cutthroat about many things that are orthogonal to being successful adults
The cutthroatness is what we should be optimizing for long term. Competition is what begets innovation.
Also, even at solid middle tier universities like UC Riverside, the calibre of student has increased dramatically over the past decade [0]
[0] - https://news.ycombinator.com/item?id=39369724
> The cutthroatness is what we should be optimizing for long term. Competition is what begets innovation.
The difficult pill that society refuses to swallow is that most of the simple innovations that are feasible under our constraints have already been innovated. There are no more individual 'inventors'. The source of modern technological innovation are lucky discoveries that manage to scrape from the bottom of the easy-to-find barrel, genuine one in a million genuises and (mostly) the hard work of large groups over many years. Even the brightest top 0.1% of people mostly won't be coming up with ideas that a thousand people before them didn't already have.
Yet people will not settle for this. Instead of acknowledging it, we'd rather apply the good old human model of 'if 1x of something is good, then we need 1000000x of that' and take healthy competition to transform it into a ruthless race to the bottom.
Past a certain point of competition, you stop selecting for the metric you care about (academic performance, curiosity, creativity) and start selecting for people who are willing to slave away sunrise-to-sunset to work on tasks they know are almost meaningless but that contribute to your selection criteria. The metric becomes the only goal. You're also indirectly selecting for wealth, because rich parents can afford to spend lots of time with their child, not make the child work, pay for all their extracurriculars and tutoring.
The diminishing returns are immense, and we can already see that in countries that never fought this race to the bottom. Their students are more miserable than ever, and for what? Frankly, forcing children to waste away their best days to make them clear arbitrarily rising barriers that we know are meaningless is unthinkably heartless.
> There's a reason the west was so productive in terms of new scientific and technological discoveries in the twentieth century
> it's not that our scientists were the most consistent conscientious students who prepared extensively for exams and padded their resumes in just the right way.
Instead, a large portion were immigrants or the children of immigrants who arrived in the US as part of trans-national brain drain from countries with strict education systems (eg. Hungarian Jewish Americans in WW2, Eastern Europeans in the 1980s to present, Asian Americans today).
There's a reason Asian Americans, Eastern European Americans, and immigrant African Americans are overrepresented in leadership and white collar industries despite the very real handicap of having extended periods time without US citizenship or a greencard.
Instead of optimizing for feel-happy edge cases, we should be optimizing for building the best talent where possible, and that requires being competitive.
> We're selecting for robots.
Frankly, this is insulting as well. Yes there are some late bloomers, but they are outliers. If they can truly succeed they would stil find a non-beaten path to succeed in a competitive ecosystem.
> consistently through a phase of life that is widely understood as tumultuous for many
Only to y'all "heritage" Americans. For those of us who are kids of immigrants, we learnt that life is a race, either you compete or you fall to the wayside.
Of late, US folks coming out of university optimize for money and head for things like finance - people who hire STEM people. Those more interested in the field than the bucks go for advanced STEM degrees. And if the Yanks don't go for the bucks, they go for the MD.
Yep, and we end up in a situation where the US-born people are working on "people skills", getting MBAs, and then going into management positions to manage the financialized companies.
Except that there aren't that many management positions. And once _everyone_ is doing complex financial stuff, you end up losing competitive edge against other countries.
> US folks coming out of university optimize for money and head for things like finance...
Not really, and I say this as someone who works in VC with peers in PE, Growth Equity, and other segments of high finance.
If you have the resume to get hired as an IB Analyst you will also get hired as a new grad SWE or APM at OpenAI, Google, or Roblox where they would earn the same or more than as an IB analyst with chiller work hours.
People overestimate finance salaries - it's the same as big tech with worse hours.
> And if the Yanks...
I don't think you live here in the States or Canada and as such haven't experienced our job market.
Please butt out of the convo.
---
Edit: can't reply
> Have you experienced the US job market outside of the Valley and NYC
Yes.
Before I switched to VC, I've managed teams and hiring for teams or product lines that reported to me in North Carolina, Georgia, Virgina, Texas, Washington, Massachusetts, and Colorado, and helped open my previous employer's Prague and Warsaw offices.
I also started my career outside of Bay Area or NYC tech before my stint as a staffer.
Additionally, the majority of tech hiring in the US remains consolidated in a handful of geographical locales [0].
[0] - https://www.bls.gov/oes/2022/may/oes151252.htm
Judging from your career path, you shifted from tech to money towards the start. I hope you were a good boss when you were starting up, as such people are rare and wondrous creatures. Forgive my choice of jargon - I divide the world into people who do the work and those who talk about the work. And the latter group tends to set the rules and collect the profits. I preferred the hacking. And tech != STEM, as biotech/pharma is rather different than selling ads and harvesting information.
And FWIW, I've worked all over N America.
> Judging from your career path, you shifted from tech to money towards the start
What the hell does that even mean?!?
I started my career working on on low latency computing at a networking company and was paid roughly the same I would have earned as an IB Analyst at JPMC. If I kept climbing the tech ladder I actually would have ended up earning more than I do today.
> I preferred the hacking...
Vast swathes of the tech industry are working on actually innovative stuff AND being paid top dollar, such as my stuff in HPC being dual use.
> And tech != STEM, as biotech/pharma is rather different than selling ads and harvesting information
I know. Not all "Big Tech" is AdTech. And salaries for scientists in the biopharma industry are comparable to big tech salaries as well (not for the SWEs in that industry though - they're cost centers not generating IP that matters).
Have you experienced the US job market outside of the Valley and NYC?
The US has always been able to identify the truly talented students and get them into the best learning environments.
> [...] you get a degree of viciousness, competitiveness, and steel-eyed execution
I think there's a lot of truth to that. (Aside: Many manage without the viciousness part. It's not their fault their parents lined them up with an internship and a research paper co-author in high school, and they're not jerks about it.)
Though the current generation of students didn't invent hyper-competitive. Before software engineering jobs (and startups) were high-income and high-status, you'd see that mentality among many people on track for Wall Street, for example.
Another example: Before CS was a go-to for the hyper-competitive, a mentor of mine actually switched from pre-med to CS, at an Ivy, because a percentage of pre-med students were outright sabotaging other students, and it turned him off of the field.
> that a lot of older Americans just aren't used to.
Though, there have been -- and hopefully will remain to be -- people doing it for the love of the field, who are not impressed.
Other than the genuine people being crowded out of admissions slots and fratbro interviews by Wall Street types...
If a Palo Alto helicopter-parented overachiever McDojo black belt tries to pick a fight... with a humble rope-belted person in Asia, who's studied martial arts for the love of it... the latter will chuckle good-naturedly, and help the Californian up off the ground.
> a Valedictorian, NHS member, JV or Varsity sports team member in HS, getting a 2100/1500+ on the SAT, and taking 6-7 APs are now table stakes
This is very true in my experience, except I subbed out Valedictorian with multiple varsity sports/student government and the SAT with ACT and I didn’t even get waitlisted at top schools.
“ getting a 2100/1500+ on the SAT” Typo? 2100/2400 << 1500/1600 in terms of rarity.
Extreme competition isn't good. It will just lead to race to bottom.
I'd rather that than leet code dancing around: "It's not an IQ test, since those are bad, but this is okay though!"
My problem is it's self reported, so it ends up being a "are they smart or did they lie?" game. You can't easily verify it for anybody over 23 or so.
Canonical has a job application where you are supposed to rank yourself on a percentile (up to like 1 in 10,000) on how good you were at math in high school. It's a very easy way to incentivize lying, and also to hire people with an excessively high appraisal of themselves. There are a lot of people who are reasonably good at math, and have avoided humbling environments like the Putnam, and have convinced themselves they are God's gift to math, when in reality they were just the brightest kid in a class of 100 high school students.
Fair enough, but the vast majority companies that interview like this would be totally fine with the brightest 1 of 100, if not the top 20 of 100.
If you want the job, can’t you just lie? Or are SAT scores something that cannot be faked? I dunno, I also say I know Kotlin when I have more experience in Java (and honestly I couldn’t care less about specific tech stacks), or that I know about tcp/udp when all I have is read a couple of (good) books about it.
I don’t feel bad lying about some stupid requirement
> If you want the job, can’t you just lie? Or are SAT scores something that cannot be faked?
I would assume that if you progress to the point of an offer, they would ask you to have the official scores sent by the College Board. Apparently they hang onto scores back to 2005 and can send them for a fee.
I’ll do you one better:
I was denied a role with a major engineering firm based on my 3.something GPA!
They needed a 3.4 or 3.5.
I remember the Cisco (!!) table was the most popular table at the campus job fair, and they turned away anyone without a 3.8+ GPA. I had a 2.42 (2.40 was minimum degree requirement).
Dodged a bullet there. I've worked happily at a FAANG for many years now and somehow I've avoided living in a cardboard box by a dumpster.
Many years ago I was walking around a college job fair when a recruiter was yelling at a college student. His crime: "wasting everybody's time" by presenting a Resume with a 3.4 GPA when the company clearly listed a 3.5 GPA as the minimum they would accept.
If this is how they treat people that don't yet work for them, it doesn't bode well for how they will treat people that work for them.
> One of the least effective predictors was unstructured interviews or 'chats’
Yeah those are the worst, one time I had an “interview” with a company that I really liked, the founder is also an awesome guy and we chatted few times and all is well. Then I got invited to their facility, great place and team, some of them were structured on how they evaluate, but most of them were an absolute mess, and some of them were hostile as if I would get hired it will get them fired the day after (the passive aggressive of trying to belittle your projects or work and not trying to understand your approach it but to attack it instead) and when I would ask them in a good faith about something they did, you would get a fake halo effect with “oh I can’t tell it’s secret! NDA bla bla” as if they did a patented work.. it was horrible method to hire people despite the great founder I knew.
In my opinion, the best way is what I usually do, after initial screening, I give them an assignment that they can do in few days and then return the work, the quality of the output will determine that, and it’s exactly how you will do in real work anyway, and you get to measure their critical thinking and problem solving rather than how would they sell or articulate something on the spot (maybe they are overwhelmed and their head went blank), as I am looking for an engineer not a sales dude, and they would tale some time to build and solve it.
I've been talking to a few companies lately, and one just keeps stringing it along, having me talk to manager after manager. It's been 6 weeks, and still no end in sight
but that doesn't mean anything
If it's been 10+ years and an employer wants your SAT scores, 1600 is as good an answer as any. Anyone asking for that data point doesn't actually care about the accuracy, they just want to see if you'll compliantly jump through a pointless hoop.
(Save the "but that's fraud!" replies. It's not material to the job, so it isn't).
That cost-benefit analysis makes no sense if you had a pretty good scores. What's the point in fudging a few extra points if it means that diligence reveals you to be a liar?
It would also be somewhat suspicious if you went to a so-so college but allegedly had a perfect SAT. It would only make sense to lie if your score was well under 1600, you went to a college that makes sense for someone with a perfect SAT, and you didn't think it was likely they would follow up with a request for the official score report.
In service to the point, if you're _really_ smart—like definitely going to grad school smart—you apply to universities with the best faculty for your intended area of study. Often that is a sole professor at a university you've never heard of, but the researchers in that area of study regard as a top school.
"You're partly making your decision based on who someone was as a 17 year old."
Sure, but IQ tests show a high degree of stability over a person's life. It's not unreasonable to be interested in it for sorting.
SAT isn’t an IQ test, and probably all sorts of people took it before they had cultural awareness or a diagnosis that would have lead to different testing conditions had it been taken after diagnosis, let alone the fact that test scores are not comparable.
GPAs similarly not comparable over large time ranges, schools, or degrees without normalization you can’t get.
SAT isn't a perfect IQ test but it's not bad for a first-pass filter. Nearly all who scored 1600 will be bright and nearly all who score 800 won't be.
It's a bit like BMI. Yes, if you're Peter Dinklage or Arnold Schwarzenegger it will be pretty meaningless. But most people aren't and BMI works pretty well for them.
SAT isn't an IQ test but IQ is very correlated with SAT scores.
While edge cases may exist, for the most part, a test score / GPA / anything where the numerator is almost as big as the denominator is a good signal.
Can you say the same for SAT tests, where the score is best of N and N is however many the person can afford and varies between candidates?
And the meaning of the score changes over the years based on the test itself changing. Same goes for the company's GPA requirements where there have clearly been shifts across schools on the amount of grade inflation allowed or even encouraged.
As an aside, I'm not sure if I or the College Board can prove my score at this point.
Not only that, but the SAT is not an IQ test and you can definitely study for it. Students with wealthy or motivated parents can get study books or tutors which makes a huge difference in score.
The Princeton Review promises a 200 point score improvement with some of their packages. And they can fairly-reliably achieve it too.
And I guarantee you that claim is based on an intentionally flawed experiment where they take students who have never seen the test before vs after completing the program. The actual control should be against students who have taken a couple of cheaply available practice tests.
You can study for most IQ tests as well.
Yes; though SAT is less prep-resistant and it'd be smart to apply a "+/- 100pts" fuzz to a score.
I never done an iq test, but just curious, can’t one simply “rehearse/study” for such tests? I dunno, let’s say you do a few past IQ tests (with answers available), I guess one could get a higher score just by doing that
You can. IQ scores are not stable throughout one’s life.
I dislike this argument. I think in some dimensions these types of tests can work, but I’ve never been the type of person who’s been able to score well, and I don’t test particularly well in general, yet I did my PhD work at <IVY LEAGUE> and have had a great career despite this. I think that testing is good for people who can be adequately evaluated, but for people like me it just leads to a lifetime of feeling like something’s wrong with you.
Test-taking may only roughly correlate to intelligence, and is just one dimension of a human... but they likely care less about false negatives (like you) than they do about false positives in alternative assessments.
I can't speak for others here and we don't have SATs in this country, but these things can be very unfair. When I was seventeen, I had a lot of things to deal with which were not of my own making, such as caring for a terminally ill relative and wondering if I would even have a roof over my head a few months down the line. That kind of thing tends to take your mind off school work. Several years later and I was in a much better place.
I think it's fair enough to say teenagers in general have more instability in their life even without this.
If you think it matters (I don’t) and think you scored unfairly low at 17, you can take (or retake) the SAT at any age.
We don't have SATs in this country (like I said). Yes, we can retake the equivalent here, but retaken exams are valued less by both employers and universities.
I spent much of that period visiting hospices and hospitals, and eventually ending up homeless for a while as a result of all this. I was more interested in being able to support myself financially for the next few years, and recovering psychologically.
My qualifications after this consisted of diplomas and degrees.