I am teaching some math courses, and I see how LLMs disrupted all standard approaches, and I don't know what to do.
I rely on written exams that are open book, but forbid any use of computers and smartphones in class.
The university insists that homework can't be optional, but it lost its meaning. I've tried to explain to my students that it's in their interest to think about home assignments on their own, but the majority will obviously use gen AI, and it's such a waste of time to check and grade LLM output.
I just had an experience where all students were given "sample problems" to try at home and prepare for the written test. Many of them just dumped the document into an LLM, asked it to produce some "exam guide", and showed up with that thing printed out, asking me during the test to explain what LLM output meant.
It seems like the university also has many people pushing for AI use for everything, but I teach basic stuff where the goal is to make students think on their own and digest some fundamental ideas, LLMs can produce perfect solutions, but relying on them is pointless.
I teach math-heavy economics courses. I found that grading homeworks for completeness and pairing the homework due date with an in-class quiz on the problem set material (taken in class the day the problem set is due, after the set is turned in) has reduced the incentive for students to use LLMs to complete the homework. Exam questions closely mirror what's on the homework, and exam grades correlate with students who make legitimate attempts on their homework. I can sus out legit attempts typically by handwriting. No erasing and fast writing means it was copied.
For grade weights, I weight the homework the lowest of any grading category, with in-class quizzes next, and exams scores the highest.
Homework is a tool for students. It is where they practice. I see my job as a prof to curate the problems I think will help students best prepare for the exams, and use that space to help them gain the intuition about the problems I don't have sufficient time to cover in class, like through an application. As my profs before me used to say, true learning comes in doing problems.
I find bright people who are self-motivated tend to do problems on their own, but this is not the modal student. This is like the top 10-15% of any class. The modal student is motivated by getting a degree so they can enter the working world, and sees coursework as an obstacle to that. They'll take any and every shortcut to get the highest grades with the least amount of effort. However, particularly with first and second year undergrads (but I've also seen the same behavior in grad students), they do not understand the purpose of pedagogy. That doing problems is where you learn the material.
So if you make homework optional, the 10-15% hold steady and do it but the bulk of the class will stop doing problems and tank their grades on the exams. They will not internalize the lesson after that first bad exam, because it is morally and practially cheaper for them to believe they had a bad day or that their professor (me) is just a terrible meanie who constructed an exam with problems they've "never seen before" and "had no connection to the lecture" (actual evaluation comments I've had when I've tried to the homework-optional method).
Perhaps they are also dealing with administration that says things like “The university insists that homework can't be optional”, as in the grandparent comment.
The university administration needs flexibility rather than draconian rules that dont work.
In any case, even if the uni insist on homework, the teacher can simply mark everyone who hands in their homework 100%. It is impossible for the administration to police it, or if they do, they'd need to hire someone to mark the homework (which conveniently solves the teacher's issue).
Students who copy all solutions from LLMs maybe learn something, but it's like 5% compared to what someone can learn by doing the assignment without external help, or even by asking their peers. It's a bit disheartening, because I invest quite some time and thought in designing the assignments.
Exactly, it's the most important part of the math education, there is no point in memorizing definitions and theorems without putting everything into practice.
But it seems like the temptation to get perfect solutions from an LLM is too big for many students nowadays, they lack the maturity to understand that using LLMs for basic assignments is counterproductive.
Yes it's all fun and games reading uni level maths nodding along to all the proofs until you're suddenly asked to do the exercises and suddenly nothing made sense and you are completely confounded by such an innocent looking question.
This is a great approach. I remember even pre-AI having issues where, at a big state school, basically all the answers to homework could be found online. Sometimes I made the choice to copy a homework and tell myself I’d catch up later. But the classes where there was a scheduled quiz right away just forced me to keep up.
> The university insists that homework can't be optional, but it lost its meaning.
The problem is not LLMs, but your university. In my undergrad, almost all the math courses had no required HW. They'd assign it, but you wouldn't turn it in. You'd come to office hours for help on the HW, or ask in class (they'd often dedicate the first 10-15 minutes of each lecture to Q&A).
It was awesome for people like me. No time wasted on neatly presented HWs. I'd do it quickly, verify the answers, and study for the exams.
The flip side was there were many exams (you don't want your grade catered if you do poorly in one exam). A course like calculus could have 4-5 "midterms", and then the final. Often, they'd drop your lowest midterm score so you're allowed one bad day.
Indeed, I've switched over to model similar to this in the course I teach. Weekly optional problem set + in-class quiz every week with one homework problem verbatim + an extension. The homeworks are completely open resource, LLM, friend, sorcery, w/e. I figure that, even if they're given an oracle, they have to internalize the ideas to the point where they can generalize on the follow-up. I'll only really know when the course finishes, but I think this will work out. I want to pair this with an oral midterm and final, but that'll have to wait a semester.
In my country of origin (not US) that's how it worked starting in high school. Homework was called "suggested problems". The teacher reviewed them in class. No grading. We ended up doing it because that was almost the only way to pass the exams (which, btw, could be up to 50 questions long...)
I heard from a friend who went to German university that the homework was never graded, maybe you could get help with it but the professor didn’t have TAs and had a large class. Lots of Midterms were also not feasible given the setup, so often 100% of your grade depended on the final.
I can’t imagine as American how that would go down with most American students.
I think a possible solution is to make homework carry no grade weight, and simply mark it as complete or incomplete, like a presence check.
Homework still provide valuable practice for students who are truly committed to learning. An automated system could provide an initial assessment and feedback. However, students can request human feedback paired with an in-person meeting. Such requests would require a mandatory explanation of what feedback they want or what the issue is. Professors and TAs will therefore only read and answer such requests themselves and will not waste time on AI output. At the same time, the system will still support students who are genuinely interested in learning.
Students who use AI to do their homework are not interested in such feedback anyway, and they will also avoid in-person meetings because it will naturally reveal that they are unprepared and did not do the homework themselves, which is humiliating.
To make this system work, we should present the in-person meetings as collaborative 'working sessions' or group office hours. Encouraging students to sign up in pairs or small groups also reduces individual anxiety.
The main channels for grading remain exams and oral presentations.
Committed to learning and going to jump ahead here to say if smart glasses (without cameras or in time with) become the norm and Ai is always right how many will not take the time to learn the information is always right in front of them in their view.
Myself as a student i take my class notes and the subject matter the teacher provides and have Ai create multiple choice quizzes. It's a much quicker way to learn tho not as quickly as AI glasses I mentioned.
Explaining something to an audience, even with notes, is hard to do without actually understanding what you are saying. It’s kind of like a rubber duck.
It's crazy that we went from "we found a significant number of students used AI in our course and gave them a 0" to "yeah they all use it we can't stop it". Basically just accepting widespread cheating, normalizing and accepting the complete abandonment of the first lesson in ethics we are taught and held to in academia.
I know this is absolutely impossible because colleges and universities have headed in completely the opposite direction, but the solution: Literally just expel them. Just like they did when people cheated when I was in college. Don't make it a small punishment so that they turn it into a risk/reward assessment. Colleges are overpopulated anyway; just trim the fat of students who shouldn't be there. Do a follow up oral exam interview about their project and if they can't explain what someone like me spent 30+ hours building and can speak about for 4 hours if asked, then expel them. They can make your coffee at the nearby Starbucks, who cares.
If you don't do this then understand that you are telling your students not using AI that they will need to begin cheating like the others to be able to keep up with the changes. OP mentioned they massively increased the scope of their assignments, which means OP is forcing their students to use AI now.
I'm taking a year-long sabbatical to study a language at university right now and they're changing their assignment processes to curtail AI use. We have two versions of written assignments - the first to be written by hand in class with just a Swedish-Swedish word book available, and the second version to be typed at home implementing the professor's corrections from v1.
At least in this course students seem to be taking their strict 'no AI' policy quite seriously. Then again this is also the kind of course you do when you want to actually learn the material. It provides a qualification for further higher education in Swedish, so if we don't legitimately learn the stuff there's just no point as we won't manage in future courses.
Recently it has been reported that many Chinese students in Moscow are no longer learning Russian during their years at the university (to the dismay of Russian officials who hoped this would be a form of soft power): even for in-person lectures their Bluetooth earbuds will just translate the lecturer's Russian into Chinese in real time. So, perhaps people around the world will start feeling that, thanks to technology, exams testing language skills are a mere formality and can be safely cheated on.
Russia is a bad example due to its current international reputation. On your second point, would an example of this be an oral exam where the student has tiny hidden earbuds? Outrageous but you're probably right it's going to get worse.
Actually, language is no longer the barrier it once was, it wouldn’t surprise me if some universities start opening enrollment up beyond the native language of the university accordingly.
There is indeed no point in grading home assignments anymore, but why not randomly ask a student at the start of each class to present their solution on the board?
And do a final exam on oen and paper and nothing else, and an exam that relies more on thinking than rote memorization of formulas (or print the relevant parts of the course on the exam document itself).
> why not randomly ask a student at the start of each class to present their solution on the board
This makes the class really awful. I had professors with schemes like this, maybe calling on someone's name randomly 1-2 times in class to answer a question they'd just asked, just to ensure everyone is scared into paying attention.
there probably is no time for 100 students to each do that in a university mathematics lecture, but there are probably ways to make sure students are still learning by restructuring the university into mandatory recitation and labs or something
there aren't 100 university lectures a semester, so there is a small chance you are picked. meaning, good odds to not give a shit or good incentive to not go to lecture if you are behind. also, wasting 100 students' time to watch someone struggle through one gotcha, is more of a punishment for everyone, not something to make students learn.
That sounds pretty frustrating. I wonder if it wouldn't be a good approach to, say:
1. Make the exam the only thing that determines the grade (or technically, proves that the student learned enough);
2. Give out test assignments students can do, but don't have to;
3. Offer students for the professor to grade their work if they want to get some feedback on how they're doing.
Would that lead to less time wasted by tutors grading AI generated answers? From where I stand, students should be allowed to prepare for exams any way they like, with or without their professor's help. If they managed to learn, they pass.
According to the article, #1 leads to "failure and drop-out rates of 50 to 80%". #2 and #3 might work for organized and motivated students, but graded assignments from other classes will be prioritized by most.
Given the education literally does not matter anymore for white collar work, I'm inclined to agree with you.
I'd rather a MUCH larger amount of professors and teachers come to class to find their classes deserted (by their own hand), and indeed, their own justification for existing gone. The world would be a lot better off. Gatekeepers are anti-human.
Then their parents come in and complain about where the $30k that semester went. This is a consequence of the exponential rise in tuition. You're now in lawsuit territory if you don't deliver the goods.
When I was in engineering school ~20 years ago, some lecturers openly said that they gave the assignments just enough credit to be worth doing. Something like 10-15% of your final grade would be from assignments, the rest from the exam. So we didn't blow off the assignments to go drinking.
Of course, this a lot easier for some subjects than others. Subjects like Film Studies relied on exams much less, and assignments much more.
I took some classes like this and they were great. I felt they were much fairer than classes where mere diligence/compliance could masquerade as competency by way of final grades padded by homework scores.
They were disliked by many other students, though, and some people really think university grades should be about things like diligence, compliance, time management, etc., rather than pure subject mastery.
1. is not a terrible idea, if structured correctly, but it doesn't adequately incentive the learning process, which requires continuous practice. For 1, I'd say put 80% of the grade in three to four exams taken over the semester, and allow one midterm exam grade to be dropped or replaced by the final exam grade. The remaining 20% of the grade is to incentivize practicing problems: homework graded based on completeness not correctness, short in-class quizzes randomly sampling a homework problem to incentivize students to legitimately complete the homework that are graded for correctness, and student participation in classtime activities.
How about cutting out the open book so that the students who don't study and to their homework actually fail? When I was in school many years ago "open book" was only used for special ed students and others who couldn't carry a regular academic workload. Homework was assigned, but our grades were determined solely by testing our mastery of the material. If you could master the material without doing the homework or showing up for class on non-testing days, more power to you. College has become more like kindergarten with all of the hand-holding.
When I was in college, whether or not an exam was open book seemed orthogonal to the difficulty of the exam as well as the class's median uncurved score. Allowing the textbook or notes lets you write an exam that might otherwise be unreasonable.
In the classes I took with open note exams where I did well, I barely ended up consulting my notes anyway. The students who didn't study could flip through the textbooks all they wanted, but they lost so much time doing it that they couldn't complete all of the problems on the exam. For the students who studied well, the textbook was way easier to jump around in because they knew it well and knew what they were looking for.
Am exam that is designed to be closed book/closed note where a specific student is allowed some amount of notes as a disability accommodation is just different from one that is designed to be open note/open book from the start. The latter, testing the same material, is generally longer or harder to make better use of the format.
Open book exams IME are usually more difficult, since the exams can be crafted on the assumption that you don't need to dumb anything down since students can look up certain things.
We had a formula "cheat sheet" on every exam paper, but if you don't know when or why you're supposed to use a formula, it might as well not have been in the cheatsheet at all
Open book need not be easier, it entirely depends on the exam.
This was too long ago for me to remember the details of but, just making an example here, we might have had a course on homogeneization of elliptic problems using a model PDE with a div(A grad u) term throughout, and then the exam was about a PDE with a curl(A curl u) and some other added terms.
Notes carry the method and theoretical tools but you then still need to apply them to a new problem.
So, if anything, our open book exams were harder and a better test of understanding and mastery. You couldn’t rote your way into a good grade.
Put a mixture of questions in that are impossible to answer but don't tell them, and as usual give them marks for explaining their working. Tell them you've devised the questions so some are "AI resistant" but don't tell them how. They'll never know if the AI has tripped up and hallucinated or given them the right answer. Only those that really understand what's going on will set out some working that shows they were on the right route and then got stuck where they were supposed to.
That will either lead to students becoming frustrated after spending inordinate amounts of time on the unsolvable problem or students giving up too early on the solvable problems.
I've had the same experience, and likewise I have no idea what to do.
Or rather, I do have ideas, but they do require substantial restructuring. E.g.:
1) Strict, in-person (no exceptions), electronics-free exams with written and oral portion.
2) Transition to guided independent learning: students are given written materials and recommended exercises to study. They are shown how to use AI assistants productively. Instead of lectures, there are scheduled discussions, where students can ask questions and listen to additional explanations from the lecturer. They can also seek feedback for completed exercises. However, none of this contributes to grades in any way.
With such an incentive realignment, I think it's possible that higher education could be salvaged. But... apart from general institutional inertia, there is also the political angle standing in the way. Everybody needs a degree in the "developed economy", and the above changes are basically the opposite of the nobody left behind policy that enables half the population to attain one...
This is a nice idea, but if you follow the Carnegie model of the number of hours per week it takes to learn a subject per credit hour, which is estimated at something between 6-9 hours total including classtime and out-of-class study per week for a normal-semester 3 credit course, then you can't meet the amount of time students need to be exposed to lecture (explanations, demonstrations, derivations), to do the background readings which provide necessary context that can't always be adequately covered in lecture, and to practice the concepts and tools on their own.
For me it's about time management, not about getting math help. I am consistent at attending classes because it's mandatory, but not consistent at finding time to practice math every day. If I could practice during classes, I think that would help me.
That’s an entirely different problem, and a hard one to solve. I would still suggest looking at the tutoring center or groups since there is time accountability there. If you don’t show up it is more consequential than skipping class from a personal point of view.
Dude, hop on the Gemini student plan and try Gemini Notebook and Gemini Live. Live is a personal TA, Notebook literally podcasts the readings and what you need to learn
That's a terrible model though. If someone is paying to be taught and you just hand them video lectures... We already have video lectures at home. Why would they pay you? What product are you meaningfully offering in that world?
Tutoring, office hours, teaching assistants. One of the best classes I took was a "studio style" class where we would work on a project and occasionally the teacher or a TA would interrupt and give a brief lecture or demonstration based on a "teachable moment" that had occurred.
You are selling a diploma. Some universities also sell some degree of attestation to the high resilience of their graduates. And of course the space to network. Good teaching is not on the list in my experience.
In general university lectures are horrible (of course there are exceptions). The lecturures oftentimes are required to teach as a condition of research funds, so for many it's not a priority anyways.
On top of that most couldn't care less if their teaching style is didactically sound at all.
?? The model is to flip the roles, not to eliminate in-person teaching.
The product is in-class homework and at home "lectures". I'm not saying this is appropriate for everything, but seems it would be useful for mathematics at some level.
> asking me during the test to explain what LLM output meant.
Perhaps (sarcastically) this is the new approach, because this is now the point where they’re forced to struggle and therefore actually learn. An exam every other day, let them ask the questions.
Not sarcastically, this is the flipped classroom approach. Read the examples and watch the lecture on your own time, then come and do the exercises with a teacher available to explain when you realize you didn’t understand anything.
Just don't make it open note then? Or you provide the only notes or cheat sheet available to them at the start of each test. That's what some math tests I've had had done before and I believe the SAT also does the same thing with formulas in the back of the test to reference.
> The university insists that homework can't be optional, but it lost its meaning. I've tried to explain to my students that it's in their interest to think about home assignments on their own, but the majority will obviously use gen AI, and it's such a waste of time to check and grade LLM output.
Can't you just not grade it at all? If the student has done their homework, just consider it passed. And then don't count it at all for the final grade.
My favorite math class I ever took had optional homework, but every exam problem was taken directly from the homework. So if you did all the homework the exams were very easy.
I imagine this would work very well in the current period since there’s no reason to use LLMs for this homework. Of course even back then many students didn’t do it and got low grades, but you have to be responsible at some point.
I agree and that's my idea as well, exams are open book and extremely close to assignments. My issues are with the first year undergraduates, who are not mature enough to do homework that is optional, and when it's not optional, they'll do it with LLMs and call it a day.
Grade homework for completion, have a weekly in-class quiz that is basically a few problems that are near to identical to the homework. You lose a bit of class time to the quiz, but it creates a strong incentive to complete the homework, and also eases in the midterm/final for students instead of going nothing nothing - boom here's 30%+ of your grade.
Why not just make the assignments "reading" based and set an expectation of randomized quizzes that take 5 minutes at the start of class to fill out to get the homework credit? When I say randomized, like multiple versions. You can use LLMs to churn out PDFs to a tailored format and utilize a pool of questions.
It seems to me that the only realistic option is to have more exams, closed book, and then use gen ai yourself to help reduce the workload of grading. Possibly with some kind of curve assigning more weight later in the semester.
Hmm people can now do a lot more than they were supposed to be able to without LLMs - shouldn’t we lean into that?
Force people to go all in with “impossible” tasks and see how they do with the power of LLMs. Then spend some significant time teaching people _with_ that solution in place what was actually built and what would have been a better architecture? A … software archeology session if you will.
That’s what most of what people do would be anyway - building PoC fast and then figuring out how to scale them?
> Many of them just dumped the document into an LLM, asked it to produce some "exam guide", and showed up with that thing printed out, asking me during the test to explain what LLM output meant.
That student presumably performed poorly on the test. Your approach is working as intended here, isn't it?
Also, is the AI-generated exam guide all that different to a human-written guide they could have downloaded a decade ago?
> I am teaching some math courses, and I see how LLMs disrupted all standard approaches, and I don't know what to do.
I wonder what the financial cost of AI will be to teaching? Does teaching get more expensive? More things need to be supervised, more things need to be presentations or in person?
> I've tried to explain to my students that it's in their interest to think about home assignments on their own, but the majority will obviously use gen AI, and it's such a waste of time to check and grade LLM output.
I think the problem is the students also take ~4-5 subjects per semester, and each of them may be just as demanding. High school is even more spread out with 6-8 subjects per semester. There's only a certain number of hours during the day for study and homework. For many, math homework ends up being the most time-intensive. Also these exams determine their GPA which in turn plays in part in determining their standing for the next few decades of their lives.
I've always wanted to find out if this would work:
If I knew most of my students were using LLMs I would encourage the rest of them to use them too. I would then change the marking criteria such that if they get a single point wrong that's 20% off the entire grade. 5 mistakes gets you a big fat zero.
LLMs are great but they make mistakes. At this point your students would have had to spend so long checking, re-checking and triple checking the LLM's output that they will have accidentally learn what they need to. They will likely need to cross-reference multiple LLMs and at least have read their output which is likely an upgrade on today.
Or some variation of the above, I'd be interested to hear your thoughts.
This sounds like it would come at the cost of the honest students who don't want to use LLM's and train their own understanding.
Reviewing is just not the same as working through a problem yourself. You often can take shortcuts when checking answers for correctness, which generation (with mind or LLM) of the solution can't take. This isn't limited to Math but equally true for programming and many other skills.
I'd push back slightly on this: I spend more of my time reading / reviewing code than writing it yet I still find reading / reviewing code much more mentally demanding. I don't think I'm alone in that.
The people that do their own work would be at a huge advantage as the LLM(s) can work as reviewers instead of doing the work so there is more chance of catching a mistake before you lose 20%.
On top of that there will be some issues where the LLM is just plain wrong. Being able to work out when that is the case is a hugely useful skill in 2026. It is also something the people who blindly feed their work into an LLM and print its answer without reading it will be unable to discover. Much to their own detriment.
> I spend more of my time reading / reviewing code than writing it yet I still find reading / reviewing code much more mentally demanding. I don't think I'm alone in that.
That's because we write much smaller quantities of code than we read. Not that LoC is a useful metric, but I'd be surprised if I write more than a few hundred lines a code a day in a legacy codebase. Most of the time is spent planning and deliberating over what to write, and perhaps revising code a few times. Meanwhile, understanding what a snippet of code does can easily spiral into having you read thousands of lines of code across several modules.
But that's for work. writing is a stronger learning tool than reading, and we needed to write tens of thousands of lines of code before we got into the door and started reading more than we wrote. It's still preferable for students to do the same and write as much for their homework as possible.
Explaining that paradigm shift from learning to working isn't trivial, and is often poorly done. But it's needed if we ever want people to frame these tools as productivity boosters, and not shortcuts through tedium. Learning by its nature involves some tedium.
I'm not trying to suggest that reviewing (code) can't feel (more) mentally demanding. The thing with at least reviews is that if you have worked through the problem yourself, you will also hit certain walls where stuff didn't work or wasn't as beautiful as you wanted (ie trial and error), and you have to activate your brain to think of other solutions, ie really think a problem through multiple times with active feedback. As a reviewer you don't get the active feedback usually.
Now this i think applies in lesser degree to the comment I was replying to. With maths, and some comp sci problems, you often can check an answer much faster than actually solving it, eg by using simple substitution for simple math problems. That is a useful skill, but a different one from solving the problem yourself. It employs different types of skills.
I think you might be letting perfect be the enemy of good here. Right now some students are using an LLM to do all of their work. I don't suggest for a moment my method is perfect but I do claim it could be better than the status quo (or at least warrants further investigation in my opinion).
What you would grade with that approach is how well your students can operate a LLM as well as the failure rate of whatever LLM infrastructure the individual student is using. This is usually not the goal of your educational format and hence not what you would grade for - apart from some (meta-)skill courses offered by uni libraries and the like. Those are mostly ungraded, though.
Anyone that knows the subject matter well enough will spot the LLMs mistakes and correct them. If you don't know the subject matter well enough to spot a mistake then you don't know the subject matter well enough.
It must be tiresome to be a student these days… with each teacher having different pet theories about AI use and sometimes trying to “trick” students who use AI, or implement some draconian rules about the way homework or exams must be done. It’s tiresome for you, but just imagine getting varying lectures on AI use as you go from class to class as a student. Must be hell
> such a waste of time to check and grade LLM output
Small nitpick: you teach math, why don't you use an automated grader?
I agree homework should be optional and your college is wrong. Even without AI, for some students it's easy to cheat and maybe for some students it's a waste of time. Maybe you can assign it a very low score, like 5% of the final grade, and/or award the grade based on attempting rather than getting the right answer.
It would be a waste of time. LLMs have become better but not perfect. Personal experience; it's easier/faster to grade for me than have the LLMs grade and cross-check each one.
Good homework feedback for a proof isn't "Did you get this right?", it's "Why did you not get this right?" or "How can we make your correct solution better?", neither of which fall out of a Lean translation-and-verification pass.
1. We don't exactly do a great job showing upcoming adults their options and paths in life to begin with.
2. We're also actively giving them less paths in life to begin with. How many high school grads @ 18 will find any job whatsoever without additional school/training, and how else will they afford housing with zero income? The military is the only consistent employer at that age, and the only one who'd provide shelter with the deal.
3. On top of all that, it's in university's best interests to keep 1) and 2) the way they are. Their incentives start at recruitment, and end at enrollment. Past keeping them in college for 4-6 years, they do not care about performance, post-grad job placement, nor societal wellbeing. And part of that is due to a lack of pressure in the recruitment stage to make universities focus on such metrics. Turns out a fancy new gym is a cheaper investment.
By the time you are 18 and going to college it’s time to reap your lifetime of prep, you’re about to spend a ton of money to specialize. What does it mean to show you all the doors and motivations of life when you’re basically going “all in” on a domain?
Hopefully it doesn’t mean extending high school two years into college, still prepping to make a bet not yet made.
>What does it mean to show you all the doors and motivations of life when you’re basically going “all in” on a domain?
I don't know about you, but I sure didn't have my life set at 18.I could have easily been a mechanical or electrical engineer if my alma mater rejected me. And that was more focused than most students.
It pretty much means what it says on the tin. Before hoisting an 18yo with the sudden responsibility of an adult, show them what lies beyond their GED. That's not just limited to college; there's many more roads out there and seeing what options motivate you can let you try to specialize while you're still in your prep phase.
That's precisely why some student go into a major undeclared or even take a gap year after graduating. No one gave them that exploratory transition time, so some make it for themselves.
That lifetime of prep is completely disconnected from actual job & life experience. The kids are shown all the doors but have to choose without first opening any. Aside from the occasional internship they might have done by that point or a glimpse into what their parents are doing, which is very hit or miss.
As long as students ask the teacher what they're going to need understanding of maths for they are not prepared adequately.
UK (and the wider Anglo-American system) is different from the continent, where in many countries the MA is the basic degree -- a mere BA will get you nowhere -- and this easily takes 5-6 years to achieve.
I have been feeling for ages like I am too old for the tech industry.
Reading this has made me understand that the recruitment industry will soon have to adopt the default position that anyone who graduated after mid 2025 is likely misrepresenting the depth of their understanding; people who graduated before that are the equivalent of low-background steel, until our processes catch up.
Seems the core problem is that many/most students aren't there to learn the material, just to get grades. The class is a complex mechanism to force them to learn material against their will in a validation arms race. The teacher's job becomes something like a military commander. Seems depressing.
This is the way it's always been, sadly. Before AI submissions I had to regularly look for where my materials were posted on sites like Course Hero. Students copy-pasted from Wiki, articles, blogs, etc without attribution. Before that, they were buying or finding solutions books and copying out of there, or hiring people to do their homework/write papers/sit exams for them.
The primary difference now is that the both costs of cheating and ability to verify cheating on the instructor side have decreased dramatically, in tandem. There's also been a kind of mass effect, a cultural change in the attitude of students, where the sheer prevalence of cheating among their peers has made them less likely to feel fear, shame, or guilt over cheating. Students in the past who were caught were likely to confess immediately. Students now are increasingly belligerent and more likely to deny cheating, not just because they believe the professor can't 100% prove it, but also because they believe being caught out for cheating when most of their peers are getting away with it is unfair.
This is because we shield children too much from reality. If they had a fire under their ass that made them realize if they don't study well, they will forever be locked into flipping burgers or being homeless, most of them would move their assess.
Don't do anything productive for the society but still get to live a decent life on benefits, job security etc combined with addictive social media have ruined most kids will to study.
And the schools are even worse. When I was in school, teachers encouraged learning things; whether for school or outside, with teacher or self help. Everything was fair game and supported. Debate was normal.
Nowadays the teachers in schools are like a cult. Kids are tuned to reject anything from outside; anything that contradicts teacher is a no go zone. Anything that's "too advanced" compared to what teacher is doing is forbidden. Kids are scared of learning by themselves because the teachers punish it. What do you think these kids will do when they grow up?
> If they had a fire under their ass that made them realize if they don't study well, they will forever be locked into flipping burgers or being homeless, most of them would move their assess.
That strikes me as backward. The higher the expected monetary value of good grades, the more students will prioritise grades over deep learning.
> This is because we shield children too much from reality. If they had a fire under their ass that made them realize if they don't study well, they will forever be locked into flipping burgers or being homeless, most of them would move their assess.
This seems pretty inaccurate to me at just about every turn.
The fire under their asses is for grades and a degree because they've erroneously been told they need one or else they'll be flipping burgers.
It's such a cliche thing to say but cheating at school cheats yourself of the education you could be getting but, since that's not what most people are there for, they cheat. They are there because they've been told that they need a degree to make a decent living and the degree doesn't technically require learning.
The least paragraph... has always been the case. My parents told me stories about teachers denying facts not to lose authority. I distinctly recall my teacher pretending that peripheral vision doesn't exist not to lose authority.
The easiest path is to allow students to cheat on homework and watch them fail in the non-coursework-based exam. Your classes, then, abandon to the un-motivated; with AI enlarging the educational gap.
If you’re forced to attain a pass rate, and can’t watch so many fail, then the certificate becomes worthless in the workplace. Or just offer paid retakes and watch AI make you money.
If you insist on persisting with coursework then a short but heavily-weighted oral part seems to be the only answer, something that will diminish teaching time as a downside, as the article and comments about the flipped classroom touched on.
Isn't that the point? To learn? If you're setting a goal that the students must pass, who cares if they get the information from the professor or they teach themselves?
I don't think "learning the material by other means" is the exam being "gamed".
I think the point is most students are cheating and not learning. Where they get the information does seem to matter. Adding friction was needed for most.
if they use ai to "cheat" the exam by having the ai produce material to learn off, then they literally have just replicated a paid tutor (for cheaper).
Why is that not desirable? As long as the exam does not allow ai usage during it, i don't see the problem.
You're giving a charitable interpretation which is not warranted in reality. Scores are down across the board and even ivy leagues need to re-think its long held honor code.
I don't think it's shocking to realize that most 20 year olds (especially those growing up and seeing that success is not tied to intelligence nor hard work at all)are taking advantage of a convenient tool to cheat when given the chance.
The point is that they're not using AI to learn anything, they're using them to pass assignments without even giving it a once over. They're basically blindly copy/pasting whatever the assignment is and then copy/pasting the answer.
Maybe the Germans were right? One in-person test, preferably an interview, at the end of the semester should be enough to motivate the smarter part of the bunch to make sure they understand the issues at hand. The rest is chaff anyway. Don't optimize for chaff.
As was sort of offhandedly mentioned in the article, all pre-LLM research into what results in students actually learning the material (as a distinct thing from passing the class) has landed pretty solidly on the side of frequent, low-stakes exams having better outcomes than rare high-stakes exams. The best students learn the material regardless of how you evaluate them; some weaker students will benefit from more frequent nudges to keep up rather than trying to cram at the end, some students simply won't realize they're falling behind without regular feedback, and some students successfully cram at the end and pass the class without retaining much of the material. The final group often expresses a preference for the one-exam-at-the-end model despite being disserved by it if the goal is to learn and not merely pass.
Yeah and look at where this soft approach landed us. We have entitled crybabies voting for Hitler 2.0 because they can't stomach the uncomfortable feeling of seeing a diverse person in their pure white neighbourhoods. Children and young adults need to be hit with the big hard fist of reality repeatedly until they become good people. If some should fall through the cracks, they would not have made for good members of society anyway, so best they fall early and not drag the rest of us with them.
I had that a variation of that experience. This was mostly in the pre-AI days.
Homework assignments were mandatory but only counted towards exam admission not the final grade.
Much better for motivation.
When I was a student, nobody cared if you did your home work. The teacher would go through the home work in the next class, and if you didn't do it, you would just lose the opportunity.
Then there was the written exam at the end of each course, which constituted 100% of the grade. The only reason that got abolished was because it made too many students fail, and the board hates that. So that became a race to the bottom, and apparently, we can still go lower.
It's reaching levels where most uni diplomas don't matter any longer. The only thing that will then matter is your network. That's the end of social mobility.
"Don't optimize for chaff." is being chaff a permanent state of a child or person?
I think I understand what you mean though, and I kind of agree. I just don't like the use of the word chaff, like something you would discard permanently.
I'm pretty sure I was "chaff" at some point (maybe multiple) in my life growing up.
Funny to see this comment, that's exactly how studying math in Cologne was. On the very first day one of our profs said:
"This is math, you can find all solutions to the course problems in old material since I've been teaching here for 20 years, but know this you're cheating yourself and at the end of the semester you're taking a three hour test on pen and paper so good luck"
I see zero reason why AI should impact teaching. You can grade exercises as information to the students as to how well they're doing, but simply have an exam at the end of the course. Needlessly to say like 60-70% flunked through Analysis I and Linear Algebra. But they're adults and they're there voluntarily, so why is this an issue
Honest question: doesn't this highlight how mundane and rote teaching has become? It's been distilled to the point that the lowest of the lowest common denominators can pass. The kids going through it are forced to learn things they will never use, unless it's going to be used in the more advanced class of the thing they'll never use until they graduate.
My hope is that this is a wake up call to teachers / professors to find a better way to evaluate students. To determine who has actually come away with a real understanding, vs those who were running around copying the assignments off of their peers anyway. These same people are now just skipping the peers and going straight to the all knowing oracle. It's the same as it ever was, degree mills.
> doesn't this highlight how mundane and rote teaching has become
The example questions given in the article do not assess wrote learning.
> It's been distilled to the point that the lowest of the lowest common denominators can pass.
Have pass rates been increasing? Unless I missed it, the article doesn't say so.
> forced to learn things they will never use
A computer science degree is not a coding academy, it's a basic grounding in an area of study. A computer science graduate should have some understanding of theoretical computer science and of computer architecture, say, even if they're unlikely to apply these topics directly in their careers.
> My hope is that this is a wake up call to teachers / professors to find a better way to evaluate students. To determine who has actually come away with a real understanding, vs those who were running around copying the assignments off of their peers anyway.
Academics are already aware of the importance of fair assessment, but it isn't easy, especially with LLMs in the mix.
> These same people are now just skipping the peers and going straight to the all knowing oracle. It's the same as it ever was, degree mills.
Students that cheat, and degree mills, are two different things.
It's going to be a rough decade but I think we're going to come out a lot better in the long run. Education, especially in the US, has been really stagnant for a long time, and primary and secondary education has been more and more obviously just a mill to turn out middle-wage earners to dutifully pay taxes and buy products and not try to think to hard about things.
Now that we have systems that can pass for students at this baseline level, educational systems are facing a reckoning and we can, hopefully, move past cookie-cutter education solutions and recognize the need for more hands-on, engaged education. Will it be an easy transition? No. Will all schools adapt successfully? Also, no.
But will we finally have a path away from standardized testing, middle-of-the-road education? I am hopefully optimistic. We'll rediscover the need for human growth and development in our schools. It's either that or we lose. I'll choose the optimistic route and join and support those who believe the same.
> But will we finally have a path away from standardized testing, middle-of-the-road education? I am hopefully optimistic. We'll rediscover the need for human growth and development in our schools.
I worry that the availability of this path will be extremely unevenly distributed, amplifying existing educational gaps along class, race, and language-proficiency lines.
That might be better than a forced average denominator, but the social effects of further amplifying those gaps are pretty severe and don’t take that long to happen. Combine that with the fact that education isn’t like new miracle drugs, where improvements inevitably tend to become more and more widely available to different groups, and I’m not as optimistic about the future.
Otoh, ai is much, much better at bridging language gaps than humans and human institutions.
The real question for education is whether we can get 'tutor-quality' education from ai interactions. One on one education is leagues better than classroom education; it's the one-true-result in education research. There's a massive opportunity here to improve outcomes, and in a generally available way. You can't clone a good teacher, but you can give everyone an ai tutor.
+1. I’m doing a masters now, and typically I have lectures/slides/textbook open in one tab and Claude in another. When I don’t understand something - which is frequent - I discuss it with Claude until I do. Having an always-on, infinitely patient, phenomenally knowledgeable tutor is AMAZING. Schools could mitigate equity issues by providing their own portals for this (with some guardrails to prevent abuse obviously).
Of course the onus is on me to do this at all, I don’t need to in order to pass the class. I set my own standard for what it means to ACTUALLY understand anything rather than just say “oh yeah that makes sense” and move on.
> we can, hopefully, move past cookie-cutter education solutions and recognize the need for more hands-on, engaged education.
I'm not convinced children need "hands-on, engaged" education at the age of 10. At some point there's no shortcut to being socialized that patterns in numbers are interesting (which is one of the national goals of maths for 3rd graders in my country). 10 year olds won't find that stuff "engaging" on their own, they must be informed by authority figures that it's a thing worth paying attention to.
Too much of this "fuck the schools" mentality treats children as perfect little beings that must only be supported to grow into wonderful adults, when the reality is that socializing children is a brutal exercise in fitting them into norms.
They certainly can find things like patterns in numbers interesting. I just found a blog post I wrote when my daughter was a few weeks short of seven that says she found simple sequences interesting. "She was instantly fascinated, and thought that trying to figure out the rules of each sequence was a wonderful game. Whenever I ran out of ideas, she pestered me for more sequences."
You replaced the rote recitation of faceless homework with engagement from an authority. She was not interested in the sequence of numbers, but interested in your interest. She loves you, and is reflecting your engagement back at you in hopes of getting more validation. Children do that a lot.
You describe the homework as
> basically lots or practice in basic arithmetic
Which is absolutely what you need for these sequences to have any meaning. There is no "pure math" to have a game about, without the "pathetic" examples of Lucy's 5 apples and Tom's 2. If somebody hadn't (brutally) drilled the concept of numbers into her skull, the sequences of random symbols would not mean anything to her.
Engaging with your child, who has already been partially socialized, is not proof that this socialization wasn't necessary. For that, you would have to have locked your daughter in the basement for those 7 years, and then find that she was spontaneously able to do multiplication. That is not going to happen.
> No, you really, really should not be brutal to little children.
I hope you understand that _brutality_ as spiritual. It's the brutality of going against nature, of putting a monkey in a suit and locking it in an office for 8 hours a day to feed itself. You don't beat math into your children, but they sure as hell don't learn it from eating candy and watching Minecraft let's plays.
> they sure as hell don't learn it from eating candy and watching Minecraft let's plays.
I learned math from flash cards my mother was using for my brother. He got M&Ms for getting them right, so of course I wanted to learn and join in. Many years later, I graduated with a degree in math.
So you absolutely can learn math from eating candy. I did.
> You replaced the rote recitation of faceless homework with engagement from an authority. She was not interested in the sequence of numbers, but interested in your interest. S
She was quite capable of expressing what she was not interested in. You are just refusing to take experiences that do not match your preconceptions.
There is a huge different between engagement and "being informed by an authority figure" which implies they have no choice.
> She loves you, and is reflecting your engagement back at you in hopes of getting more validation. Children do that a lot.
Read the blog post. She demanded more of things she enjoyed rather than what I wanted her to do (finishing her home work). That is not seeking validation.
> If somebody hadn't (brutally) drilled the concept of numbers into her skull, the sequences of random symbols would not mean anything to her.
No, nothing brutal needed to drill the concept. You can teach very gently and make it fun.
> Engaging with your child, who has already been partially socialized, is not proof that this socialization wasn't necessary. For that, you would have to have locked your daughter in the basement for those 7 years, and then find that she was spontaneously able to do multiplication
Strawman. I did not say socialisation was not necessary, I said it did not have to be brutal.
> I hope you understand that _brutality_ as spiritual. It's the brutality of going against nature, of putting a monkey in a suit and locking it in an office for 8 hours a day to feed itself.
Who gets locked in an office?
> but they sure as hell don't learn it from eating candy and watching Minecraft let's plays.
That would be neglect. no one things complete neglect is bad.
> I hope you understand that _brutality_ as spiritual. It's the brutality of going against nature
Salman Khan of KhanAcademy many years ago suggested that study should be done at home and homework should be done in-person in class with the help of a teacher. That would solve a lot of the homework problems AI access is causing. But it would require extra effort on the part of the teachers.
I was at EduLearn (a major edtech conference) this year and talked to many professors exactly about this. A lot of them are pulling work back into the classroom, but keep running into the same problems:
- some skills only develop through the actual writing (similar to how you can't learn to code just by watching YouTube videos)
- in-class assignments are not always scalable (and students end up using AI anyway)
- students aren't motivated enough to participate actively
I think looking at the writing process is a more realistic approach, so I built Turingo (https://www.turingo.net). It replays how a Google Doc was written, highlights unusual edits (large pastes) and tracks how that text changed afterwards. Surprisingly, several professors reported that students often underestimate how much LLM-generated text is left in their final drafts, so just seeing the replay changes the conversation.
It's not meant to be a verdict machine like the existing AI detectors (another big problem in education), more of a starting point for the kind of conversations the author's TAs are having. In fact, we're also beta testing a feature that suggests a few questions for the professor to ask based on the replay.
And yes, it's not perfect and autotypers do exist, but they're dumber than you'd expect and nowhere close to mimicking a real human (I hope to address that soon).
It sad that it has come to this, but your approach is correct - it is the act of writing (code or prose) that is important, not the finished output itself.
Interestingly, this applies not just for learning, but also to a great extent in production/work contexts too.
> I now shifted to in-person interactions with a TA. After every assignment, each student needs to schedule a 15-minute meeting with a TA to answer a couple of questions in a live conversation
Isn't the answer the 'flipped classroom' - students are assigned reading/studying to do before a class and the class becomes more interactive, answering questions/discussing topics/solving problems etc depending on subject. Of course, this is more expensive than a hall with 400 people listening to a lecture with a couple of tests/essays.
This only works if you either have very intrinsically motivated students or you verify that really everyone has actually read the assigned materials before class. Otherwise, some people will turn up unprepared, you will have to explain stuff they should have gotten from the reading assignment, other students will be annoyed, students will stop preparing for class, vicious cycle.
And then you fail them. I suspect an institute genuinely empowered to do so, where the expectations were declared up front and followed through on consistently, would fare well, after the dust settled.
As far as I can tell (based on university and high school level instruction forums) institutions increasingly are not backing the teachers that want to let kids that have earned that F fail the students.
I've heard of places where the minimum points for an assignment is 50% even if you never turn it in which is insanity.
I don't fail them usually - I expose them; and once I asked a student to leave the room because they were blatantly unprepared despite me warning them before.
My main point is: flipped classroom works great in some
contexts but is no panacea and requires setting clear expectations.
Aside: as English is not my native language: "you fail them" can mean both "you give them a failing grade" and "you fail to provide the support they need", right?
> "you fail them" can mean both "you give them a failing grade" and "you fail to provide the support they need", right?
Yes, but as written the latter would not be a reasonable parse. For that sense: “you have failed them”, sure; “you are failing them”, sure; even “in acting so, you fail them”; but “and then you fail them” as an entire sentence would be very weird. The emphasis I placed on “fail” also doesn’t feel particularly compatible with that sense of the word.
As a former flipped classroom student I can attest to the effectiveness of "exposure". It requires some skill to pull off: the first time someone is exposed it should be very lenient and then get gradually harsher. The risk of embarrassment in front of the class can be a very strong motivator if utilized right.
I've been in uni classes which used this concept (about 10 years ago though). It also requires that content is given in the right size and complexity each week, otherwise there will also be people coming to class who did do their homework but didn't understand it properly and just like the people who didn't read it need full explanations.
I also felt having multiple explanations (in a regular class where you could also read the book in advance) from regular lecturers helped me in gaining understanding because having something explained in multiple ways made sure there would be at least one way, or a combination, which made it 'click'.
I always liked the regular lectures where you would be required to do some pre class reading and a smaller set of exercises the most, as you would be engaged in the class itself and would be getting repetition with different explanations.
Flipped classrooms are like, the quintessential siren song for teachers. EVERY teacher dreams of a day when they can focus on what they're best at (teaching deep topics, getting 'aha' moments, making connections, asking questions, and so on) and do less of what they dislike. But sadly this is just not how things work.
Flipped classrooms have incredibly weak causal evidence. However they are popular in spite of this - perhaps because of this if you're cynical - as the educational research field is pretty questionable in its standards and the profit motive is a strong thing lurking in the background. Wikipedia has a big warning on the article for a reason, but even then it has serious issues.
"Active learning" is a thing (insofar as it has meaning rather than be a useless buzzword catch-all) that does work. But the "flip" specifically does not, or rather, sometimes a mix of extra effort on behalf of the teacher and other loosely related effects occasionally can make a modest impact that makes up for its failings.
Most of the people pushing flipped classrooms are the same people who think that students learn best when they "discover" concepts on their own, maybe with a bit of guidance. This is the other, more serious, lie in education right now, akin to the whole idea that you didn't need to teach phonics to kids when they learn to read.
One of my colleagues tried it for a few terms. Unfortunately, it didn't work well. Most students don't care about learning, they are here for grades :(
I cannot shake the feeling that we are chasing the wrong goose.
The end goal of education is to help you understand how things work, with problem-solving being a means of developing and applying that understanding. LLMs shortcut the former and turn the latter into a pointless arms race that we cannot win.
I loved the example of increasing the complexity of learning projects, but I still feel like one question is missing: are people really trying to learn the same thing we were trying to teach them in the first place?
I have a fairly long experience of both teaching and grading, even though I stopped a couple of years ago. After a couple of years of trial-and-error, I think I found the "right" way:
* Alternate between theory and practice in the course of a single 2-hours slot. Theory is necessary, but the attention span of people is very limited.
* Grade either a homework assignment in the form of a real project with specifications. You give the assignment half-semester, so that students who start early can ask questions. Otherwise, grade a in-session exam, with every resource available, including the course and the whole Internet. However, you don't assess the knowledge, but if the student is able to apply their knowledge to the different tasks at hand.
These approaches are now completely moot in the age of AI.
I was surprised the first time a colleague asked me to do an oral exam. I thought it was a burden on people with poor social interaction skills. It went surprisingly well: you could see in a couple of minutes if the student had integrated the concepts of the course, even if they were shy. Now I wonder if it's the way to go. There's one caveat, though: it doesn't scale.
I was thinking that one of the great ironies of artificial intelligence is that it will make good teaching even more labour intensive. Those kids that can afford the one to one tuition required to push past our natural inclination to be lazy will learn things, while everyone else will succumb to outsourcing all their thinking.
There’s no way I’d have learnt everything I know now about software in today’s environment. So much of software development - debugging techniques, architectural decisions, structuring data etc is learned through trial and error. As the models get better is becoming easier and easier to just not look too closely at their output. I suspect it’s human nature.
The oral exams scale better than one might expect, at least if you're not just doing multiple choice exams and you have to actually grade the exam. The time spent on grading the exam could have also been spent on an oral exam... For bachelor level it's usually 15-20min (usually quicker for the well prepared students), and for master courses it is 30min.
Usually you know after a few minutes if it's going to be a fail, and then otherwise you only need to figure out where on the passing scale the student ends up.
That's the point. As a professor you're allowed to outsource the grading of written exams to you assistants (ongoing PhD teaching stuff). But you must do an oral exam personally.
> usually quicker for the well prepared students
That's true. More time is needed only in case the answer is missing and you have to check whether you hit accidentally the only poorly understood material piece or there are more troubles.
> Alternate between theory and practice in the course of a single 2-hours slot.
2 hours is a lot! In my universities, classes were either 50 minutes, or 70 minutes. I could handle 50 minute ones just fine, but would often have trouble with challenging courses that had 70 minutes. Particularly for stuff like math, I hadn't properly digested the material in the first half, and now he's already proving theorems with that material in the 2nd half. I can follow the logic, but the ideal is when you can digest the prior material before being exposed to theorems relying on them.
Even when alternating theory/application like you say, the amount of attention I'd give to the application in the 70 minute class was less than the 50 minute one. I could follow, but I wouldn't think critically.
The country of Argentina, despite its crippling debt, is able to provide free university education where the majority of classes for most majors are graded with an oral exam.
They've shown that it is entirely possible to scale a system like that, as long as the society sincerely values the role of an educator.
Oral exams don't scale at the beginning with hundreds of students per course, agreed. So make them in-person written exams, German universities managed that just fine a few decades ago. Oral exams were reserved to advance from Grundstudium (basic) to Hauptstudium (main).
In my very limited experience (I have German major degrees in Informatik and Philosophy, I was a TA for Logic in Philosophy) the more advanced the topic the thinner the attendance anyway.
20 years ago my department head was saying homework was pretty pointless because everyone was copying each other. They assigned a token percentage for homework so at least the students were submitting homework.
We still have developers because we can't outsource responsibility to AI. If shit breaks, we still need someone to fix it. Maybe you think that can also be automated. I have a counter-example. AI will rarely use the existing state when solving a bug. It will tack on it's own event queue, it will make up it's own messaging system, it is a cancerous growth. And this observation is based on both opus5.5 and gpt5.6-astra. The results are way past magical at this point, but you still have to weigh in pretty heavily for orderly code growth where solutions are succint, focused and preserve the existing structure. They are still noobs at fixing/adding stuff.
This makes me think of exams — I always did well on open-book exams, but I didn't do well on exams where I couldn't look at the book. Whether it's because of ADHD or just a lack of memorization ability.
Similarly, I did well in interviews where I could use the internet, but I didn't do well in interviews where I couldn't use the internet.
Everyone has a testing format they're good at. I'm curious about how this kind of thing is evaluated, and whether there are any papers related to this.
> I do not believe that suggesting using low-cost open-weights models would be a fair alternative. (...) We now mention the expectation to use a commercial subscription in our syllabus.
I think structuring a course to be taken using a frontier model subscription is outrageous. If you refine the assignments each semester to be only solvable by a frontier model, your students will always be exposed to the actual pricing of said models. It may be 20 USD/month for now (which is already too much to ask in a syllabus in my opinion), but what if prices increase; where do you draw the line?
What's next -- make the students pay for cloud credits in DevOps courses?
I agree with the need to increase the weight of oral examinations / interviews with TAs though.
If the current state of education is students having LLMs do their work and teachers grading assignments with LLMs, we might not need either participant soon. What is the value in doing things this way?
> Similarly, I always provided a safety net where students can make mistakes and resubmit a limited number of assignments to regain lost points (a core recommendation of specifications grading and grading for equity to focus on learning outcomes, not the process), but we felt that this process was abused with AI: first submit a generated assignment solution without thinking and only look at the issues raised in grading for a resubmission (the typical story of externalizing the cost of AI use).
Long before LLMs the couple of actually good teachers I had gave very little (if any) homework. Instead their lessons were more dense and exhausting, but once it was over it was over and we otherwise mostly had to prepare for exams.
The education systems were already hopeless before LLMs and now they got caught with their pants down. Of course some problems were unavoidable, but this overreliance on homework was always a problem in my eyes.
> For example, the evidence favors frequent low-stakes assessments with feedback (e.g., homework, quizzes) over few high-stakes ones (e.g., exams) – but AI is undermining practice in low-stakes settings and pushing us more toward exams.
Why would AI undermine quizzes? I took an intro to Japanese class where we would have a quiz at the beginning of each class which I quite enjoyed. I can’t see how AI would change the utility of that.
I teach mostly smaller courses but my approach to grading has been the same for years: A combination of a semester-long group project (ending in a paper, a prototype and a group presentation) with individual oral exams about aspects of the project and of the lecture. The lecture is flanked by optional TA sessions as well as access to labs which, for most hours of the day, have some experienced/teaching people being around (this results in a lot of ad-hoc learning situations and a general diffusion of knowledge and practices).
This gives me/us a lot of angles to understand what progress has been made throughout the semester as well as to consider and react to individual differences between students. In my book, it’s also resistant to faking it convincingly throughout all modalities. However, scaling that approach to big entry level courses would require more money for teaching than any research group in Europe I’ve ever seen has available.
I dunno, if I had to teach students now I think I’d just force them to stay in the lecture hall without internet for the hour I need them to work on their assignments that week. I just can’t see any other reasonable way to do this. Everyone leaves their phones at the door in some designated locker or something.
— watches, calculators, discrete technological devices. I've seen kids encode text as patterns of dashes on their desks, sharing the cipher solution with their desk mates in other periods.
The problem isn't skirting the effort. The problem is systemic. Why does the system reward taking shortcuts? I think it's because most areas are focused on short horizon tasks, min-maxing effort:output. It's not a problem with those technical fields. It's a problem with the current crop of the C-Suite across the entire occupational plane. They're short sighted.
> Minor note: We observed that some students used AI during live discussions over Zoom (e.g. Cluely)
I frequently see this in remote interviews for senior developers. These are conversational interviews, not coding tests, but some candidates seem incapable of having a conversation without AI.
I can't imagine how these candidates would be able to meaningfully contribute to a whiteboard design discussion with real people.
For a previous client, we resorted to a simple in-person pen-and-paper screening questionnaire with 5 very basic questions. Many candidates declined to attend, and most of those that attended failed dismally.
It's worth noting that the he's not afraid to us LLMs when it cuts the teacher's workload. The Slack bots are all automated.
I generally think this is okay because the students are getting more feedback, but some students are starting to call this out as hypocrisy. At Dartmouth, there's a debate running because the Provost was using LLMs to improve his writing. He's not in a class. He's not breaking any official rules. But somehow people are feeling like he's cheating.
I see reasoning models being used to change the way to draft assignments and formulate teaching. Accelerating the learning gradient for students now requires a better understanding of how they learn and working with their motivators.
E.g. if students learn better through gameplay - can assignments become like game levels?
Rather than building for verifiable outputs - build assignments that gauge process and help elicit what they have learnt not what they have produced.
AI is effectively a personal tutor. Personalized tutoring is the best possible way to learn!
The problem is that the student is in charge of the tutor and often tells them to just do their homework for them and not check whether they have learned anything.
LLM providers should formalize a tutoring mode. We should have prompts available to create this mode. This should be the default way students use LLMs.
LLMs may not be good enough to do this now, but I think there is already enough raw intelligence and its a matter of training models to be good tutors, to be experts and specific subjects, and developing software systems for tutoring.
In this model there are few lectures. This is more similar to Montessori but with on demand tutoring. The teacher provides objectives and materials for the students and tutors. The tutoring process can be done during class for teachers to ensure it is working well. There can be larger blocks of office hours for students to do their tutoring during.
As AI gets more intelligent, it will also be capable of doing all the grading. This includes oral exams. Teachers need to proctor the exams in person.
Anyone will be able to tutor themselves on any subject. Proctored examinations to certify knowledge do not need to be very expensive. The value of the school environment will not be to transfer knowledge from a teacher but to have peers working on the same thing and teachers and an environment invested in training the next generation.
Incredibly insightful and makes me feel bad for those who have to put a lot more work to just maintain basic standards (Teachers, TAs etc.).
It was also interesting to see that everyone is now forced to use LLMs to do the assignments. I wonder if students are using any kind of subsidised access to LLMs here.
If this is the case, then this education is largely done.
This means that education needs to have a high teacher to student ratio. This is not feasible in any society currently on this planet, because we simply cannot afford to have that many teachers per student.
Think of education as a wishing well. There is a wishing well for math, law, dentistry, baking, and so on.
Once you have a wishing well - i.e. you own an eduction in baking or dentistry or law or math or whatever - you can ask it anything and it will grant you your wish. If others want something from your well, they have to come and ask you. What you give them has your name attached to it. So, your reputation rides on the quality of what your wishing well produces.
Prior to AI, universities chose which wishing well they would specialize in building and offered to share that knowledge with students.
In the age of AI, the creation of personal wishing wells has become trivial but universities still think they are in the business of teaching people how to create wishing wells.
Ultimately, the only truth left is for owners of the wishing well to start offering the product of their wishing well to the market and attach their name to it. If there is a market for their product, great. If not, go make another wishing well.
Education used to be pets. Now, it is livestock. Universities are not willing to accept this shift in mindset.
> more importantly I do think that students need to learn responsible use of these technologies anyway.
But you're not doing that, you've just given up and allowed any use!
> How to use AI coding agents effectively is not a learning goal
Exactly
> violate evidence-based best pedagogy practices, and I made them anyway.
Of course you have, when have educators cared much about best practices
> We have since built infrastructure to autograde code and written reports of homework with LLMs (institutionally approved LLMs)
Further reducing the point of your own existence
> Debriefing after such a failure is a good learning opportunity to talk about automation bias and different forms of human oversight (and how this is really hard in practice).
It would be if there were any learning, but since the incentives are for more AI use and less learning, there will be no application of the supposed learning, so you'll continue to have those 80% fails
that just means home work needs to become class work and in session. Activate a network jammer(or just collect their phones) give the class couple hours of isolated time to work on the low stakes problems relevant to the session to gather calibration data points. This also means you will need personalized pace of studying/teaching. Probably studying at home becomes the norm with personalized AI tutoring and class work becomes the calibration point to tune the next lessons and socialize with other students.
I find it offensive that your blog is named after thelastpsychiatrist when the writing is not even close in quality. Come on man, I expected you to start swinging. Are you building an agent tunnel? Get outta here.
I don't like it. Now, the TAs have high cognitive load. Earlier, you only need to know the material of the course. But now, you're forced to navigate AI output. And you got to keep with all new model developments.
Make all reports and assignments open source and public. Their credibility as students and professionals will be on the line. It will be obvious who puts in the effort and who doesn't
building a community around open source is hard. building one that analyses, comments, and votes on contributions is harder. who is going to read and react to the open sourced content that students submit?
In my experience a significant proportion of "evidence-based best pedagogy practices" is hogwash. I'm pretty anti-AI, but going against education-school dogma isn't a reason to avoid AI.
I recently did a course on teaching bachelor-level education, and had to study a couple of books. They all seemed to quote their sources and I got the impression that they're rooted in science.
It's science-based in a sense, but a pretty weak sense. I did some work with education researchers and my impression was that their methodologies were quite haphazard and they were sometimes just trying to find a way to "tell a story" that they already had in mind. Also, my impression is that a lot of the research on "best practices" is based on relatively limited comparisons between schools or teaching methods, and there is insufficient evidence that this or that is truly a "best practice" in a more general sense. I also interacted a good bit with education grad students when I was in grad school and they almost uniformly seemed to have no coherent sense of what they were doing or what their field actually knew. I've known people who had similar experiences with education schools elsewhere.
I've also read news stories and stuff that support the idea that a lot of "education education" is of questionable value. This story is pretty old by now but has stuck with me: https://www.latimes.com/archives/la-xpm-2010-may-10-la-oe-zi... . A relevant quote:
> Of the 1,300 institutions awarding graduate degrees for teaching, Harvard’s director of teacher education told a 2009 conference, only about 100 prepared students for the classroom. The rest, she added, “could be shut down tomorrow.”
Some of these experiences/articles are about teaching K-12 schools rather than teaching college, but there would have to be a pretty huge difference between the two for me to think that college-level pedagogical theory was meaningfully better than that for younger ages, and I've seen nothing to give me that impression.
I’ve put that article on my reading list. Yes, I think it’s true that there are situations where the science is used to support “telling the story”. As a counterpoint, I got taught about the “illusion of learning”, where it’s pretty clear that a regular college lesson (teacher droning from a PowerPoint), retention is down the drain. So we teachers know that a good curriculum should incorporate repetition and exercises.
Now the question is, are these curriculi being rebuilt? Here in The Netherlands, yes.
If we're talking higher ed., expell students that use LLMs to do their assignments for them. They obviously aren't there to learn and don't belong there.
Heh, dumb idea of the day: do a "flipped classroom" in a different sense -- instead of a student taking a quiz or exam, you provide a clueless AI and the student has to train it on the subject at hand, and then the AI takes the test.
Lest anyone think this is a brilliant idea: an LLM would likely be much better at doing the ~~RLHF~~ RLMF, so you haven't actually gained anything. But I feel like there may still be the kernel of a good idea somewhere in here.
Perhaps your assignment is to iteratively train the AI, which is pre-prompted to seek out every reasonably possible failure mode when it generates solutions. So the learning mechanism is to train an adversarial AI on the subject matter as a way of learning it yourself. This does not solve the problem of cheating with an LLM, it's an alternative pedagogical approach.
I was an (adjunct) prof at Stanford for over 20 years. I taught the top level interaction analysis course - basically cognitive science applied to human interaction with complex systems, including AI based system. As soon as I saw students turning in AI papers and projects, I knew all of upper education as we know it was fucked. I retired. Education is fucked, but learning isn’t! This will be a golden age where you can learn anything you want to. If you don’t want to learn anything, you’ll be a meatbot working for those who do. Many people may be happy in that role. I know someone who is extremely happy working at as a barista. He was also happy having gone to college for a useless degree. He’s happy he did that. He tells me the things he’s learning on his own every week from AI tutors. Quantum Mechanics. People will find a way to make their way, mostly. Education as currently organized is fucked. Teachers should be there to help students learn what they want to learn (at least after elementary school, which is still important and won’t be disintermediated). Like flight instructors. People who want to learn to fly learn most of it on their own, and the flight instructor is there to help (and keep you safe at least at first, until you solo). People seek out flight instructors, and will continue to do so (at least until airplanes fly themselves - but that’s a different discussion). Teachers will become tutors. And I dare say that teachers will like it better than their current role of having to force-feed 2/3rds of a class who doesn’t want to learn. Instead they’ll enjoy helping the few students who actually care to fly to fly.
AI is great for those who want to learn. Earlier, one has to find all relevant books, search specific forums, or use google in order to find answers to questions that teachers or fellow smart-students find silly. Even if AI gives false answers, it is better than having no answer. At least, the search space to find an answer is reduced.
Is society really trending in the direction where learning to do homework assignments is meaningful?
I think society is moving back toward nature at rapid pace. More adversarial in general, and advantage taking behavior is the way to survive if you don't have a solid community. Society will be culled to a much smaller percentage of people than are present today, and it probably won't be pretty. Smaller communities instead of larger communities, with self-sufficient systems in place and a lot of warfare...
Don't see how AI overlords controlling everything will work out. If no one can buy the food from the large producers, then they will dissolve too. Supply chain will fail at some point in the next couple years maybe? Diesel is already outpacing wages for truckers.
The people with all the money are positioning themselves to reap the assets from others to give themselves as much leverage as possible.
At this point I don't really see how it could go any other way.
I don't know. I'm trying to figure out what's going to happen, but it doesn't really seem like anything good is going to happen because the trend is wanton disregard for morality in exchange for stealing wealth.
Makes me really sad. I'm also on fixed income so I worry quite a lot about my future.
School homework is one of those things which, like the OnlyFans economy, I hope AI utterly destroys, despite my general detestation for the corrosive effects of AI on society.
> For example, the evidence favors frequent low-stakes assessments with feedback (e.g., homework, quizzes) over few high-stakes ones (e.g., exams) – but AI is undermining practice in low-stakes settings and pushing us more toward exams.
Only because you're obsessed with being able to assign grades and fail cheaters.
What is the point of learning something an AI can already do better than you? There are better methods of brain training.
The situation reminds me of what my teachers used to tell me. "Learning how to look up things in a book will be a skill you'll need all your live and you won't carry a calculator with you at all times when you are grown up." Both assumptions were proven wrong before I was even out of school.
If the only remaining value of education is "to become a well rounded person" then why waste your time on it? It used to be you went to university because that was the only place to get knowledge from. Which is also why it was dominated by the rich and powerful. These days university feels a lot like paper money. Everyone just pretends it has a real value and because everyone agrees, that's where the value comes from.
Okay why do we need literacy if we have such beautiful screen readers? Maybe because it's better when you actually absorb these fundamental skills rather than rely on a disability aid to relieve you of a self imposed handicap.
Practicality. It's more convenient to read door and street signs, restaurant menus, name badges, ticket machines etc. etc. without going through a screen reader.
It's little effort for a large benefit. But anything you learn past middle school in formal education is a moderate to large effort for a small to neglectable benefit. And you'll forget it anyway if it is uninteresting and not useful in your daily life.
Anything interesting or useful in your daily life, you'll pick up by yourself even without formal education.
This same logic applies to all the upstream education. It's convinient to have a more limited calculator in your own brain, or better yet, the ability to simply see numerical properties factors, in a way a calculator doesn't replace. It's good when you can yourself understand the physics of everyday systems, or perform useful calculations about them. And if your work is indeed tied to it, it's good to know basic semiconductor physics, even if the ordinary person has a million more immediately useful things to study.
Knowledge is not a tool for answering the absolutely minimum questions you encounter in daily life, but a way of seeing the world. Reading allows you to use language in ways you could not imagine with spoken word alone, just as math or physics or computer science enable new ways of thinking inaccessible without intense study.
I don't disagree that this is all useful to have. My point of contention is that the effort required is not worth the time investment if you are going to forget about it in a few years anyway because you never needed it in your daily life.
Our brains can't hold infinite amounts of data and a lot of information will become inaccessible if not used regularly. This is also why most of us do not speak twelve different languages, despite this being a supremely useful skill that is relatively easy to pick up compared to semiconductor physics.
The students who want to learn aren't having the AI do it for them. And they will learn better in the evidence based system.
The ones who don't want to learn will cheat themselves of an education. This is their own problem and we shouldn't punish the actual students to catch these others.
This is exactly what I said. The obsession with grades (GPA) over education is the cause here. Intentionally making the education worse in order to protect the grading system.
It's funny how people talk about "teaching" getting more difficult with LLMs.
Clearly teaching got a lot better, since we now have a new teaching tool at our use, the LLMs. It's evaluating that got harder, because students can use the same tool to cheat in traditional forms of evaluation.
If "teaching the students" means the same as "evaluating the students" to you, maybe you shouldn't teach in the first place.
Teaching is imo mostly about techniques that battle laziness. Teaching intrinsically motivated intelligent students has always been trivial. All progress in education is about scaling it to work for people who only reluctantly engage with the material.
Right, and part of that is nurturing student interest and buy-in, because it can increase. Traditionally smart teaching and the right structure could result in some percentage of reluctant learners being drawn into good habits and better retention. I think the subtle part of the AI in education crisis is how it basically has smothered that "nurturing" in the cradle, right from the get-go, so it's a bit of a cold-start situation in many classes.
It’s not just evaluating. Students used to learn skills while doing an assignment. LLMing through assignments teaches copy-paste at best.
It’s like copy-paste Wikipedia presentations 2 decades ago. People used to learn things doing research for a project. Then suddenly it became copying off Wikipedia and people not even pre-reading what they copied into PPT.
Teaching and evaluation are of course not the same thing, but evaluation is an important part of teaching.
There is a mismatch between students’ overall desire to learn things and the moment-to-moment experience that learning is hard.
Even devoid of all the economic and social pressures that grades impose students would compromise their learning experience.
Ever regretted looking up the solution to a puzzle in a video game or sneaked a peak at the crossword puzzle solutions?
One of the hardest part of teaching is keeping students from self-sabotage.
Evaluation helps with that.
Beyond that evaluation is of course also useful just as feedback.
What you’re calling evaluating is a part of the learning process for the student, not just about confirming that the student knows the thing. LLMs are actively undermining learning in this way. There was useful friction which has been removed.
That outlook is way too bleak. As a person who has taught at universities for fifteen years and has run lots of open educational formats and infrastructure with other public institutions and citizen groups, my experience is the opposite. If anything, the desire to learn and curiosity are among the strongest common traits humans possess. Grades are just a tool and a proxy that most people working in formalized educational settings have to work with at specific points.
Motivation can come from many sources, not only grading, e.g. individual encouragement, exciting applications, day-to-day relevance, interest shown by peers. Also students can just be intrinsically motivated.
It may come in different styles. But what do you do with students who don’t have motivation beyond grading/punishment? Some people just do not have motivation unless somebody standing by them with a stick.
Of course what's interesting is that when viewed from the perspective of 'I have a kid and I'm willing to orchestrate their education myself', AI means the kid can get an education unheard-of even ten years ago. Personal, genius-level tutor, expert in any subject in which the kid can muster a question.
The only people really having a crisis here are teachers who want to feel relevant.
Why is that interesting? How often does that happen? Has it happened even once? Seems like fantasy; this has been possible for a long time, and despite it being possible, it almost never actually happens. Before AI, kids willing to orchestrate their own education could use the internet to become experts in anything. Before the internet kids could use libraries and books to become experts in anything. But how many take the time and effort to match a 4 year degree? Statistically, approximately zero.
One major problem: how do you get a job for which a degree is required, if you teach yourself via books/internet/AI? How do you get into graduate schools for which a degree is required? How do you envision advanced degrees working without funding or labs or mentors?
All of education is undergoing dramatic seismic shifts because of AI right now. Attempting to insult teachers for talking about it doesn’t reflect on teachers, it reflects on you.
It's interesting because it clarifies that the problem here isn't the kid's education; the 'problem' is that teachers and colleges are upset that they are no longer offering something valuable.
The kids are fine; the rent seekers aren't.
And to your larger point: the entire university/graduate/lab/scientific system was created to formalize what was originally citizen scientists trying to understand the world from their basements. Scientific journals are just a formalization of nerds writing letters to each other about what they figured out. Consider from first principles how much of what exists now actually still serves that original purpose.
You want it to be a jobs program? Sounds good. Take their computers away and make them use pencils and you can create even more jobs.
You want to do science? Figure out how to do science. You want the kids to learn? Figure out how to teach them.
Just don't protect a jobs program under the guise of something nobler.
I disagree that’s the problem, and I don’t think your claim is true. Degrees are still extremely valuable, and income stats alone prove it conclusively. I don’t find your suggestion very interesting, not because colleges are gatekeeping, but for the reason already stated: because most people do not and cannot teach themselves and curate their own educations at a university level, and because even if they did, which they don’t, it still wouldn’t help them get jobs or advanced degrees in today’s workforce. It might be an interesting suggestion if that happened with any statistical significance, but it doesn’t, and isn’t a realistic path for education for society.
Figuring out how to teach kids is exactly what this article is doing. How do you propose to teach kids, when they refuse to put in the work, and reach for AI for everything? Your suggestion to make them use pencils seems naïve/ignorant/obtuse; this article is written by a computer science teacher. You can’t force people to use only pencils for homework (as the article points out), and it would be ridiculous to use pencils for writing computer programs in 2026. That suggestion is a non-starter. FWIW, the impression I’m getting is not that you have any helpful suggestions for education, but that you don’t have experience teaching and don’t understand what’s happening in schools right now.
Who is protecting jobs programs under the guise of something nobler? This seems like a straw man unrelated to your previous comment. I’m getting the feeling you’re anti-education for some reason, maybe have some bone to pick with the university system, is that accurate? Asking kids to teach themselves or forcing them to use pencils are not going to be successful ways to improve education.
You're making ad hominem comments about me; your argument about the degrees is making some heavy assumptions that such things won't be massively changed by AI.
And you're taking the pencil comment weirdly literally, like it apparently went completely over your head. We're clearly talking past each other. I don't see any purpose to talking about this further.
Things are being massively changed by AI. That’s what this article is about, the article you seem intent on poo-pooing. I only asked you a question about your stance, it was not an attack. My question was aimed at trying to uncover why you’ve cast insults toward teachers multiple times, and implicitly the one who authored this article. The only ad-hominems were yours, and they started with your first comment.
What heavy assumptions are you referring to? If you mean assuming that kids probably won’t skip university and instead will be self-motivated enough to use AI non-stop for several years in order to teach themselves enough to get a job as a biologist or a ballet dancer or a civil engineer, and if you mean assuming that companies and organizations probably won’t suddenly start accepting self-AI-taught candidates as being the same as having a degree… if these are the assumptions you’re referring to, then you’re right - I am assuming those things and I believe them. No matter how AI changes things, it’s a pretty heavy assumption you’ve made to think kids will self-motivate an education just because information is available, especially given that was always possible and still chosen so rarely that it effectively doesn’t exist. And it’s a pretty heavy assumption to think that universities and organizations will change together in a way that can vette the merit of prospective candidates any time in foreseeable future. AI will change a lot of things, but you’re underestimating our social structures and what things like the accreditation system do for the workforce and the economy. You can argue it’s got problems, but assuming like you did that AI can replace it anytime soon is a very heavy assumption.
If the pencil comment and suggestion to take away computers wasn’t literal, then you’re right, I have no idea what you meant. Feel free to explain. If you can’t, then I’d agree, there’s no further purpose.
Replace the profs with ai. Force every student to do a three hour one on one interview test with the ai prof to demonstrate mastery and get a final grade. Have other ai proctor. Replace homework with similar “ai sessions” where the student is guided by ai and given specific exercises tailored to the students learning trajectory and personal profile. Students can and will learn better with ai “professors” than they do now. The problem is that higher education is not about learning ultimately. The problem is ai in education is that ai exposes the lie, it was never about learning, it was about credentials, money and control.
If my department ever reached a point of such disrespect for their students, I'd quit my degree on the spot. I understand what a university degree is, and wouldn't pay full tuition for this sad pantomime of one.
Do you recall universities going after the MOOCs? I do. Do you think students benefit from smaller class sizes and more one on one time with professors? If you ask a professor what qualifies them do they say their research or their pedagogical skills? Do people pursue higher education opportunities at the same rate when no degree or certificate is offered? What happened to MOOCs anyway? If Harvard is so great for the nation, why not 10x its enrollment? I’ve probably thought about this much longer than you have.
I am teaching some math courses, and I see how LLMs disrupted all standard approaches, and I don't know what to do.
I rely on written exams that are open book, but forbid any use of computers and smartphones in class.
The university insists that homework can't be optional, but it lost its meaning. I've tried to explain to my students that it's in their interest to think about home assignments on their own, but the majority will obviously use gen AI, and it's such a waste of time to check and grade LLM output.
I just had an experience where all students were given "sample problems" to try at home and prepare for the written test. Many of them just dumped the document into an LLM, asked it to produce some "exam guide", and showed up with that thing printed out, asking me during the test to explain what LLM output meant.
It seems like the university also has many people pushing for AI use for everything, but I teach basic stuff where the goal is to make students think on their own and digest some fundamental ideas, LLMs can produce perfect solutions, but relying on them is pointless.
I teach math-heavy economics courses. I found that grading homeworks for completeness and pairing the homework due date with an in-class quiz on the problem set material (taken in class the day the problem set is due, after the set is turned in) has reduced the incentive for students to use LLMs to complete the homework. Exam questions closely mirror what's on the homework, and exam grades correlate with students who make legitimate attempts on their homework. I can sus out legit attempts typically by handwriting. No erasing and fast writing means it was copied.
For grade weights, I weight the homework the lowest of any grading category, with in-class quizzes next, and exams scores the highest.
Humm doesn’t this perhaps suggest you should do away with the homework, offer optional practice exercises and double down on in-class testing?
Homework is a tool for students. It is where they practice. I see my job as a prof to curate the problems I think will help students best prepare for the exams, and use that space to help them gain the intuition about the problems I don't have sufficient time to cover in class, like through an application. As my profs before me used to say, true learning comes in doing problems.
I find bright people who are self-motivated tend to do problems on their own, but this is not the modal student. This is like the top 10-15% of any class. The modal student is motivated by getting a degree so they can enter the working world, and sees coursework as an obstacle to that. They'll take any and every shortcut to get the highest grades with the least amount of effort. However, particularly with first and second year undergrads (but I've also seen the same behavior in grad students), they do not understand the purpose of pedagogy. That doing problems is where you learn the material.
So if you make homework optional, the 10-15% hold steady and do it but the bulk of the class will stop doing problems and tank their grades on the exams. They will not internalize the lesson after that first bad exam, because it is morally and practially cheaper for them to believe they had a bad day or that their professor (me) is just a terrible meanie who constructed an exam with problems they've "never seen before" and "had no connection to the lecture" (actual evaluation comments I've had when I've tried to the homework-optional method).
Perhaps they are also dealing with administration that says things like “The university insists that homework can't be optional”, as in the grandparent comment.
The university administration needs flexibility rather than draconian rules that dont work.
In any case, even if the uni insist on homework, the teacher can simply mark everyone who hands in their homework 100%. It is impossible for the administration to police it, or if they do, they'd need to hire someone to mark the homework (which conveniently solves the teacher's issue).
This is basically my solution.
Students who copy all solutions from LLMs maybe learn something, but it's like 5% compared to what someone can learn by doing the assignment without external help, or even by asking their peers. It's a bit disheartening, because I invest quite some time and thought in designing the assignments.
It was very different before LLMs.
Seems to me that requiring the homework and having it paired with the quiz works well in terms of cause-effect for the students.
It sounds like doing the homework is a form of practice exercises for the quizzes.
Homework is actually useful practice if you’re not stupid enough to use an LLM to complete it.
Exactly, it's the most important part of the math education, there is no point in memorizing definitions and theorems without putting everything into practice.
But it seems like the temptation to get perfect solutions from an LLM is too big for many students nowadays, they lack the maturity to understand that using LLMs for basic assignments is counterproductive.
I might argue that homework is, for mathematics, where the useful learning happens.
Yes it's all fun and games reading uni level maths nodding along to all the proofs until you're suddenly asked to do the exercises and suddenly nothing made sense and you are completely confounded by such an innocent looking question.
This is a great approach. I remember even pre-AI having issues where, at a big state school, basically all the answers to homework could be found online. Sometimes I made the choice to copy a homework and tell myself I’d catch up later. But the classes where there was a scheduled quiz right away just forced me to keep up.
> The university insists that homework can't be optional, but it lost its meaning.
The problem is not LLMs, but your university. In my undergrad, almost all the math courses had no required HW. They'd assign it, but you wouldn't turn it in. You'd come to office hours for help on the HW, or ask in class (they'd often dedicate the first 10-15 minutes of each lecture to Q&A).
It was awesome for people like me. No time wasted on neatly presented HWs. I'd do it quickly, verify the answers, and study for the exams.
The flip side was there were many exams (you don't want your grade catered if you do poorly in one exam). A course like calculus could have 4-5 "midterms", and then the final. Often, they'd drop your lowest midterm score so you're allowed one bad day.
Indeed, I've switched over to model similar to this in the course I teach. Weekly optional problem set + in-class quiz every week with one homework problem verbatim + an extension. The homeworks are completely open resource, LLM, friend, sorcery, w/e. I figure that, even if they're given an oracle, they have to internalize the ideas to the point where they can generalize on the follow-up. I'll only really know when the course finishes, but I think this will work out. I want to pair this with an oral midterm and final, but that'll have to wait a semester.
In my country of origin (not US) that's how it worked starting in high school. Homework was called "suggested problems". The teacher reviewed them in class. No grading. We ended up doing it because that was almost the only way to pass the exams (which, btw, could be up to 50 questions long...)
I heard from a friend who went to German university that the homework was never graded, maybe you could get help with it but the professor didn’t have TAs and had a large class. Lots of Midterms were also not feasible given the setup, so often 100% of your grade depended on the final.
I can’t imagine as American how that would go down with most American students.
I think a possible solution is to make homework carry no grade weight, and simply mark it as complete or incomplete, like a presence check.
Homework still provide valuable practice for students who are truly committed to learning. An automated system could provide an initial assessment and feedback. However, students can request human feedback paired with an in-person meeting. Such requests would require a mandatory explanation of what feedback they want or what the issue is. Professors and TAs will therefore only read and answer such requests themselves and will not waste time on AI output. At the same time, the system will still support students who are genuinely interested in learning.
Students who use AI to do their homework are not interested in such feedback anyway, and they will also avoid in-person meetings because it will naturally reveal that they are unprepared and did not do the homework themselves, which is humiliating.
To make this system work, we should present the in-person meetings as collaborative 'working sessions' or group office hours. Encouraging students to sign up in pairs or small groups also reduces individual anxiety.
The main channels for grading remain exams and oral presentations.
In most colleges, if too many students fail the course, the instructor is blamed. So you cannot optimize solely on catching AI cheaters.
Committed to learning and going to jump ahead here to say if smart glasses (without cameras or in time with) become the norm and Ai is always right how many will not take the time to learn the information is always right in front of them in their view.
Myself as a student i take my class notes and the subject matter the teacher provides and have Ai create multiple choice quizzes. It's a much quicker way to learn tho not as quickly as AI glasses I mentioned.
> oral presentations.
These can be written by AI. Or do you mean something more like the defending of a thesis?
Explaining something to an audience, even with notes, is hard to do without actually understanding what you are saying. It’s kind of like a rubber duck.
It's crazy that we went from "we found a significant number of students used AI in our course and gave them a 0" to "yeah they all use it we can't stop it". Basically just accepting widespread cheating, normalizing and accepting the complete abandonment of the first lesson in ethics we are taught and held to in academia.
I know this is absolutely impossible because colleges and universities have headed in completely the opposite direction, but the solution: Literally just expel them. Just like they did when people cheated when I was in college. Don't make it a small punishment so that they turn it into a risk/reward assessment. Colleges are overpopulated anyway; just trim the fat of students who shouldn't be there. Do a follow up oral exam interview about their project and if they can't explain what someone like me spent 30+ hours building and can speak about for 4 hours if asked, then expel them. They can make your coffee at the nearby Starbucks, who cares.
If you don't do this then understand that you are telling your students not using AI that they will need to begin cheating like the others to be able to keep up with the changes. OP mentioned they massively increased the scope of their assignments, which means OP is forcing their students to use AI now.
Agreed. It’s very sad modern society is stopping using ethics and at the same time increase using different firms of policing.
I'm taking a year-long sabbatical to study a language at university right now and they're changing their assignment processes to curtail AI use. We have two versions of written assignments - the first to be written by hand in class with just a Swedish-Swedish word book available, and the second version to be typed at home implementing the professor's corrections from v1.
At least in this course students seem to be taking their strict 'no AI' policy quite seriously. Then again this is also the kind of course you do when you want to actually learn the material. It provides a qualification for further higher education in Swedish, so if we don't legitimately learn the stuff there's just no point as we won't manage in future courses.
Recently it has been reported that many Chinese students in Moscow are no longer learning Russian during their years at the university (to the dismay of Russian officials who hoped this would be a form of soft power): even for in-person lectures their Bluetooth earbuds will just translate the lecturer's Russian into Chinese in real time. So, perhaps people around the world will start feeling that, thanks to technology, exams testing language skills are a mere formality and can be safely cheated on.
Russia is a bad example due to its current international reputation. On your second point, would an example of this be an oral exam where the student has tiny hidden earbuds? Outrageous but you're probably right it's going to get worse.
There is a well-known scene in a famous Russian comedy about exactly that: https://www.youtube.com/watch?v=m3n4LptYWNo&t=2482s
Actually, language is no longer the barrier it once was, it wouldn’t surprise me if some universities start opening enrollment up beyond the native language of the university accordingly.
There is indeed no point in grading home assignments anymore, but why not randomly ask a student at the start of each class to present their solution on the board?
And do a final exam on oen and paper and nothing else, and an exam that relies more on thinking than rote memorization of formulas (or print the relevant parts of the course on the exam document itself).
> why not randomly ask a student at the start of each class to present their solution on the board
This makes the class really awful. I had professors with schemes like this, maybe calling on someone's name randomly 1-2 times in class to answer a question they'd just asked, just to ensure everyone is scared into paying attention.
there probably is no time for 100 students to each do that in a university mathematics lecture, but there are probably ways to make sure students are still learning by restructuring the university into mandatory recitation and labs or something
Having two random people picked each time might cause all of them to at least pay attention.
I’m wondering if this is the way, but more faculty time is needed for assessing. So education costs go up.
They said “one person at the start of class”?
there aren't 100 university lectures a semester, so there is a small chance you are picked. meaning, good odds to not give a shit or good incentive to not go to lecture if you are behind. also, wasting 100 students' time to watch someone struggle through one gotcha, is more of a punishment for everyone, not something to make students learn.
That sounds pretty frustrating. I wonder if it wouldn't be a good approach to, say:
1. Make the exam the only thing that determines the grade (or technically, proves that the student learned enough);
2. Give out test assignments students can do, but don't have to;
3. Offer students for the professor to grade their work if they want to get some feedback on how they're doing.
Would that lead to less time wasted by tutors grading AI generated answers? From where I stand, students should be allowed to prepare for exams any way they like, with or without their professor's help. If they managed to learn, they pass.
According to the article, #1 leads to "failure and drop-out rates of 50 to 80%". #2 and #3 might work for organized and motivated students, but graded assignments from other classes will be prioritized by most.
Failing people who didn't learn anything during the course sounds like a desirable outcome
Given the education literally does not matter anymore for white collar work, I'm inclined to agree with you.
I'd rather a MUCH larger amount of professors and teachers come to class to find their classes deserted (by their own hand), and indeed, their own justification for existing gone. The world would be a lot better off. Gatekeepers are anti-human.
If education doesn't matter, you've had the wrong education.
Maybe more drop outs is a good thing, we need more skilled labor anyway
In many cases it probably is. But lower graduation rates cuts against the incentives structures in place at many universities and colleges.
Then their parents come in and complain about where the $30k that semester went. This is a consequence of the exponential rise in tuition. You're now in lawsuit territory if you don't deliver the goods.
When I was in engineering school ~20 years ago, some lecturers openly said that they gave the assignments just enough credit to be worth doing. Something like 10-15% of your final grade would be from assignments, the rest from the exam. So we didn't blow off the assignments to go drinking.
Of course, this a lot easier for some subjects than others. Subjects like Film Studies relied on exams much less, and assignments much more.
I took some classes like this and they were great. I felt they were much fairer than classes where mere diligence/compliance could masquerade as competency by way of final grades padded by homework scores.
They were disliked by many other students, though, and some people really think university grades should be about things like diligence, compliance, time management, etc., rather than pure subject mastery.
Give a short quiz at the start of the class with one or two problems directly from the homework.
1. is not a terrible idea, if structured correctly, but it doesn't adequately incentive the learning process, which requires continuous practice. For 1, I'd say put 80% of the grade in three to four exams taken over the semester, and allow one midterm exam grade to be dropped or replaced by the final exam grade. The remaining 20% of the grade is to incentivize practicing problems: homework graded based on completeness not correctness, short in-class quizzes randomly sampling a homework problem to incentivize students to legitimately complete the homework that are graded for correctness, and student participation in classtime activities.
>I rely on written exams that are open book
How about cutting out the open book so that the students who don't study and to their homework actually fail? When I was in school many years ago "open book" was only used for special ed students and others who couldn't carry a regular academic workload. Homework was assigned, but our grades were determined solely by testing our mastery of the material. If you could master the material without doing the homework or showing up for class on non-testing days, more power to you. College has become more like kindergarten with all of the hand-holding.
When I was in college, whether or not an exam was open book seemed orthogonal to the difficulty of the exam as well as the class's median uncurved score. Allowing the textbook or notes lets you write an exam that might otherwise be unreasonable.
In the classes I took with open note exams where I did well, I barely ended up consulting my notes anyway. The students who didn't study could flip through the textbooks all they wanted, but they lost so much time doing it that they couldn't complete all of the problems on the exam. For the students who studied well, the textbook was way easier to jump around in because they knew it well and knew what they were looking for.
Am exam that is designed to be closed book/closed note where a specific student is allowed some amount of notes as a disability accommodation is just different from one that is designed to be open note/open book from the start. The latter, testing the same material, is generally longer or harder to make better use of the format.
Open book exams IME are usually more difficult, since the exams can be crafted on the assumption that you don't need to dumb anything down since students can look up certain things.
We had a formula "cheat sheet" on every exam paper, but if you don't know when or why you're supposed to use a formula, it might as well not have been in the cheatsheet at all
Open book need not be easier, it entirely depends on the exam.
This was too long ago for me to remember the details of but, just making an example here, we might have had a course on homogeneization of elliptic problems using a model PDE with a div(A grad u) term throughout, and then the exam was about a PDE with a curl(A curl u) and some other added terms.
Notes carry the method and theoretical tools but you then still need to apply them to a new problem.
So, if anything, our open book exams were harder and a better test of understanding and mastery. You couldn’t rote your way into a good grade.
Put a mixture of questions in that are impossible to answer but don't tell them, and as usual give them marks for explaining their working. Tell them you've devised the questions so some are "AI resistant" but don't tell them how. They'll never know if the AI has tripped up and hallucinated or given them the right answer. Only those that really understand what's going on will set out some working that shows they were on the right route and then got stuck where they were supposed to.
That will either lead to students becoming frustrated after spending inordinate amounts of time on the unsolvable problem or students giving up too early on the solvable problems.
> Put a mixture of questions in that are impossible to answer
Students will complain to the admins and waste more of your time.
The AI would just output that it can't be solved..
And the students will think "damn that's the AI resistant question", or those that actually understand it will answer why with their working
I've had the same experience, and likewise I have no idea what to do.
Or rather, I do have ideas, but they do require substantial restructuring. E.g.:
1) Strict, in-person (no exceptions), electronics-free exams with written and oral portion.
2) Transition to guided independent learning: students are given written materials and recommended exercises to study. They are shown how to use AI assistants productively. Instead of lectures, there are scheduled discussions, where students can ask questions and listen to additional explanations from the lecturer. They can also seek feedback for completed exercises. However, none of this contributes to grades in any way.
With such an incentive realignment, I think it's possible that higher education could be salvaged. But... apart from general institutional inertia, there is also the political angle standing in the way. Everybody needs a degree in the "developed economy", and the above changes are basically the opposite of the nobody left behind policy that enables half the population to attain one...
> “…students were given "sample problems" to try at home….Many…just dumped the document into an LLM…”
I think this development should make all educational programs reconsider the flip-model: homework at school and video lessons at home.
Homework sucks. Learning should be at your own pace (play/pause/rewind).
This is a nice idea, but if you follow the Carnegie model of the number of hours per week it takes to learn a subject per credit hour, which is estimated at something between 6-9 hours total including classtime and out-of-class study per week for a normal-semester 3 credit course, then you can't meet the amount of time students need to be exposed to lecture (explanations, demonstrations, derivations), to do the background readings which provide necessary context that can't always be adequately covered in lecture, and to practice the concepts and tools on their own.
This. I am currently a university student and I wish I could practice in class and not at home.
Office hours, tutoring centres, and self formed study groups are all great options.
For me it's about time management, not about getting math help. I am consistent at attending classes because it's mandatory, but not consistent at finding time to practice math every day. If I could practice during classes, I think that would help me.
That’s an entirely different problem, and a hard one to solve. I would still suggest looking at the tutoring center or groups since there is time accountability there. If you don’t show up it is more consequential than skipping class from a personal point of view.
Dude, hop on the Gemini student plan and try Gemini Notebook and Gemini Live. Live is a personal TA, Notebook literally podcasts the readings and what you need to learn
I am not buying this. I use AI to check my work, but it is absolutely terrible at teaching or explaining.
That's a terrible model though. If someone is paying to be taught and you just hand them video lectures... We already have video lectures at home. Why would they pay you? What product are you meaningfully offering in that world?
Tutoring, office hours, teaching assistants. One of the best classes I took was a "studio style" class where we would work on a project and occasionally the teacher or a TA would interrupt and give a brief lecture or demonstration based on a "teachable moment" that had occurred.
You are selling a diploma. Some universities also sell some degree of attestation to the high resilience of their graduates. And of course the space to network. Good teaching is not on the list in my experience.
In general university lectures are horrible (of course there are exceptions). The lecturures oftentimes are required to teach as a condition of research funds, so for many it's not a priority anyways. On top of that most couldn't care less if their teaching style is didactically sound at all.
Yes. You've described the problem very well
?? The model is to flip the roles, not to eliminate in-person teaching. The product is in-class homework and at home "lectures". I'm not saying this is appropriate for everything, but seems it would be useful for mathematics at some level.
Homework is learning at your own pace! You pick the time and place and pace
Perhaps let them take a practice exam early on, so they can realise they'll fail the real one unless they actually study?
> asking me during the test to explain what LLM output meant.
Perhaps (sarcastically) this is the new approach, because this is now the point where they’re forced to struggle and therefore actually learn. An exam every other day, let them ask the questions.
Not sarcastically, this is the flipped classroom approach. Read the examples and watch the lecture on your own time, then come and do the exercises with a teacher available to explain when you realize you didn’t understand anything.
Just don't make it open note then? Or you provide the only notes or cheat sheet available to them at the start of each test. That's what some math tests I've had had done before and I believe the SAT also does the same thing with formulas in the back of the test to reference.
> The university insists that homework can't be optional, but it lost its meaning. I've tried to explain to my students that it's in their interest to think about home assignments on their own, but the majority will obviously use gen AI, and it's such a waste of time to check and grade LLM output.
Can't you just not grade it at all? If the student has done their homework, just consider it passed. And then don't count it at all for the final grade.
My favorite math class I ever took had optional homework, but every exam problem was taken directly from the homework. So if you did all the homework the exams were very easy.
I imagine this would work very well in the current period since there’s no reason to use LLMs for this homework. Of course even back then many students didn’t do it and got low grades, but you have to be responsible at some point.
I agree and that's my idea as well, exams are open book and extremely close to assignments. My issues are with the first year undergraduates, who are not mature enough to do homework that is optional, and when it's not optional, they'll do it with LLMs and call it a day.
Grade homework for completion, have a weekly in-class quiz that is basically a few problems that are near to identical to the homework. You lose a bit of class time to the quiz, but it creates a strong incentive to complete the homework, and also eases in the midterm/final for students instead of going nothing nothing - boom here's 30%+ of your grade.
Why not just make the assignments "reading" based and set an expectation of randomized quizzes that take 5 minutes at the start of class to fill out to get the homework credit? When I say randomized, like multiple versions. You can use LLMs to churn out PDFs to a tailored format and utilize a pool of questions.
It seems to me that the only realistic option is to have more exams, closed book, and then use gen ai yourself to help reduce the workload of grading. Possibly with some kind of curve assigning more weight later in the semester.
Hmm people can now do a lot more than they were supposed to be able to without LLMs - shouldn’t we lean into that?
Force people to go all in with “impossible” tasks and see how they do with the power of LLMs. Then spend some significant time teaching people _with_ that solution in place what was actually built and what would have been a better architecture? A … software archeology session if you will.
That’s what most of what people do would be anyway - building PoC fast and then figuring out how to scale them?
If grades matter, the students will cheat.
> Many of them just dumped the document into an LLM, asked it to produce some "exam guide", and showed up with that thing printed out, asking me during the test to explain what LLM output meant.
That student presumably performed poorly on the test. Your approach is working as intended here, isn't it?
Also, is the AI-generated exam guide all that different to a human-written guide they could have downloaded a decade ago?
> I am teaching some math courses, and I see how LLMs disrupted all standard approaches, and I don't know what to do.
I wonder what the financial cost of AI will be to teaching? Does teaching get more expensive? More things need to be supervised, more things need to be presentations or in person?
> I've tried to explain to my students that it's in their interest to think about home assignments on their own, but the majority will obviously use gen AI, and it's such a waste of time to check and grade LLM output.
I think the problem is the students also take ~4-5 subjects per semester, and each of them may be just as demanding. High school is even more spread out with 6-8 subjects per semester. There's only a certain number of hours during the day for study and homework. For many, math homework ends up being the most time-intensive. Also these exams determine their GPA which in turn plays in part in determining their standing for the next few decades of their lives.
I've always wanted to find out if this would work:
If I knew most of my students were using LLMs I would encourage the rest of them to use them too. I would then change the marking criteria such that if they get a single point wrong that's 20% off the entire grade. 5 mistakes gets you a big fat zero.
LLMs are great but they make mistakes. At this point your students would have had to spend so long checking, re-checking and triple checking the LLM's output that they will have accidentally learn what they need to. They will likely need to cross-reference multiple LLMs and at least have read their output which is likely an upgrade on today.
Or some variation of the above, I'd be interested to hear your thoughts.
This sounds like it would come at the cost of the honest students who don't want to use LLM's and train their own understanding.
Reviewing is just not the same as working through a problem yourself. You often can take shortcuts when checking answers for correctness, which generation (with mind or LLM) of the solution can't take. This isn't limited to Math but equally true for programming and many other skills.
I'd push back slightly on this: I spend more of my time reading / reviewing code than writing it yet I still find reading / reviewing code much more mentally demanding. I don't think I'm alone in that.
The people that do their own work would be at a huge advantage as the LLM(s) can work as reviewers instead of doing the work so there is more chance of catching a mistake before you lose 20%.
On top of that there will be some issues where the LLM is just plain wrong. Being able to work out when that is the case is a hugely useful skill in 2026. It is also something the people who blindly feed their work into an LLM and print its answer without reading it will be unable to discover. Much to their own detriment.
> I spend more of my time reading / reviewing code than writing it yet I still find reading / reviewing code much more mentally demanding. I don't think I'm alone in that.
That's because we write much smaller quantities of code than we read. Not that LoC is a useful metric, but I'd be surprised if I write more than a few hundred lines a code a day in a legacy codebase. Most of the time is spent planning and deliberating over what to write, and perhaps revising code a few times. Meanwhile, understanding what a snippet of code does can easily spiral into having you read thousands of lines of code across several modules.
But that's for work. writing is a stronger learning tool than reading, and we needed to write tens of thousands of lines of code before we got into the door and started reading more than we wrote. It's still preferable for students to do the same and write as much for their homework as possible.
Explaining that paradigm shift from learning to working isn't trivial, and is often poorly done. But it's needed if we ever want people to frame these tools as productivity boosters, and not shortcuts through tedium. Learning by its nature involves some tedium.
I'm not trying to suggest that reviewing (code) can't feel (more) mentally demanding. The thing with at least reviews is that if you have worked through the problem yourself, you will also hit certain walls where stuff didn't work or wasn't as beautiful as you wanted (ie trial and error), and you have to activate your brain to think of other solutions, ie really think a problem through multiple times with active feedback. As a reviewer you don't get the active feedback usually.
Now this i think applies in lesser degree to the comment I was replying to. With maths, and some comp sci problems, you often can check an answer much faster than actually solving it, eg by using simple substitution for simple math problems. That is a useful skill, but a different one from solving the problem yourself. It employs different types of skills.
I think you might be letting perfect be the enemy of good here. Right now some students are using an LLM to do all of their work. I don't suggest for a moment my method is perfect but I do claim it could be better than the status quo (or at least warrants further investigation in my opinion).
What you would grade with that approach is how well your students can operate a LLM as well as the failure rate of whatever LLM infrastructure the individual student is using. This is usually not the goal of your educational format and hence not what you would grade for - apart from some (meta-)skill courses offered by uni libraries and the like. Those are mostly ungraded, though.
Anyone that knows the subject matter well enough will spot the LLMs mistakes and correct them. If you don't know the subject matter well enough to spot a mistake then you don't know the subject matter well enough.
It must be tiresome to be a student these days… with each teacher having different pet theories about AI use and sometimes trying to “trick” students who use AI, or implement some draconian rules about the way homework or exams must be done. It’s tiresome for you, but just imagine getting varying lectures on AI use as you go from class to class as a student. Must be hell
Learning and understanding were always and option, and still are.
Short in person quizzes frequently.
They don't even have to be a significant part of the rubric.
Just ensure the students know what they don't know.
> such a waste of time to check and grade LLM output
Small nitpick: you teach math, why don't you use an automated grader?
I agree homework should be optional and your college is wrong. Even without AI, for some students it's easy to cheat and maybe for some students it's a waste of time. Maybe you can assign it a very low score, like 5% of the final grade, and/or award the grade based on attempting rather than getting the right answer.
> why don't you use an automated grader?
LLM verifies output of another LLM. The circle is complete, and nobody has any doubts that this is just a theatre.
With manual grading, that one student who genuinely tries still has a chance to meaningfully engage.
LLMs have become much better at avoiding hallucinations. Moreover, students can submit grade appeals, which should be reviewed by a human.
It would be a waste of time. LLMs have become better but not perfect. Personal experience; it's easier/faster to grade for me than have the LLMs grade and cross-check each one.
Translate the proof to lean and verify it.
If the proof isn't trivially translatable to lean, is it actually a good proof?
Good homework feedback for a proof isn't "Did you get this right?", it's "Why did you not get this right?" or "How can we make your correct solution better?", neither of which fall out of a Lean translation-and-verification pass.
Good math grading is looking at things you can't trivially automate. Process, proofs, etc
Things that, in 2026, are trivial to automate grading of.
No?
I wonder if you could see this the other way round? Prior to AI you were limited in your ability to grade handwritten tests.
But now you could have the students take a written exam every other week and use a LLM to grade it?
I'm pretty sure Opus 5.5 has good enough vision capabilities to auto grade with the right prompt.
In the future, homework and officework will only exists to generate AI providers revenue.
> I've tried to explain to my students that it's in their interest
Perhaps some people do not belong at university? It is just a waste of everyone time, to insist 50% of population studies until 25 years old.
Who’s insisting that? Where does 25 come from? 4 year degree attainment is just under 40% in the U.S., and most of them finish by 22 or 23.
People are choosing to go to university. It might have something to do with having better job options and statistically higher pay…
It's like we're putting everybody on crack and then say they're too weak if they become addicted.
If you do not want to learn, and take every shortcut available to avoid it, university might not be the path for you.
1. We don't exactly do a great job showing upcoming adults their options and paths in life to begin with.
2. We're also actively giving them less paths in life to begin with. How many high school grads @ 18 will find any job whatsoever without additional school/training, and how else will they afford housing with zero income? The military is the only consistent employer at that age, and the only one who'd provide shelter with the deal.
3. On top of all that, it's in university's best interests to keep 1) and 2) the way they are. Their incentives start at recruitment, and end at enrollment. Past keeping them in college for 4-6 years, they do not care about performance, post-grad job placement, nor societal wellbeing. And part of that is due to a lack of pressure in the recruitment stage to make universities focus on such metrics. Turns out a fancy new gym is a cheaper investment.
By the time you are 18 and going to college it’s time to reap your lifetime of prep, you’re about to spend a ton of money to specialize. What does it mean to show you all the doors and motivations of life when you’re basically going “all in” on a domain?
Hopefully it doesn’t mean extending high school two years into college, still prepping to make a bet not yet made.
>What does it mean to show you all the doors and motivations of life when you’re basically going “all in” on a domain?
I don't know about you, but I sure didn't have my life set at 18.I could have easily been a mechanical or electrical engineer if my alma mater rejected me. And that was more focused than most students.
It pretty much means what it says on the tin. Before hoisting an 18yo with the sudden responsibility of an adult, show them what lies beyond their GED. That's not just limited to college; there's many more roads out there and seeing what options motivate you can let you try to specialize while you're still in your prep phase.
That's precisely why some student go into a major undeclared or even take a gap year after graduating. No one gave them that exploratory transition time, so some make it for themselves.
That lifetime of prep is completely disconnected from actual job & life experience. The kids are shown all the doors but have to choose without first opening any. Aside from the occasional internship they might have done by that point or a glimpse into what their parents are doing, which is very hit or miss.
As long as students ask the teacher what they're going to need understanding of maths for they are not prepared adequately.
That's easy to say. I might as well say nobody wants to be addicted to crack (or whatever drug), yet here we are.
25? At least in the UK, most degrees are 3 years long and most people start when they are 18.
UK (and the wider Anglo-American system) is different from the continent, where in many countries the MA is the basic degree -- a mere BA will get you nowhere -- and this easily takes 5-6 years to achieve.
[dead]
I have been feeling for ages like I am too old for the tech industry.
Reading this has made me understand that the recruitment industry will soon have to adopt the default position that anyone who graduated after mid 2025 is likely misrepresenting the depth of their understanding; people who graduated before that are the equivalent of low-background steel, until our processes catch up.
Maybe there’s still room for me.
Seems the core problem is that many/most students aren't there to learn the material, just to get grades. The class is a complex mechanism to force them to learn material against their will in a validation arms race. The teacher's job becomes something like a military commander. Seems depressing.
This is the way it's always been, sadly. Before AI submissions I had to regularly look for where my materials were posted on sites like Course Hero. Students copy-pasted from Wiki, articles, blogs, etc without attribution. Before that, they were buying or finding solutions books and copying out of there, or hiring people to do their homework/write papers/sit exams for them.
The primary difference now is that the both costs of cheating and ability to verify cheating on the instructor side have decreased dramatically, in tandem. There's also been a kind of mass effect, a cultural change in the attitude of students, where the sheer prevalence of cheating among their peers has made them less likely to feel fear, shame, or guilt over cheating. Students in the past who were caught were likely to confess immediately. Students now are increasingly belligerent and more likely to deny cheating, not just because they believe the professor can't 100% prove it, but also because they believe being caught out for cheating when most of their peers are getting away with it is unfair.
This is because we shield children too much from reality. If they had a fire under their ass that made them realize if they don't study well, they will forever be locked into flipping burgers or being homeless, most of them would move their assess.
Don't do anything productive for the society but still get to live a decent life on benefits, job security etc combined with addictive social media have ruined most kids will to study.
And the schools are even worse. When I was in school, teachers encouraged learning things; whether for school or outside, with teacher or self help. Everything was fair game and supported. Debate was normal.
Nowadays the teachers in schools are like a cult. Kids are tuned to reject anything from outside; anything that contradicts teacher is a no go zone. Anything that's "too advanced" compared to what teacher is doing is forbidden. Kids are scared of learning by themselves because the teachers punish it. What do you think these kids will do when they grow up?
> If they had a fire under their ass that made them realize if they don't study well, they will forever be locked into flipping burgers or being homeless, most of them would move their assess.
That strikes me as backward. The higher the expected monetary value of good grades, the more students will prioritise grades over deep learning.
The quote is not about grades
> This is because we shield children too much from reality. If they had a fire under their ass that made them realize if they don't study well, they will forever be locked into flipping burgers or being homeless, most of them would move their assess.
This seems pretty inaccurate to me at just about every turn.
The fire under their asses is for grades and a degree because they've erroneously been told they need one or else they'll be flipping burgers.
It's such a cliche thing to say but cheating at school cheats yourself of the education you could be getting but, since that's not what most people are there for, they cheat. They are there because they've been told that they need a degree to make a decent living and the degree doesn't technically require learning.
The least paragraph... has always been the case. My parents told me stories about teachers denying facts not to lose authority. I distinctly recall my teacher pretending that peripheral vision doesn't exist not to lose authority.
The easiest path is to allow students to cheat on homework and watch them fail in the non-coursework-based exam. Your classes, then, abandon to the un-motivated; with AI enlarging the educational gap.
If you’re forced to attain a pass rate, and can’t watch so many fail, then the certificate becomes worthless in the workplace. Or just offer paid retakes and watch AI make you money.
If you insist on persisting with coursework then a short but heavily-weighted oral part seems to be the only answer, something that will diminish teaching time as a downside, as the article and comments about the flipped classroom touched on.
The non-coursework based exam can also probably be gamed in the end.
Students could just use the AI to teach/tutor the material to them instead, but the institutions are clutching the pearls too tightly.
Isn't that the point? To learn? If you're setting a goal that the students must pass, who cares if they get the information from the professor or they teach themselves?
I don't think "learning the material by other means" is the exam being "gamed".
Reminds me of this skit: https://www.youtube.com/watch?v=jgYYOUC10aM
I thought it was going to be the Monty Python sketch https://www.youtube.com/watch?v=C7fsxnzZF9w
The linked video is Planning a Heist - Key & Peele.
Please include a description of what you're linking to. People aren't likely to follow a link to an unknown YouTube video.
I think the point is most students are cheating and not learning. Where they get the information does seem to matter. Adding friction was needed for most.
if they use ai to "cheat" the exam by having the ai produce material to learn off, then they literally have just replicated a paid tutor (for cheaper).
Why is that not desirable? As long as the exam does not allow ai usage during it, i don't see the problem.
You're giving a charitable interpretation which is not warranted in reality. Scores are down across the board and even ivy leagues need to re-think its long held honor code.
I don't think it's shocking to realize that most 20 year olds (especially those growing up and seeing that success is not tied to intelligence nor hard work at all)are taking advantage of a convenient tool to cheat when given the chance.
The point is that they're not using AI to learn anything, they're using them to pass assignments without even giving it a once over. They're basically blindly copy/pasting whatever the assignment is and then copy/pasting the answer.
What’s the difference in using an AI as a tutor or paying a human tutor?
As long as they learn.
Maybe the Germans were right? One in-person test, preferably an interview, at the end of the semester should be enough to motivate the smarter part of the bunch to make sure they understand the issues at hand. The rest is chaff anyway. Don't optimize for chaff.
As was sort of offhandedly mentioned in the article, all pre-LLM research into what results in students actually learning the material (as a distinct thing from passing the class) has landed pretty solidly on the side of frequent, low-stakes exams having better outcomes than rare high-stakes exams. The best students learn the material regardless of how you evaluate them; some weaker students will benefit from more frequent nudges to keep up rather than trying to cram at the end, some students simply won't realize they're falling behind without regular feedback, and some students successfully cram at the end and pass the class without retaining much of the material. The final group often expresses a preference for the one-exam-at-the-end model despite being disserved by it if the goal is to learn and not merely pass.
That's entirely fine. Have midterms that are also interviews. Have labs where TAs interview students vocally every few weeks, etc.
Yeah and look at where this soft approach landed us. We have entitled crybabies voting for Hitler 2.0 because they can't stomach the uncomfortable feeling of seeing a diverse person in their pure white neighbourhoods. Children and young adults need to be hit with the big hard fist of reality repeatedly until they become good people. If some should fall through the cracks, they would not have made for good members of society anyway, so best they fall early and not drag the rest of us with them.
I had that a variation of that experience. This was mostly in the pre-AI days. Homework assignments were mandatory but only counted towards exam admission not the final grade. Much better for motivation.
When I was a student, nobody cared if you did your home work. The teacher would go through the home work in the next class, and if you didn't do it, you would just lose the opportunity.
Then there was the written exam at the end of each course, which constituted 100% of the grade. The only reason that got abolished was because it made too many students fail, and the board hates that. So that became a race to the bottom, and apparently, we can still go lower.
It's reaching levels where most uni diplomas don't matter any longer. The only thing that will then matter is your network. That's the end of social mobility.
"Don't optimize for chaff." is being chaff a permanent state of a child or person?
I think I understand what you mean though, and I kind of agree. I just don't like the use of the word chaff, like something you would discard permanently.
I'm pretty sure I was "chaff" at some point (maybe multiple) in my life growing up.
It may not be permanent, but by the time someone goes to university it's not the instructor's job to fix them.
Funny to see this comment, that's exactly how studying math in Cologne was. On the very first day one of our profs said:
"This is math, you can find all solutions to the course problems in old material since I've been teaching here for 20 years, but know this you're cheating yourself and at the end of the semester you're taking a three hour test on pen and paper so good luck"
I see zero reason why AI should impact teaching. You can grade exercises as information to the students as to how well they're doing, but simply have an exam at the end of the course. Needlessly to say like 60-70% flunked through Analysis I and Linear Algebra. But they're adults and they're there voluntarily, so why is this an issue
They pay as much tuition as the wheat and the schools want them to keep paying said tuition.
One test is too stressful and may be affected by bad luck. I'd recommend many small quizzes. You can use LLMs to grade.
> You can use LLMs to grade.
The article gives examples of LLMs being wrong. An AI grading of an AI submission would presumably give a good mark, to a wrong answer.
It's terrifying that some people actually do think this. Not you obviously, but I had enough conversations with other people.
on the off chance you’re being sincere: the article mentions that they allow free retakes of the oral exam
I agree.
Honest question: doesn't this highlight how mundane and rote teaching has become? It's been distilled to the point that the lowest of the lowest common denominators can pass. The kids going through it are forced to learn things they will never use, unless it's going to be used in the more advanced class of the thing they'll never use until they graduate.
My hope is that this is a wake up call to teachers / professors to find a better way to evaluate students. To determine who has actually come away with a real understanding, vs those who were running around copying the assignments off of their peers anyway. These same people are now just skipping the peers and going straight to the all knowing oracle. It's the same as it ever was, degree mills.
> doesn't this highlight how mundane and rote teaching has become
The example questions given in the article do not assess wrote learning.
> It's been distilled to the point that the lowest of the lowest common denominators can pass.
Have pass rates been increasing? Unless I missed it, the article doesn't say so.
> forced to learn things they will never use
A computer science degree is not a coding academy, it's a basic grounding in an area of study. A computer science graduate should have some understanding of theoretical computer science and of computer architecture, say, even if they're unlikely to apply these topics directly in their careers.
> My hope is that this is a wake up call to teachers / professors to find a better way to evaluate students. To determine who has actually come away with a real understanding, vs those who were running around copying the assignments off of their peers anyway.
Academics are already aware of the importance of fair assessment, but it isn't easy, especially with LLMs in the mix.
> These same people are now just skipping the peers and going straight to the all knowing oracle. It's the same as it ever was, degree mills.
Students that cheat, and degree mills, are two different things.
It's going to be a rough decade but I think we're going to come out a lot better in the long run. Education, especially in the US, has been really stagnant for a long time, and primary and secondary education has been more and more obviously just a mill to turn out middle-wage earners to dutifully pay taxes and buy products and not try to think to hard about things.
Now that we have systems that can pass for students at this baseline level, educational systems are facing a reckoning and we can, hopefully, move past cookie-cutter education solutions and recognize the need for more hands-on, engaged education. Will it be an easy transition? No. Will all schools adapt successfully? Also, no.
But will we finally have a path away from standardized testing, middle-of-the-road education? I am hopefully optimistic. We'll rediscover the need for human growth and development in our schools. It's either that or we lose. I'll choose the optimistic route and join and support those who believe the same.
> But will we finally have a path away from standardized testing, middle-of-the-road education? I am hopefully optimistic. We'll rediscover the need for human growth and development in our schools.
I worry that the availability of this path will be extremely unevenly distributed, amplifying existing educational gaps along class, race, and language-proficiency lines.
That might be better than a forced average denominator, but the social effects of further amplifying those gaps are pretty severe and don’t take that long to happen. Combine that with the fact that education isn’t like new miracle drugs, where improvements inevitably tend to become more and more widely available to different groups, and I’m not as optimistic about the future.
Otoh, ai is much, much better at bridging language gaps than humans and human institutions.
The real question for education is whether we can get 'tutor-quality' education from ai interactions. One on one education is leagues better than classroom education; it's the one-true-result in education research. There's a massive opportunity here to improve outcomes, and in a generally available way. You can't clone a good teacher, but you can give everyone an ai tutor.
+1. I’m doing a masters now, and typically I have lectures/slides/textbook open in one tab and Claude in another. When I don’t understand something - which is frequent - I discuss it with Claude until I do. Having an always-on, infinitely patient, phenomenally knowledgeable tutor is AMAZING. Schools could mitigate equity issues by providing their own portals for this (with some guardrails to prevent abuse obviously).
Of course the onus is on me to do this at all, I don’t need to in order to pass the class. I set my own standard for what it means to ACTUALLY understand anything rather than just say “oh yeah that makes sense” and move on.
> we can, hopefully, move past cookie-cutter education solutions and recognize the need for more hands-on, engaged education.
I'm not convinced children need "hands-on, engaged" education at the age of 10. At some point there's no shortcut to being socialized that patterns in numbers are interesting (which is one of the national goals of maths for 3rd graders in my country). 10 year olds won't find that stuff "engaging" on their own, they must be informed by authority figures that it's a thing worth paying attention to.
Too much of this "fuck the schools" mentality treats children as perfect little beings that must only be supported to grow into wonderful adults, when the reality is that socializing children is a brutal exercise in fitting them into norms.
They certainly can find things like patterns in numbers interesting. I just found a blog post I wrote when my daughter was a few weeks short of seven that says she found simple sequences interesting. "She was instantly fascinated, and thought that trying to figure out the rules of each sequence was a wonderful game. Whenever I ran out of ideas, she pestered me for more sequences."
https://pietersz.co.uk/2009/10/schools-maths-wron
> when the reality is that socializing children is a brutal exercise in fitting them into norms.
No, you really, really should not be brutal to little children.
You replaced the rote recitation of faceless homework with engagement from an authority. She was not interested in the sequence of numbers, but interested in your interest. She loves you, and is reflecting your engagement back at you in hopes of getting more validation. Children do that a lot.
You describe the homework as
> basically lots or practice in basic arithmetic
Which is absolutely what you need for these sequences to have any meaning. There is no "pure math" to have a game about, without the "pathetic" examples of Lucy's 5 apples and Tom's 2. If somebody hadn't (brutally) drilled the concept of numbers into her skull, the sequences of random symbols would not mean anything to her.
Engaging with your child, who has already been partially socialized, is not proof that this socialization wasn't necessary. For that, you would have to have locked your daughter in the basement for those 7 years, and then find that she was spontaneously able to do multiplication. That is not going to happen.
> No, you really, really should not be brutal to little children.
I hope you understand that _brutality_ as spiritual. It's the brutality of going against nature, of putting a monkey in a suit and locking it in an office for 8 hours a day to feed itself. You don't beat math into your children, but they sure as hell don't learn it from eating candy and watching Minecraft let's plays.
> they sure as hell don't learn it from eating candy and watching Minecraft let's plays.
I learned math from flash cards my mother was using for my brother. He got M&Ms for getting them right, so of course I wanted to learn and join in. Many years later, I graduated with a degree in math.
So you absolutely can learn math from eating candy. I did.
> You replaced the rote recitation of faceless homework with engagement from an authority. She was not interested in the sequence of numbers, but interested in your interest. S
She was quite capable of expressing what she was not interested in. You are just refusing to take experiences that do not match your preconceptions.
There is a huge different between engagement and "being informed by an authority figure" which implies they have no choice.
> She loves you, and is reflecting your engagement back at you in hopes of getting more validation. Children do that a lot.
Read the blog post. She demanded more of things she enjoyed rather than what I wanted her to do (finishing her home work). That is not seeking validation.
> If somebody hadn't (brutally) drilled the concept of numbers into her skull, the sequences of random symbols would not mean anything to her.
No, nothing brutal needed to drill the concept. You can teach very gently and make it fun.
> Engaging with your child, who has already been partially socialized, is not proof that this socialization wasn't necessary. For that, you would have to have locked your daughter in the basement for those 7 years, and then find that she was spontaneously able to do multiplication
Strawman. I did not say socialisation was not necessary, I said it did not have to be brutal.
> I hope you understand that _brutality_ as spiritual. It's the brutality of going against nature, of putting a monkey in a suit and locking it in an office for 8 hours a day to feed itself.
Who gets locked in an office?
> but they sure as hell don't learn it from eating candy and watching Minecraft let's plays.
That would be neglect. no one things complete neglect is bad.
> I hope you understand that _brutality_ as spiritual. It's the brutality of going against nature
nature is a lot more brutal than modern life.
> You are just refusing to take experiences that do not match your preconceptions.
I am both validating your experience, and rejecting your analysis.
> Who gets locked in an office?
If you can't engage with that statement in more thoughtful way than that, we have no common ground for a discussion.
> I am both validating your experience, and rejecting your analysis.
My analysis is based on facts and observation.
Yours is based on what exactly? It is contradicted by the observed facts.
You seem to think kids always do what their parents show they want them to do, which is detached from reality.
Salman Khan of KhanAcademy many years ago suggested that study should be done at home and homework should be done in-person in class with the help of a teacher. That would solve a lot of the homework problems AI access is causing. But it would require extra effort on the part of the teachers.
https://en.wikipedia.org/wiki/Flipped_classroom
I was at EduLearn (a major edtech conference) this year and talked to many professors exactly about this. A lot of them are pulling work back into the classroom, but keep running into the same problems:
- some skills only develop through the actual writing (similar to how you can't learn to code just by watching YouTube videos)
- in-class assignments are not always scalable (and students end up using AI anyway)
- students aren't motivated enough to participate actively
I think looking at the writing process is a more realistic approach, so I built Turingo (https://www.turingo.net). It replays how a Google Doc was written, highlights unusual edits (large pastes) and tracks how that text changed afterwards. Surprisingly, several professors reported that students often underestimate how much LLM-generated text is left in their final drafts, so just seeing the replay changes the conversation.
It's not meant to be a verdict machine like the existing AI detectors (another big problem in education), more of a starting point for the kind of conversations the author's TAs are having. In fact, we're also beta testing a feature that suggests a few questions for the professor to ask based on the replay.
And yes, it's not perfect and autotypers do exist, but they're dumber than you'd expect and nowhere close to mimicking a real human (I hope to address that soon).
It sad that it has come to this, but your approach is correct - it is the act of writing (code or prose) that is important, not the finished output itself.
Interestingly, this applies not just for learning, but also to a great extent in production/work contexts too.
> I now shifted to in-person interactions with a TA. After every assignment, each student needs to schedule a 15-minute meeting with a TA to answer a couple of questions in a live conversation
Isn't the answer the 'flipped classroom' - students are assigned reading/studying to do before a class and the class becomes more interactive, answering questions/discussing topics/solving problems etc depending on subject. Of course, this is more expensive than a hall with 400 people listening to a lecture with a couple of tests/essays.
https://fltmag.com/the-flipped-classroom/
https://en.wikipedia.org/wiki/Flipped_classroom
This only works if you either have very intrinsically motivated students or you verify that really everyone has actually read the assigned materials before class. Otherwise, some people will turn up unprepared, you will have to explain stuff they should have gotten from the reading assignment, other students will be annoyed, students will stop preparing for class, vicious cycle.
And then you fail them. I suspect an institute genuinely empowered to do so, where the expectations were declared up front and followed through on consistently, would fare well, after the dust settled.
As far as I can tell (based on university and high school level instruction forums) institutions increasingly are not backing the teachers that want to let kids that have earned that F fail the students.
I've heard of places where the minimum points for an assignment is 50% even if you never turn it in which is insanity.
FWIW, this isn't a problem at my German university.
This is for American schools, unfortunately. And it's not a good development.
It stems from the idea that "no child left behind" is a goal. It shouldn't have ever been.
I don't fail them usually - I expose them; and once I asked a student to leave the room because they were blatantly unprepared despite me warning them before.
My main point is: flipped classroom works great in some contexts but is no panacea and requires setting clear expectations.
Aside: as English is not my native language: "you fail them" can mean both "you give them a failing grade" and "you fail to provide the support they need", right?
> "you fail them" can mean both "you give them a failing grade" and "you fail to provide the support they need", right?
Yes, but as written the latter would not be a reasonable parse. For that sense: “you have failed them”, sure; “you are failing them”, sure; even “in acting so, you fail them”; but “and then you fail them” as an entire sentence would be very weird. The emphasis I placed on “fail” also doesn’t feel particularly compatible with that sense of the word.
As a former flipped classroom student I can attest to the effectiveness of "exposure". It requires some skill to pull off: the first time someone is exposed it should be very lenient and then get gradually harsher. The risk of embarrassment in front of the class can be a very strong motivator if utilized right.
[dead]
I've been in uni classes which used this concept (about 10 years ago though). It also requires that content is given in the right size and complexity each week, otherwise there will also be people coming to class who did do their homework but didn't understand it properly and just like the people who didn't read it need full explanations.
I also felt having multiple explanations (in a regular class where you could also read the book in advance) from regular lecturers helped me in gaining understanding because having something explained in multiple ways made sure there would be at least one way, or a combination, which made it 'click'.
I always liked the regular lectures where you would be required to do some pre class reading and a smaller set of exercises the most, as you would be engaged in the class itself and would be getting repetition with different explanations.
Flipped classrooms are like, the quintessential siren song for teachers. EVERY teacher dreams of a day when they can focus on what they're best at (teaching deep topics, getting 'aha' moments, making connections, asking questions, and so on) and do less of what they dislike. But sadly this is just not how things work.
Flipped classrooms have incredibly weak causal evidence. However they are popular in spite of this - perhaps because of this if you're cynical - as the educational research field is pretty questionable in its standards and the profit motive is a strong thing lurking in the background. Wikipedia has a big warning on the article for a reason, but even then it has serious issues.
"Active learning" is a thing (insofar as it has meaning rather than be a useless buzzword catch-all) that does work. But the "flip" specifically does not, or rather, sometimes a mix of extra effort on behalf of the teacher and other loosely related effects occasionally can make a modest impact that makes up for its failings.
Most of the people pushing flipped classrooms are the same people who think that students learn best when they "discover" concepts on their own, maybe with a bit of guidance. This is the other, more serious, lie in education right now, akin to the whole idea that you didn't need to teach phonics to kids when they learn to read.
One of my colleagues tried it for a few terms. Unfortunately, it didn't work well. Most students don't care about learning, they are here for grades :(
> Most students don't care about learning, they are here for grades
It’s a virtuous circle though, as many universities are there for the money, so it’s a beautiful exchange.
Comments like this are why HN needs a "you beautiful snarky bastard" feedback in addition to a simple up or down vote.
I cannot shake the feeling that we are chasing the wrong goose.
The end goal of education is to help you understand how things work, with problem-solving being a means of developing and applying that understanding. LLMs shortcut the former and turn the latter into a pointless arms race that we cannot win.
I loved the example of increasing the complexity of learning projects, but I still feel like one question is missing: are people really trying to learn the same thing we were trying to teach them in the first place?
I have a fairly long experience of both teaching and grading, even though I stopped a couple of years ago. After a couple of years of trial-and-error, I think I found the "right" way:
* Alternate between theory and practice in the course of a single 2-hours slot. Theory is necessary, but the attention span of people is very limited. * Grade either a homework assignment in the form of a real project with specifications. You give the assignment half-semester, so that students who start early can ask questions. Otherwise, grade a in-session exam, with every resource available, including the course and the whole Internet. However, you don't assess the knowledge, but if the student is able to apply their knowledge to the different tasks at hand.
These approaches are now completely moot in the age of AI.
I was surprised the first time a colleague asked me to do an oral exam. I thought it was a burden on people with poor social interaction skills. It went surprisingly well: you could see in a couple of minutes if the student had integrated the concepts of the course, even if they were shy. Now I wonder if it's the way to go. There's one caveat, though: it doesn't scale.
I was thinking that one of the great ironies of artificial intelligence is that it will make good teaching even more labour intensive. Those kids that can afford the one to one tuition required to push past our natural inclination to be lazy will learn things, while everyone else will succumb to outsourcing all their thinking.
There’s no way I’d have learnt everything I know now about software in today’s environment. So much of software development - debugging techniques, architectural decisions, structuring data etc is learned through trial and error. As the models get better is becoming easier and easier to just not look too closely at their output. I suspect it’s human nature.
The oral exams scale better than one might expect, at least if you're not just doing multiple choice exams and you have to actually grade the exam. The time spent on grading the exam could have also been spent on an oral exam... For bachelor level it's usually 15-20min (usually quicker for the well prepared students), and for master courses it is 30min.
Usually you know after a few minutes if it's going to be a fail, and then otherwise you only need to figure out where on the passing scale the student ends up.
> and you have to actually grade the exam
That's the point. As a professor you're allowed to outsource the grading of written exams to you assistants (ongoing PhD teaching stuff). But you must do an oral exam personally.
> usually quicker for the well prepared students
That's true. More time is needed only in case the answer is missing and you have to check whether you hit accidentally the only poorly understood material piece or there are more troubles.
> Alternate between theory and practice in the course of a single 2-hours slot.
2 hours is a lot! In my universities, classes were either 50 minutes, or 70 minutes. I could handle 50 minute ones just fine, but would often have trouble with challenging courses that had 70 minutes. Particularly for stuff like math, I hadn't properly digested the material in the first half, and now he's already proving theorems with that material in the 2nd half. I can follow the logic, but the ideal is when you can digest the prior material before being exposed to theorems relying on them.
Even when alternating theory/application like you say, the amount of attention I'd give to the application in the 70 minute class was less than the 50 minute one. I could follow, but I wouldn't think critically.
The country of Argentina, despite its crippling debt, is able to provide free university education where the majority of classes for most majors are graded with an oral exam.
They've shown that it is entirely possible to scale a system like that, as long as the society sincerely values the role of an educator.
Oral exams don't scale at the beginning with hundreds of students per course, agreed. So make them in-person written exams, German universities managed that just fine a few decades ago. Oral exams were reserved to advance from Grundstudium (basic) to Hauptstudium (main).
In my very limited experience (I have German major degrees in Informatik and Philosophy, I was a TA for Logic in Philosophy) the more advanced the topic the thinner the attendance anyway.
They do scale if the examiner is an LLM.
The article mentions study content that AI has got confidently wrong. That’s going to be some crap markings.
If the ai is good enough to do the homework, it should be good enough to check the homework.
I mean some people are good at Tetris and some aren’t. Tetris can tell you which ones are which.
That’s the problem though - the article has an example of the AI screwing up the homework and the students don’t notice even when warned.
It scales just fine. Use an AI to do oral exam (recorded). Flag the problematic orals and reevaluate the scripts/recorded session.
Finally, if a student feels their result is unjustified, reevaluate the recorded session.
Use more AI to solve the problem of AI?
20 years ago my department head was saying homework was pretty pointless because everyone was copying each other. They assigned a token percentage for homework so at least the students were submitting homework.
We still have developers because we can't outsource responsibility to AI. If shit breaks, we still need someone to fix it. Maybe you think that can also be automated. I have a counter-example. AI will rarely use the existing state when solving a bug. It will tack on it's own event queue, it will make up it's own messaging system, it is a cancerous growth. And this observation is based on both opus5.5 and gpt5.6-astra. The results are way past magical at this point, but you still have to weigh in pretty heavily for orderly code growth where solutions are succint, focused and preserve the existing structure. They are still noobs at fixing/adding stuff.
This makes me think of exams — I always did well on open-book exams, but I didn't do well on exams where I couldn't look at the book. Whether it's because of ADHD or just a lack of memorization ability.
Similarly, I did well in interviews where I could use the internet, but I didn't do well in interviews where I couldn't use the internet.
Everyone has a testing format they're good at. I'm curious about how this kind of thing is evaluated, and whether there are any papers related to this.
> I do not believe that suggesting using low-cost open-weights models would be a fair alternative. (...) We now mention the expectation to use a commercial subscription in our syllabus.
I think structuring a course to be taken using a frontier model subscription is outrageous. If you refine the assignments each semester to be only solvable by a frontier model, your students will always be exposed to the actual pricing of said models. It may be 20 USD/month for now (which is already too much to ask in a syllabus in my opinion), but what if prices increase; where do you draw the line? What's next -- make the students pay for cloud credits in DevOps courses?
I agree with the need to increase the weight of oral examinations / interviews with TAs though.
If the current state of education is students having LLMs do their work and teachers grading assignments with LLMs, we might not need either participant soon. What is the value in doing things this way?
Jobs and daycare.
Case in point: If none of us have jobs any more, we also no longer need daycare. Problem solved!
> Similarly, I always provided a safety net where students can make mistakes and resubmit a limited number of assignments to regain lost points (a core recommendation of specifications grading and grading for equity to focus on learning outcomes, not the process), but we felt that this process was abused with AI: first submit a generated assignment solution without thinking and only look at the issues raised in grading for a resubmission (the typical story of externalizing the cost of AI use).
This is how we do PRs now.
Long before LLMs the couple of actually good teachers I had gave very little (if any) homework. Instead their lessons were more dense and exhausting, but once it was over it was over and we otherwise mostly had to prepare for exams.
The education systems were already hopeless before LLMs and now they got caught with their pants down. Of course some problems were unavoidable, but this overreliance on homework was always a problem in my eyes.
> For example, the evidence favors frequent low-stakes assessments with feedback (e.g., homework, quizzes) over few high-stakes ones (e.g., exams) – but AI is undermining practice in low-stakes settings and pushing us more toward exams.
Why would AI undermine quizzes? I took an intro to Japanese class where we would have a quiz at the beginning of each class which I quite enjoyed. I can’t see how AI would change the utility of that.
I teach mostly smaller courses but my approach to grading has been the same for years: A combination of a semester-long group project (ending in a paper, a prototype and a group presentation) with individual oral exams about aspects of the project and of the lecture. The lecture is flanked by optional TA sessions as well as access to labs which, for most hours of the day, have some experienced/teaching people being around (this results in a lot of ad-hoc learning situations and a general diffusion of knowledge and practices).
This gives me/us a lot of angles to understand what progress has been made throughout the semester as well as to consider and react to individual differences between students. In my book, it’s also resistant to faking it convincingly throughout all modalities. However, scaling that approach to big entry level courses would require more money for teaching than any research group in Europe I’ve ever seen has available.
I dunno, if I had to teach students now I think I’d just force them to stay in the lecture hall without internet for the hour I need them to work on their assignments that week. I just can’t see any other reasonable way to do this. Everyone leaves their phones at the door in some designated locker or something.
> "Everyone leaves their phones at the door"...
— watches, calculators, discrete technological devices. I've seen kids encode text as patterns of dashes on their desks, sharing the cipher solution with their desk mates in other periods.
The problem isn't skirting the effort. The problem is systemic. Why does the system reward taking shortcuts? I think it's because most areas are focused on short horizon tasks, min-maxing effort:output. It's not a problem with those technical fields. It's a problem with the current crop of the C-Suite across the entire occupational plane. They're short sighted.
Why enforce effort rather than outcome? Just have exams be without phones.
If someone manages to learn without doing the homework, or finds some way to learn better using an LLM, fine. The exam will show that.
Because we're afraid students might lack self control, or be unable to evaluate how well they absorbed the material.
> Minor note: We observed that some students used AI during live discussions over Zoom (e.g. Cluely)
I frequently see this in remote interviews for senior developers. These are conversational interviews, not coding tests, but some candidates seem incapable of having a conversation without AI.
I can't imagine how these candidates would be able to meaningfully contribute to a whiteboard design discussion with real people.
For a previous client, we resorted to a simple in-person pen-and-paper screening questionnaire with 5 very basic questions. Many candidates declined to attend, and most of those that attended failed dismally.
Thanks for sharing. Interesting to see how teachers are adapting to LLMs. I agree that trying to ban AI use is futile.
It's worth noting that the he's not afraid to us LLMs when it cuts the teacher's workload. The Slack bots are all automated.
I generally think this is okay because the students are getting more feedback, but some students are starting to call this out as hypocrisy. At Dartmouth, there's a debate running because the Provost was using LLMs to improve his writing. He's not in a class. He's not breaking any official rules. But somehow people are feeling like he's cheating.
I see reasoning models being used to change the way to draft assignments and formulate teaching. Accelerating the learning gradient for students now requires a better understanding of how they learn and working with their motivators.
E.g. if students learn better through gameplay - can assignments become like game levels?
Rather than building for verifiable outputs - build assignments that gauge process and help elicit what they have learnt not what they have produced.
AI is effectively a personal tutor. Personalized tutoring is the best possible way to learn!
The problem is that the student is in charge of the tutor and often tells them to just do their homework for them and not check whether they have learned anything.
LLM providers should formalize a tutoring mode. We should have prompts available to create this mode. This should be the default way students use LLMs. LLMs may not be good enough to do this now, but I think there is already enough raw intelligence and its a matter of training models to be good tutors, to be experts and specific subjects, and developing software systems for tutoring.
In this model there are few lectures. This is more similar to Montessori but with on demand tutoring. The teacher provides objectives and materials for the students and tutors. The tutoring process can be done during class for teachers to ensure it is working well. There can be larger blocks of office hours for students to do their tutoring during.
As AI gets more intelligent, it will also be capable of doing all the grading. This includes oral exams. Teachers need to proctor the exams in person.
Anyone will be able to tutor themselves on any subject. Proctored examinations to certify knowledge do not need to be very expensive. The value of the school environment will not be to transfer knowledge from a teacher but to have peers working on the same thing and teachers and an environment invested in training the next generation.
Incredibly insightful and makes me feel bad for those who have to put a lot more work to just maintain basic standards (Teachers, TAs etc.).
It was also interesting to see that everyone is now forced to use LLMs to do the assignments. I wonder if students are using any kind of subsidised access to LLMs here.
If this is the case, then this education is largely done.
This means that education needs to have a high teacher to student ratio. This is not feasible in any society currently on this planet, because we simply cannot afford to have that many teachers per student.
Think of education as a wishing well. There is a wishing well for math, law, dentistry, baking, and so on.
Once you have a wishing well - i.e. you own an eduction in baking or dentistry or law or math or whatever - you can ask it anything and it will grant you your wish. If others want something from your well, they have to come and ask you. What you give them has your name attached to it. So, your reputation rides on the quality of what your wishing well produces.
Prior to AI, universities chose which wishing well they would specialize in building and offered to share that knowledge with students.
In the age of AI, the creation of personal wishing wells has become trivial but universities still think they are in the business of teaching people how to create wishing wells.
Ultimately, the only truth left is for owners of the wishing well to start offering the product of their wishing well to the market and attach their name to it. If there is a market for their product, great. If not, go make another wishing well.
Education used to be pets. Now, it is livestock. Universities are not willing to accept this shift in mindset.
> more importantly I do think that students need to learn responsible use of these technologies anyway.
But you're not doing that, you've just given up and allowed any use!
> How to use AI coding agents effectively is not a learning goal
Exactly
> violate evidence-based best pedagogy practices, and I made them anyway.
Of course you have, when have educators cared much about best practices
> We have since built infrastructure to autograde code and written reports of homework with LLMs (institutionally approved LLMs)
Further reducing the point of your own existence
> Debriefing after such a failure is a good learning opportunity to talk about automation bias and different forms of human oversight (and how this is really hard in practice).
It would be if there were any learning, but since the incentives are for more AI use and less learning, there will be no application of the supposed learning, so you'll continue to have those 80% fails
> Of course you have, when have educators cared much about best practices
Why do you say so?
This is not at all my experience, as a recent teacher it strikes me how much the teaching methods are supported by science.
that just means home work needs to become class work and in session. Activate a network jammer(or just collect their phones) give the class couple hours of isolated time to work on the low stakes problems relevant to the session to gather calibration data points. This also means you will need personalized pace of studying/teaching. Probably studying at home becomes the norm with personalized AI tutoring and class work becomes the calibration point to tune the next lessons and socialize with other students.
I find it offensive that your blog is named after thelastpsychiatrist when the writing is not even close in quality. Come on man, I expected you to start swinging. Are you building an agent tunnel? Get outta here.
I don't like it. Now, the TAs have high cognitive load. Earlier, you only need to know the material of the course. But now, you're forced to navigate AI output. And you got to keep with all new model developments.
Make all reports and assignments open source and public. Their credibility as students and professionals will be on the line. It will be obvious who puts in the effort and who doesn't
building a community around open source is hard. building one that analyses, comments, and votes on contributions is harder. who is going to read and react to the open sourced content that students submit?
In my experience a significant proportion of "evidence-based best pedagogy practices" is hogwash. I'm pretty anti-AI, but going against education-school dogma isn't a reason to avoid AI.
Why do you say so? What is your experience?
I recently did a course on teaching bachelor-level education, and had to study a couple of books. They all seemed to quote their sources and I got the impression that they're rooted in science.
It's science-based in a sense, but a pretty weak sense. I did some work with education researchers and my impression was that their methodologies were quite haphazard and they were sometimes just trying to find a way to "tell a story" that they already had in mind. Also, my impression is that a lot of the research on "best practices" is based on relatively limited comparisons between schools or teaching methods, and there is insufficient evidence that this or that is truly a "best practice" in a more general sense. I also interacted a good bit with education grad students when I was in grad school and they almost uniformly seemed to have no coherent sense of what they were doing or what their field actually knew. I've known people who had similar experiences with education schools elsewhere.
I've also read news stories and stuff that support the idea that a lot of "education education" is of questionable value. This story is pretty old by now but has stuck with me: https://www.latimes.com/archives/la-xpm-2010-may-10-la-oe-zi... . A relevant quote:
> Of the 1,300 institutions awarding graduate degrees for teaching, Harvard’s director of teacher education told a 2009 conference, only about 100 prepared students for the classroom. The rest, she added, “could be shut down tomorrow.”
Some of these experiences/articles are about teaching K-12 schools rather than teaching college, but there would have to be a pretty huge difference between the two for me to think that college-level pedagogical theory was meaningfully better than that for younger ages, and I've seen nothing to give me that impression.
I’ve put that article on my reading list. Yes, I think it’s true that there are situations where the science is used to support “telling the story”. As a counterpoint, I got taught about the “illusion of learning”, where it’s pretty clear that a regular college lesson (teacher droning from a PowerPoint), retention is down the drain. So we teachers know that a good curriculum should incorporate repetition and exercises.
Now the question is, are these curriculi being rebuilt? Here in The Netherlands, yes.
If we're talking higher ed., expell students that use LLMs to do their assignments for them. They obviously aren't there to learn and don't belong there.
Heh, dumb idea of the day: do a "flipped classroom" in a different sense -- instead of a student taking a quiz or exam, you provide a clueless AI and the student has to train it on the subject at hand, and then the AI takes the test.
Lest anyone think this is a brilliant idea: an LLM would likely be much better at doing the ~~RLHF~~ RLMF, so you haven't actually gained anything. But I feel like there may still be the kernel of a good idea somewhere in here.
Perhaps your assignment is to iteratively train the AI, which is pre-prompted to seek out every reasonably possible failure mode when it generates solutions. So the learning mechanism is to train an adversarial AI on the subject matter as a way of learning it yourself. This does not solve the problem of cheating with an LLM, it's an alternative pedagogical approach.
The most interesting part here is that AI isn't replacing the learning goals, it's replacing the evidence instructors used to measure them
I was an (adjunct) prof at Stanford for over 20 years. I taught the top level interaction analysis course - basically cognitive science applied to human interaction with complex systems, including AI based system. As soon as I saw students turning in AI papers and projects, I knew all of upper education as we know it was fucked. I retired. Education is fucked, but learning isn’t! This will be a golden age where you can learn anything you want to. If you don’t want to learn anything, you’ll be a meatbot working for those who do. Many people may be happy in that role. I know someone who is extremely happy working at as a barista. He was also happy having gone to college for a useless degree. He’s happy he did that. He tells me the things he’s learning on his own every week from AI tutors. Quantum Mechanics. People will find a way to make their way, mostly. Education as currently organized is fucked. Teachers should be there to help students learn what they want to learn (at least after elementary school, which is still important and won’t be disintermediated). Like flight instructors. People who want to learn to fly learn most of it on their own, and the flight instructor is there to help (and keep you safe at least at first, until you solo). People seek out flight instructors, and will continue to do so (at least until airplanes fly themselves - but that’s a different discussion). Teachers will become tutors. And I dare say that teachers will like it better than their current role of having to force-feed 2/3rds of a class who doesn’t want to learn. Instead they’ll enjoy helping the few students who actually care to fly to fly.
AI is great for those who want to learn. Earlier, one has to find all relevant books, search specific forums, or use google in order to find answers to questions that teachers or fellow smart-students find silly. Even if AI gives false answers, it is better than having no answer. At least, the search space to find an answer is reduced.
In the age of AI with Astra and Fable capabilities, there is little choice.
This is the way.
Is society really trending in the direction where learning to do homework assignments is meaningful?
I think society is moving back toward nature at rapid pace. More adversarial in general, and advantage taking behavior is the way to survive if you don't have a solid community. Society will be culled to a much smaller percentage of people than are present today, and it probably won't be pretty. Smaller communities instead of larger communities, with self-sufficient systems in place and a lot of warfare...
Don't see how AI overlords controlling everything will work out. If no one can buy the food from the large producers, then they will dissolve too. Supply chain will fail at some point in the next couple years maybe? Diesel is already outpacing wages for truckers.
The people with all the money are positioning themselves to reap the assets from others to give themselves as much leverage as possible.
At this point I don't really see how it could go any other way.
I don't know. I'm trying to figure out what's going to happen, but it doesn't really seem like anything good is going to happen because the trend is wanton disregard for morality in exchange for stealing wealth.
Makes me really sad. I'm also on fixed income so I worry quite a lot about my future.
Why teach someone who doesn't want to learn?
Why should we make someone work who doesn't want to work?
We don’t
very simple. All exams to be paper and pen.
Thinking too small. The interactive testing needs to be done by AI. Then the transcript is reviewed by the teacher.
School homework is one of those things which, like the OnlyFans economy, I hope AI utterly destroys, despite my general detestation for the corrosive effects of AI on society.
You're not learning anything if you don't put in your hours. Maybe you're fine with that but make sure you realize what you're getting into.
This seems much better than hand-wringing and woe-is-me that we usually see. Good job.
[flagged]
[flagged]
[flagged]
[dead]
[dead]
[dead]
Just grade the process, not the result. It's literally that simple.
AI can talk the talk, but it can't walk the walk.
A simple "explain why this is the best solution" will make it shit the bed 10 out 10 times.
[flagged]
> For example, the evidence favors frequent low-stakes assessments with feedback (e.g., homework, quizzes) over few high-stakes ones (e.g., exams) – but AI is undermining practice in low-stakes settings and pushing us more toward exams.
Only because you're obsessed with being able to assign grades and fail cheaters.
Or possibly because quizzes and homework are a good way to learn the material, but if students have the AI do it instead they won't learn.
No obsession required.
What is the point of learning something an AI can already do better than you? There are better methods of brain training.
The situation reminds me of what my teachers used to tell me. "Learning how to look up things in a book will be a skill you'll need all your live and you won't carry a calculator with you at all times when you are grown up." Both assumptions were proven wrong before I was even out of school.
If the only remaining value of education is "to become a well rounded person" then why waste your time on it? It used to be you went to university because that was the only place to get knowledge from. Which is also why it was dominated by the rich and powerful. These days university feels a lot like paper money. Everyone just pretends it has a real value and because everyone agrees, that's where the value comes from.
Okay why do we need literacy if we have such beautiful screen readers? Maybe because it's better when you actually absorb these fundamental skills rather than rely on a disability aid to relieve you of a self imposed handicap.
Practicality. It's more convenient to read door and street signs, restaurant menus, name badges, ticket machines etc. etc. without going through a screen reader.
It's little effort for a large benefit. But anything you learn past middle school in formal education is a moderate to large effort for a small to neglectable benefit. And you'll forget it anyway if it is uninteresting and not useful in your daily life.
Anything interesting or useful in your daily life, you'll pick up by yourself even without formal education.
This same logic applies to all the upstream education. It's convinient to have a more limited calculator in your own brain, or better yet, the ability to simply see numerical properties factors, in a way a calculator doesn't replace. It's good when you can yourself understand the physics of everyday systems, or perform useful calculations about them. And if your work is indeed tied to it, it's good to know basic semiconductor physics, even if the ordinary person has a million more immediately useful things to study.
Knowledge is not a tool for answering the absolutely minimum questions you encounter in daily life, but a way of seeing the world. Reading allows you to use language in ways you could not imagine with spoken word alone, just as math or physics or computer science enable new ways of thinking inaccessible without intense study.
I don't disagree that this is all useful to have. My point of contention is that the effort required is not worth the time investment if you are going to forget about it in a few years anyway because you never needed it in your daily life.
Our brains can't hold infinite amounts of data and a lot of information will become inaccessible if not used regularly. This is also why most of us do not speak twelve different languages, despite this being a supremely useful skill that is relatively easy to pick up compared to semiconductor physics.
The students who want to learn aren't having the AI do it for them. And they will learn better in the evidence based system.
The ones who don't want to learn will cheat themselves of an education. This is their own problem and we shouldn't punish the actual students to catch these others.
Failing to catch the cheaters does punish the honest students, who in your approach get a worse GPA.
This is exactly what I said. The obsession with grades (GPA) over education is the cause here. Intentionally making the education worse in order to protect the grading system.
It's funny how people talk about "teaching" getting more difficult with LLMs.
Clearly teaching got a lot better, since we now have a new teaching tool at our use, the LLMs. It's evaluating that got harder, because students can use the same tool to cheat in traditional forms of evaluation.
If "teaching the students" means the same as "evaluating the students" to you, maybe you shouldn't teach in the first place.
Teaching is imo mostly about techniques that battle laziness. Teaching intrinsically motivated intelligent students has always been trivial. All progress in education is about scaling it to work for people who only reluctantly engage with the material.
Right, and part of that is nurturing student interest and buy-in, because it can increase. Traditionally smart teaching and the right structure could result in some percentage of reluctant learners being drawn into good habits and better retention. I think the subtle part of the AI in education crisis is how it basically has smothered that "nurturing" in the cradle, right from the get-go, so it's a bit of a cold-start situation in many classes.
It’s not just evaluating. Students used to learn skills while doing an assignment. LLMing through assignments teaches copy-paste at best.
It’s like copy-paste Wikipedia presentations 2 decades ago. People used to learn things doing research for a project. Then suddenly it became copying off Wikipedia and people not even pre-reading what they copied into PPT.
Teaching and evaluation are of course not the same thing, but evaluation is an important part of teaching. There is a mismatch between students’ overall desire to learn things and the moment-to-moment experience that learning is hard.
Even devoid of all the economic and social pressures that grades impose students would compromise their learning experience. Ever regretted looking up the solution to a puzzle in a video game or sneaked a peak at the crossword puzzle solutions?
One of the hardest part of teaching is keeping students from self-sabotage. Evaluation helps with that.
Beyond that evaluation is of course also useful just as feedback.
What you’re calling evaluating is a part of the learning process for the student, not just about confirming that the student knows the thing. LLMs are actively undermining learning in this way. There was useful friction which has been removed.
Without grading, students don't learn. It's as simple as that.
That outlook is way too bleak. As a person who has taught at universities for fifteen years and has run lots of open educational formats and infrastructure with other public institutions and citizen groups, my experience is the opposite. If anything, the desire to learn and curiosity are among the strongest common traits humans possess. Grades are just a tool and a proxy that most people working in formalized educational settings have to work with at specific points.
Motivation can come from many sources, not only grading, e.g. individual encouragement, exciting applications, day-to-day relevance, interest shown by peers. Also students can just be intrinsically motivated.
It may come in different styles. But what do you do with students who don’t have motivation beyond grading/punishment? Some people just do not have motivation unless somebody standing by them with a stick.
Pretty strong opinion that overlooks all those students that study or attend courses specifically to learn.
Of course what's interesting is that when viewed from the perspective of 'I have a kid and I'm willing to orchestrate their education myself', AI means the kid can get an education unheard-of even ten years ago. Personal, genius-level tutor, expert in any subject in which the kid can muster a question.
The only people really having a crisis here are teachers who want to feel relevant.
Why is that interesting? How often does that happen? Has it happened even once? Seems like fantasy; this has been possible for a long time, and despite it being possible, it almost never actually happens. Before AI, kids willing to orchestrate their own education could use the internet to become experts in anything. Before the internet kids could use libraries and books to become experts in anything. But how many take the time and effort to match a 4 year degree? Statistically, approximately zero.
One major problem: how do you get a job for which a degree is required, if you teach yourself via books/internet/AI? How do you get into graduate schools for which a degree is required? How do you envision advanced degrees working without funding or labs or mentors?
All of education is undergoing dramatic seismic shifts because of AI right now. Attempting to insult teachers for talking about it doesn’t reflect on teachers, it reflects on you.
It's interesting because it clarifies that the problem here isn't the kid's education; the 'problem' is that teachers and colleges are upset that they are no longer offering something valuable.
The kids are fine; the rent seekers aren't.
And to your larger point: the entire university/graduate/lab/scientific system was created to formalize what was originally citizen scientists trying to understand the world from their basements. Scientific journals are just a formalization of nerds writing letters to each other about what they figured out. Consider from first principles how much of what exists now actually still serves that original purpose.
You want it to be a jobs program? Sounds good. Take their computers away and make them use pencils and you can create even more jobs.
You want to do science? Figure out how to do science. You want the kids to learn? Figure out how to teach them.
Just don't protect a jobs program under the guise of something nobler.
I disagree that’s the problem, and I don’t think your claim is true. Degrees are still extremely valuable, and income stats alone prove it conclusively. I don’t find your suggestion very interesting, not because colleges are gatekeeping, but for the reason already stated: because most people do not and cannot teach themselves and curate their own educations at a university level, and because even if they did, which they don’t, it still wouldn’t help them get jobs or advanced degrees in today’s workforce. It might be an interesting suggestion if that happened with any statistical significance, but it doesn’t, and isn’t a realistic path for education for society.
Figuring out how to teach kids is exactly what this article is doing. How do you propose to teach kids, when they refuse to put in the work, and reach for AI for everything? Your suggestion to make them use pencils seems naïve/ignorant/obtuse; this article is written by a computer science teacher. You can’t force people to use only pencils for homework (as the article points out), and it would be ridiculous to use pencils for writing computer programs in 2026. That suggestion is a non-starter. FWIW, the impression I’m getting is not that you have any helpful suggestions for education, but that you don’t have experience teaching and don’t understand what’s happening in schools right now.
Who is protecting jobs programs under the guise of something nobler? This seems like a straw man unrelated to your previous comment. I’m getting the feeling you’re anti-education for some reason, maybe have some bone to pick with the university system, is that accurate? Asking kids to teach themselves or forcing them to use pencils are not going to be successful ways to improve education.
You're making ad hominem comments about me; your argument about the degrees is making some heavy assumptions that such things won't be massively changed by AI.
And you're taking the pencil comment weirdly literally, like it apparently went completely over your head. We're clearly talking past each other. I don't see any purpose to talking about this further.
Things are being massively changed by AI. That’s what this article is about, the article you seem intent on poo-pooing. I only asked you a question about your stance, it was not an attack. My question was aimed at trying to uncover why you’ve cast insults toward teachers multiple times, and implicitly the one who authored this article. The only ad-hominems were yours, and they started with your first comment.
What heavy assumptions are you referring to? If you mean assuming that kids probably won’t skip university and instead will be self-motivated enough to use AI non-stop for several years in order to teach themselves enough to get a job as a biologist or a ballet dancer or a civil engineer, and if you mean assuming that companies and organizations probably won’t suddenly start accepting self-AI-taught candidates as being the same as having a degree… if these are the assumptions you’re referring to, then you’re right - I am assuming those things and I believe them. No matter how AI changes things, it’s a pretty heavy assumption you’ve made to think kids will self-motivate an education just because information is available, especially given that was always possible and still chosen so rarely that it effectively doesn’t exist. And it’s a pretty heavy assumption to think that universities and organizations will change together in a way that can vette the merit of prospective candidates any time in foreseeable future. AI will change a lot of things, but you’re underestimating our social structures and what things like the accreditation system do for the workforce and the economy. You can argue it’s got problems, but assuming like you did that AI can replace it anytime soon is a very heavy assumption.
If the pencil comment and suggestion to take away computers wasn’t literal, then you’re right, I have no idea what you meant. Feel free to explain. If you can’t, then I’d agree, there’s no further purpose.
Replace the profs with ai. Force every student to do a three hour one on one interview test with the ai prof to demonstrate mastery and get a final grade. Have other ai proctor. Replace homework with similar “ai sessions” where the student is guided by ai and given specific exercises tailored to the students learning trajectory and personal profile. Students can and will learn better with ai “professors” than they do now. The problem is that higher education is not about learning ultimately. The problem is ai in education is that ai exposes the lie, it was never about learning, it was about credentials, money and control.
If my department ever reached a point of such disrespect for their students, I'd quit my degree on the spot. I understand what a university degree is, and wouldn't pay full tuition for this sad pantomime of one.
What an incredibly cynical take. Might as well have people date AI boyfriends and be governed by AI politicians.
> Students can and will learn better with ai “professors” than they do now.
No, that’s not true. You have no evidence to suggest that.
> The problem is that higher education is not about learning ultimately.
It is, actually, and many institutions do a good job of this. Some don’t, but they’re not all just degree mills.
Do you recall universities going after the MOOCs? I do. Do you think students benefit from smaller class sizes and more one on one time with professors? If you ask a professor what qualifies them do they say their research or their pedagogical skills? Do people pursue higher education opportunities at the same rate when no degree or certificate is offered? What happened to MOOCs anyway? If Harvard is so great for the nation, why not 10x its enrollment? I’ve probably thought about this much longer than you have.