Accommodation Nation, by Rose Horowitch in The Atlantic
Are Colleges Getting Disability Accommodations All Wrong? Higher ed’s maximally inclusive approach hurts those it attempts to help, by Alan Levinovitz in Chronicle of Higher Education
What do time accommodations do to the predictive value of LSAT scores for legal education?, by Derek T. Muller, Notre Dame Law School
The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring, by Benjamin Bloom in Educational Researcher (1984).
The late Robert Slavin on Proven Tutoring—one of his last interviews on Principal Center Radio
Joe Liemandt of Alpha School on the Invest Like The Best podcast—listen: Spotify, iTunes
Full Transcript:
(00:00):
Welcome to the Eduleadership Show. I’m your host Justin Baeder, and in this episode we’ll talk about academic accommodations at elite colleges. We will talk about Bloom’s two Sigma problem, and I’ll share with you a paper of his from 1984 that’s getting a lot of attention today, and we’ll talk about hype in the AI ed EdTech market. First up, over in the Atlantic, Rose Horowitch argues that America’s colleges have an extra time on tests problem. In an article called Accommodation Nation, she outlines the surge in academic accommodations, especially extra time on tests that’s happening in elite colleges.
And what’s interesting is this is not happening in all colleges, in its especially not happening in community colleges, which is where you would expect students with disabilities to be concentrated. So something is going on here. She says, at the University of Chicago disability, accommodations have tripled in eight years at Berkeley, they’ve quintupled in 15 years at Brown and Harvard.
(01:14):
Fully 20% of undergrads now have some sort of academic accommodations, and at Amherst it’s 34%. So something is going on here and it is concentrated at the colleges in our country that have the best qualified students, the strongest students academically. Why are they getting these accommodations?
Well, it’s undeniable now that there are simply incentives for seeking out accommodations. These are not the kinds of accommodations students got in K 12. These are not students who had IEPs. These are disabilities that students are self-identifying and reporting to the student Office of disabilities so that they can get some sort of accommodation. And I’ve seen some discussion that the accommodation can be literal on some campuses that you do get preferential housing. Perhaps you get to have a pet with you on campus if you have some sort of accommodation for a disability identified. So there are very practical reasons that students are incentivized to seek out these accommodations, and it’s something that they really have to take seriously and grapple with, especially when it comes to competitive things like law school admissions.
(02:22):
One quote from the article that I thought was worth sharing, she says, if the rise in accommodations were purely a result of more disabled students making it to college, the increase should be more pronounced at less selective institutions than at so-called Ivy plus schools. In fact, the opposite appears to be true. According to Weis’s research, only three to 4% of students at public two year colleges receive accommodations, a proportion that has stayed relatively stable over the past 10 or 15 years. And you may remember that Alan Levinovitz had a similar article in the Chronicle of Education last year where he argued that the college disability system was somewhat out of control. And he pointed out in that article that there are real advantages to getting extra time, especially on tests like the LSAT and Derek Muller, professor of law, Notre Dame Law School, said in a blog post that LSAT takers who get accommodations on that test score four to five points higher than test takers who do not.
(03:21):
And as a result, scores over-predict the law school success of students who get extra time. In other words, students are giving law schools a false impression of how well they will do in law school because they’re using extra time to get higher scores than they would otherwise get. And the surge here is dramatic just as it is in undergrad. 6,000 students got extra time in 20 18, 20 19 or requested extra time and 15,000 students requested extra time in 20 22, 20 23. So something big is going on here. I think there’s a concern among people who work in college disability offices that we don’t want any kind of witch hunts here. We don’t want people being questioned about their disabilities. We just want to be able to support all of the students who actually need accommodations. And part of that involves not really worrying about which students actually need accommodations.
(04:14):
I don’t know if that approach is going to hold much longer because as the Atlantic article points out on some campuses, it will not be long if you extrapolate just not too far into the future. It’ll not be long before a majority of students are receiving disability accommodations. So let me know what you think about this issue.
Next up in a segment I’m calling Article Alert. I want to share with you a classic from Educational Researcher published in 1984 by none other than Benjamin Bloom. This article is entitled The Two Sigma Problem, the Search for Methods of Group Instruction as Effective as One-to-One Tutoring. Now interestingly, this article is getting a lot of attention lately because a lot of companies are trying to develop AI tutors that can produce those elusive two sigma gains. What is interesting about this article though, and if you read the article, you you’ll no doubt recognize an approach that John Hattie has continued of reviewing different studies and trying to find effect sizes that can ideally be stacked up to produce large learning gains, right?
(05:19):
If something has a one standard deviation impact, maybe we can do two of those things and have a two standard deviation impact. So I think the intent here is really good. The goal here is really good to improve learning to figure out how to do school in ways that will improve learning dramatically for large numbers of students. I think the issue in this article is that the original research has not held up. It has never been replicated. So we currently do not have a model that allows us to produce those two sigma gains. That finding has just not held up. So the idea that we would build EdTech in order to match those gains in a cost way I think is also not going to hold up. If anyone is going to promise that an EdTech product can produce those two sigma gains, they’re going to need to realize that that would be unprecedented.
(06:08):
We do not have tutoring models or anything of the sort that can reliably produce those two sigma gains. There were some unique experimental conditions, there was extra time, there were lots of things about those original experiments that resulted in them not really being replicable and scalable as solutions to gaps in learning in K 12 education. So it’s an interesting article. I want to encourage you to read it and I think you will recognize, again, many of Hattie’s ideas about stacking different practices. I think we do need to think seriously about tutoring. We need to continue to pursue tutoring as an effective support for lots of students who are behind. But we have to recognize that it is going to take time and we ultimately run out of time. We may be able to devote some money to tutors, but we still have time in the day that we have to provide for that tutoring to take place.
(06:59):
Now, I mentioned that AI and EdTech companies are picking up on this idea of a Two Sigma problem and trying to develop AI based applications that will help students learn much, much faster. There is no better example of this than Joe Liemandt of Alpha School who was on the Invest Like The Best podcast back in August and talked about what he believes is possible with AI tutoring. And of course we have to recognize that AI tutoring is not the same as one-on-one tutoring with a human. There are some big differences and we have to recognize that even human powered tutoring did not actually achieve those two Sigma impacts. But Joe says in the podcast, you can take in kids who are behind and if they’re learning twice as fast, they’re going to catch up. So you can take a kid at the 50th percentile and get them to the top 10%.
(07:53):
Again, all of this is really unproven and I’ll be happy to eat my words if there is proof that emerges that this is actually possible. I think the challenge about catching up though is if you take a kid who is at the 50th percentile, they are at the 50th percentile because they learn faster than some kids and they learn slower than some kids, and the top 10% is in the top 10% because they learn faster. So if you give the same opportunity to all of the kids, if all of the kids have the opportunity to learn more quickly with whatever AI or other ed tech technologies we come up with, well the kids who are in the top 10% currently are going to stay there because they are going to learn faster as well. The only way we can get a distribution, like the very tall squished graph on the right in Benjamin Bloom’s chart is if we devote so much time to a topic that everyone masters it completely.
(08:48):
And again, the challenge here is time we run out of time, we want to move on to other things. There is more to get to, so I don’t for even a moment believe Joe’s contention that you can take a kid at the 50th percentile, run them through an AI platform and get them to the top 10%. I think there is some real potential here. I think Alpha is developing some tools that really can help students who are very motivated and who are at the top already learn even faster, which is not a bad thing, but I think we have to be very careful about buying into the hype. Let me know what you think.
(09:20):
That’s it for this episode of the Edueadership Show. I also want to encourage you to check out my podcast Principal Center Radio, where I interview authors about their books. You can check that out at Principal Center dot com slash radio.
