Why Good Students Fail: And Why Neither Grit nor Human Tutoring is Enough

For information access, the computer has changed the world. No surprise that education would be changed as well, but not as much as it should be. What follows is my personal story of how that did and did not come about. —Clark Glymour

In Galileo in Pittsburgh, Clark Glymour, Alumni University Professor at Carnegie Mellon University, tells the story of his first year teaching at Princeton University. The year was 1969. Glymour's first year at Princeton was also the first year that the University admitted black students.

Nassau Hall, Princeton University’s oldest building. Courtesy Ken Lund CC BY-SA 2.0

Glymour was assigned to teach introductory mathematical logic to a class of about seventy students. He chose a well regarded textbook by an eminent logician; he crafted a matching set of lectures; and he devised “summative assessments” in the form of homework assignments, regular quizzes, a midterm examination, and a final examination. The students also benefited from weekly tutorials led by graduate students, who were also graders in the course.

Glymour did what every good instructor does. Or, so he believed.

The Result

What was the result?

Clark Glymour, Alumni University Professor Emeritus, Carnegie Mellon University. Source: University of Pittsburgh

One morning at the end of the semester my graders appeared in my office to tell me I had failed every black student in the course, all seven of them. (p.28)

Glymour was stunned. What had happened? Were all black students in the course “bad” students? Reflecting on his course design, Glymour writes:

These had to be pretty smart kids, or they would not have been admitted; they had to be ambitious kids, or they would not have wanted to attend an Ivy; they had to be brave kids, or they would not have wanted to attend a snobbish, traditionally racist college, which was what Princeton was trying to cease to be in 1969. (p.28)

Glymour reasoned that the problem was not the students, but “me or my course or both.”

The Thought Experiment

Glymour then conducted a thought experiment. Suppose that the black students (and others who had failed) were less well prepared than other students in the class. Would that matter? Yes, it would. “The course organization gave students little opportunity to substitute sweat for background; the lectures presumed they had followed everything up to the point of the lecture; if they hadn’t, they would be lost.” Also, suppose that the students were not as good at taking timed tests under pressure: not because of ability, but because they had not practiced in those conditions. Would that matter? Yes, it would. “The testing was one shot each, midterm and final, no do-overs, no mulligans.”

In his thought experiment, Glymour was posing a very basic question that most instructors fail to do:

Does the course design give underprepared students an opportunity to catch up?

On the basis of his thought experiment, Glymour conjectured that his course design had not taken into account the needs of all his students. Glymour hypothesized that in courses like his grit (“sweat equity”) alone simply cannot make up for lack of preparation. The tutorials, led by graduate students, wouldn’t make a difference.

The Experiment

As a good scientist, Glymour redesigned the course from scratch, running it as an experiment.

I found a textbook that divided the material into a large number of very short, very specific chapters, each supplemented with a lot of fairly easy but not too boring exercises. For each chapter I wrote approximately fifteen short exams, any of which a prepared student could do in about fifteen minutes. I set up a room staffed for several hours every weekday by a graduate student or myself where students could come and take a test on a chapter. A test was immediately graded; if failed, the student would take another test on the same material after a two-day wait. No student took the very same test twice. I replaced formal lectures with question-and-answer sessions and extemporaneous mini-lectures on topics students said they found difficult. I met with each student for a few minutes every other week, chiefly to make sure they were keeping at it. Grading was entirely by how many chapter tests were passed; no penalty for failed tests (emphasis mine).

The following semester Glymour put in place his new approach.

The New Result

What was the result of the redesigned course? Although the same number of black students enrolled as the previous semester, this time half of them earned an A and the others all received a B grade.

Glymour believed he was on to something.

I had discovered something. What I had discovered was that a lot of capable students need information in small, well-organized doses, that they often do not know what they do not know until they take a test, that need a way to recover form their errors, that they need immediate feedback, that they learn at different rates, that they need a chance to ask about they are reading and trying to do, and that, if given the chance, motivated students can and will successfully substitute sweat for background. p.29

The Conclusion

Despite his success, Glymour abandoned his new approach to teaching:

I also discovered that it is not physically possible to teach that way: I was ready for the hospital by the end of term.

Glymour’s efforts were also not appreciated, both by the students and the University. He received the “lowest teaching evaluations ever”. And the “Princetonians thought the course was a ruse to save myself the trouble of formal lectures.” “I never taught logic again, anywhere.”

Glymour goes on to note, however, that this type of course was ideal for instruction by computer, especially in combination with human instructors: “The computer could be programmed with small modules, with interactive questions and prompts, with many penalty-free tests for each module, even with automated problem-solving aids.” p.29

Glymour began his career at Princeton, but then went on to Carnegie Mellon University (CMU) where he made seminal contributions in the philosophy of science, statistics, and a number of other fields. In his essay he mentions the pioneering work of John Anderson and others to develop “cognitive tutors”. He also writes about Carnegie Mellon’s Open Learning Initiative, which has realized some of the basic insights Glymour arrived at painfully through his teaching experience.

But five decades later, good students continue to fail. Can the new AI, along with serious course redesign, turn things around? I believe it can.