The Lawlor Review

With the release of their book Academically Adrift: Limited Learning on College Campuses earlier this year, Richard Arum and Josipa Roksa shocked the higher education community with the findings of their study. In tracking more than 2,300 students at a diverse mix of 24 four-year colleges and universities from the first semester of their freshman year through the end of their sophomore year, the researchers found that “for a large proportion of them, the gains in critical thinking, complex reasoning, and written communication are either exceedingly small or empirically nonexistent. At least 45 percent of students in our sample did not demonstrate any statistically significant improvement during the first two years of college.”

Arum is a professor of sociology at New York University, with a joint appointment in NYU’s Steinhardt School of Education. He is also director of the Education Research Program of the Social Science Research Council. He earned a bachelor’s degree in political science from Tufts University (Massachusetts), a master’s degree in education from Harvard University (Massachusetts), and a doctorate in sociology from the University of California, Berkeley.

Roksa is an assistant professor of sociology at the University of Virginia, with a courtesy appointment in U.Va.’s Curry School of Education. She is also a fellow of the National Forum on the Future of Liberal Education. She earned a bachelor’s degree in psychology from Mount Holyoke College (Massachusetts) and a doctorate in sociology from New York University.

What prompted you to delve into researching national learning outcomes, and what were your goals in building your dataset?

Arum: Our interest in doing this research grew out of our awareness that research in K-12 education that looks for factors associated with learning, measured with objective indicators, has existed for decades. Yet similar research has not existed in higher education. And so when the opportunity arose to join the Council for Aid to Education and develop a dataset that would allow one to explore the factors—individual and institutional—associated with learning, we jumped at that opportunity.

Roksa: We were interested to learn how much students are learning and whether different groups are progressing at different rates. We also wanted to understand what factors may predict learning, at both the individual and institutional levels. In higher education, we don’t have measures of learning that track students from entry to completion. So we used the Collegiate Learning Assessment [CLA], which is a measure of general collegiate skills that aims to capture critical thinking, complex reasoning, and writing. We administered it to the same students during the fall of their freshman year and then at the end of their sophomore year.

What made the Collegiate Learning Assessment so well suited to your purposes?

Roksa: It takes a holistic approach; it’s not a multiple-choice test. And it aims to place students in a situation they could possibly be in after they graduate. The CLA uses open-ended scenarios, in which students are given materials to work with and are asked to respond to a particular prompt and write a response that reflects their critical thinking, analytical reasoning, and writing skills.

Arum: So students are given a set of documents, and they’re asked to interrogate the evidence, synthesize information across these documents, and then construct logical arguments based on their reading of the evidence. And these are the generic higher-order skills you would hope all college students would be developing.

Were you surprised by your findings?

Arum: We did not anticipate a set of findings that would draw attention to the limited learning in higher education. That pattern surprised us and dismayed us when we looked at the data. The key findings were that large numbers of students are going through higher education with little asked of them in terms of academic rigor, with little effort applied to their studies, and with few, if any, gains on this objective measure of learning.