Disquantified Conference Provides Context to the Data Associated with College


Minh Hua
Campus Beat Reporter

Last Thursday and Friday, May 16 and 17, UC Santa Barbara English professor Christopher Newfield and Postdoctoral Scholar Heather Steffen hosted a two-day conference called “Disquantified,” which concerned how metrics are transforming higher education.

An interdisciplinary cast of scholars, professors, and administrators attended the aptly-named conference to discuss topics ranging from the uses of wage-by-major statistics, higher education data infrastructures, and the negative effects of metrics on student writing.

The conference was the culmination of a long-term research project called “Limits of the Numerical,” which questions what happens to research and learning when they are rated, ranked, and judged by metrics. Spearheaded by Newfield, the project focuses on three topics: the value of a college education, research bibliometrics, and learning assessment.

Broadly speaking, metrics are measurements of quantitative assessment commonly used to assess performance. In higher education, metrics take the form of graduation rates, enrollment numbers, demographic data, and student-faculty ratios, measuring how well a student is learning and the contributing factors to a student’s degree completion.

The 18 presentations that spanned the two-day conference covered topics such as choosing majors, student learning, and student assessment.

Ph.D. candidate in economics Zachary Bleemer, director of the UC ClioMetric History Project, presented on the shortcomings of median earning statistics, which are the numbers that chart an undergraduate’s income after graduation and are meant to help students decide on majors.

In his talk, Bleemer said that, “while these statistics are presented with the intention of helping undergraduates choose majors that suit their future employment preferences, correlation does not imply causation; wage-by-major statistics are biased estimates of the wage impact of switching majors.”

In his talk, Bleemer referenced a study done by Lars J. Kirkeboen, Edwin Leuven, and Magne Mogstad, which examined the reason why individuals in Norway’s centralized university system may choose different majors and the payoffs of those choices. According to Bleemer’s reading, universities lower students’ future income by not letting students choose their major based on their interests.  

Speaking about Bleemer’s reading, “The paper counters the idea that everybody should study STEM if they want to raise their wages,” said Newfield. “The implication is that intrinsic motivation helps people be more successful than they would be otherwise, and [going with] your first-choice major reflects intrinsic motivation.”

Bleemer concluded his talk with a series of improvements to the usability of wage-by-major statistics, such as the inclusion of non-graduates and controls for local cost of living in the estimation, as well as significant caveats in publicly-available reports and dashboards.

Another featured speaker was was author and columnist John Warner, who has written for Inside Higher Ed and the Chicago Tribune. In his talk, titled “Why Can’t They Write? We’re Counting the Wrong Things”, Warner claims that “prior to college, students are trained to produce ‘writing-imitations’, rather than learning to write in genuine and meaningful contexts.”

According to Warner, certain techniques that are particularly detrimental to a student’s writing skills are the cookie-cutter, five-paragraph-with-a-thesis-in-the-introduction templates that high school teachers impose upon students. Additionally, Warner referenced the fact that students are memorizing acceptable words to start off a paragraph as a way to pass the AP English exam.

“It’s like mad libs for academia,” said Warner. Warner introduced the concept that the base unit of writing is the idea — not the sentence; without an idea to be expressed, a sentence is meaningless. In addition, Warner said that the idea of memorization is perpetuated by how student writing is currently assessed, and he offers alternatives that are conducive to better writing.

Instead of rating a student’s performance based on how “academic” their prose is, Warner argues that we should measure how much students are writing in terms of total words produced, their enthusiasm for writing (through measures of engagement before and after taking a course) and most importantly, time spent “flailing”, or struggling but still persisting with writing.

The conference concluded on Friday at 6 p.m. with the drafting of key points and issues that need change.

Readers interested in watching the recorded talks can find the video at https://disquantified.org/