Harvard’s Data Problem

Volume VIII, #22

My wife, Yahlin Chang, is the daughter of immigrants from Taiwan. As the youngest of three girls, by the time she arrived, her parents’ business was doing well enough for the family to buy a house in the wealthy suburb of Scarsdale, NY. Yahlin had many advantages growing up in Scarsdale. But she also had some disadvantages, like the fact that her parents weren’t able to communicate fluently in English and Yahlin never really learned Taiwanese. So rather than talking during dinner, the entire family would watch Dallas and Dynasty – which explains Yahlin’s career as a TV writer.

A recent topic of conversation at our dinner table is the Harvard admissions case. Although I tell her not to fret because our three boys will opt for Faster + Cheaper Alternatives to College, Yahlin remains concerned that their Taiwanese heritage will factor negatively in college admissions. So in response to a story about Harvard Asian-American students rallying for affirmative action as the trial began, Yahlin tweeted: “These students were already admitted to Harvard and can easily afford to support a policy that may disadvantage other Asian-Americans. Rejected Asians won’t be rallying similarly. These Harvard students are the college equivalent of limousine liberals.”

My response – also on Twitter, which I don’t recommend as a mode for family communication (even if you can’t communicate at the dinner table) – was that these debates are extremely unfortunate, pitting one group against another: “Determining disadvantage in college admissions is a data problem that a "distance traveled" metric could solve.”

***

Last year I wrote about Distance Traveled. The idea is that while major measures of advantage in the college admissions process are quantified (e.g., test scores and grades), most measures of disadvantage are not.

Over the past year, in association with the Kapor Center (whose co-founder Freada Kapor Klein originated the concept of distance traveled as it pertains to hiring), I’ve had a number of conversations on this topic with Deans of Admissions at selective colleges and universities. What they universally say is that qualitative measures of disadvantage (let’s call them QMDs) are considered in a “holistic review.”

Remarkably, Harvard’s defense at trial is that any discrimination against Asian-American applicants stems from teacher and guidance counselor recommendations that produce “lower personal ratings.” But isn’t this exactly the sort of QMD that a holistic review should pick up? While Harvard knows that applicants from inner city high schools should get a leg up in the admissions process, it is now clear that while Harvard also recognizes that guidance counselors and teachers discriminate against Asian-American applicants, it has completely whiffed on doing anything about it.

Who knows how many other QMDs aren’t being considered in Harvard’s “holistic” admissions process? So while the media is casting the Harvard trial as Asians vs. African American and Hispanics, the fundamental struggle is candidates with disregarded QMDs vs. everyone else. That is, at root, the Harvard admissions case isn’t about race or ethnicity. It’s about data.

***

Harvard’s defense is a stunning acknowledgement of the failure of holistic review. Holistic review is like trying to solve a complicated equation by looking at the math problem “holistically” rather than actually doing the work. It’s an approach that’s too arrogant even for some of the elite institutions that espouse it. More important, it’s not likely to yield an accurate answer. None of the admissions directors I’ve spoken with even attempt to make the argument that holistic review puts QMDs on a level playing field with quantified metrics of advantage like test scores and grades. And the Harvard case puts the lie to the notion that holistic review is comprehensive in its consideration of QMDs.

If elite college admissions aims to be something more than the Mega Millions lottery, it must at least attempt to normalize all characteristics so all applicants are evaluated on an apples-to-apples basis. And you can’t normalize characteristics without first quantifying them.

Now I know what you’re thinking: there’s no way to quantify QMDs. I have a different, perhaps warped perspective. In college, I double-majored in literature and economics and combined the two disciplines by attempting to quantify everything in sight. I remember trying to measure the “conceptual distance” in French Surrealist literature that maximized humorous effect. I conversed with my roommates in the metric of utils, as in “how many utils will you get from eating that extra dessert?” I suppose I’m still doing it today: working on closing the skills gap requires a certain quantification of ineffable human capabilities. It’s all very nerdy stuff, but no more nerdy, say, than attempting to quantify the quality of an entire educational institution in a single score.

If U.S. News can quantify “16 measures of academic quality,” including “academic reputation” and “student excellence” in its rankings, then we can at least attempt to quantify all known or suspected QMDs. In other words, if millions of families are making college application decisions based on quantified metrics of quality, then colleges should do the work to quantify every known or suspected QMD so the resulting admissions decisions are at least arguably fair.

A fair admissions process requires methodically quantifying all QMDs and attempting to normalize them. If it’s possible to quantify utils or academic reputation, it’s also possible to measure bias in teacher recommendations and correct for it. Naturally – like U.S. News’ rankings – this will always be a moving target. Each year will require modifications and changes in weighting – not to sell magazines, but for a nobler purpose. Making the effort means we’ll be directionally right and result in improved decision making. (In the same vein, college rankings – if used wisely, in moderation, if that’s possible per human nature – are still an improvement over a “holistic” sense of institutional quality.)

Last month, writing about the U.S. News college rankings in the Washington Post, Bridget Burns, Executive Director of the University Innovation Alliance, noted that “improving outcomes for low-income and minority students has to start with transparency.” How to do this? Incorporate more quantified measures of which colleges are best serving low-income students. The Harvard case demonstrates the importance of applying the same principle to college admissions.

I understand why universities are reluctant to do this. A “holistic” process – or as Harvard’s expert witness says – “Harvard’s whole-person evaluation” – allows for a lot of fudging, like athletic and alumni preferences, particularly for children of major donors. The problem is that this is the road to lawsuits and headlines, like this one in The Atlantic last week: “College Sports Are Affirmative Action for Rich White Students.” QMDs must be accounted for and normalized before deciding to take into account athletes, alumni, and donors; Harvard should level the playing field first.

In a tweet a few weeks ago, John Arnold recognized “the bigger issue with Harvard admissions:
In 1977, freshman class was 1,585 with endowment of $2 billion
In 2017, freshman class was 1,659 with endowment of $37 billion”
America’s colleges and universities need to work a lot harder to reflect the American Dream rather than complacently mirroring the concentration of wealth and increased inequality in American society over the past 40 years. And because no selective school is going to expand enrollment by an order of magnitude, the need for a truly fair admissions process is greater than ever.

Sooner or later, by court order or otherwise, selective schools will need to implement more transparent, and rigorous admissions processes by quantifying QMDs, perhaps with a single heavily weighted distance traveled metric. These schools are currently deploying significant talent and energy to launching new data analytics curricula and programs; why not apply a fraction of these resources to admissions? Colleges and universities that make the effort are likely to end up more truly diverse, enriching, and flat out better institutions than those that don't. And those that do it really well have a shot at supplanting holistic Harvard itself.