Admissions And Hiring Adrift In A Sea Of Subjectivity

For a lot of students, the main point of professional school is what happens when they’re not paying tuition i.e., over the summer. Business school is all about the summer; MBA career rocket launch comes from converting a prime summer job to a full-time offer. Law school is almost as summer-centric: the most lucrative law firms recruit from their summer associate programs.

Because I knew I didn’t want to be a lawyer, my law school summers were more careering than career-centric. After first year, I took advantage of my law school’s offer to pay a summer associate salary for doing public interest legal work anywhere in the United States. First, I checked to make sure they really meant anywhere – not just the continental U.S. Confirmation in hand, I flew off to the Legal Aid Society of Hawaii, which yielded rewarding work and a magical summer. My roommates had a different experience. Because they weren’t sure that going big-lawless was flawless, brother Aaron and best friend Dave pursued the summer associate path, accepting offers from white-shoe firm Ropes & Gray and shipping up to Boston. As they recounted, Ropes pulled out all the stops, wining and dining summer associates and taking everyone to a Red Sox game where an administrative assistant held an impressive bag of cash calculated to transform the prospect of years of long, tedious hours into an irresistible proposition. Every summer associate who got up to go to the concession was encouraged to dip into the bag and take what they needed.

Of course, when you’re at a baseball game, some vendors come to you, as Aaron knew very well. In high school, Aaron had slung cokes at Exhibition Stadium, the original windy lakefront home of the Toronto Blue Jays. So he knew a thing or two about stadium cokes. At the Red Sox game, Aaron hailed a passing vendor, dipped into the big bag of cash, and bought himself a coke in a lidless cup. As the vendor grabbed the bill and headed off to find his next customer, Aaron took a sip and immediately spat it out. “Hey,” he shouted at the vendor, “this coke is bad.” The vendor swiveled his head and yelled back in Boston-inflected deadpan – “they’re all bad,” and went on his merry (Fen)way.

“They’re all bad” is what every education organization and institution worth its salt is saying about last term’s Supreme Court decisions. Well, maybe not all of them (e.g., voting rights, elections, immigration), but certainly the higher education rulings on student loan forgiveness and affirmative action. But reactions to the former seem somewhat knee-jerk. As student loan forgiveness was a backward-looking Band-Aid, the Supreme Court decision will have little impact on higher education. And because President Biden subsequently doubled-down on his no-consequence-for-failure-to-repay policy until loan forgiveness part deux is struck down again, and because income-driven repayment is an entitlement growing more favorable with each passing month, the decision may not hurt angry borrowers unless and until there’s another Republican in the White House. (And as a result, a generation of Americans ill-served by tuition-hungry colleges and universities is less likely to put another Republican in the White House.)

Affirmative action is a different story. Although admissions criteria are only relevant for selective institutions, which amount to ~200 colleges, 10% of students – and truly significant for highly selective schools: 68 colleges, 3% of students – the consequences of the Supreme Court decision are magnified by two factors: first, digital technology making it much easier to apply to selective schools, causing applications to skyrocket; second, highly selective schools’ shameful disregard for expanding capacity. This toxic brew has caused acceptance rates to plummet to record lows, increasing scarcity value and providing hundreds of thousands of disgruntled potential plaintiffs for interest group lawyers. As ASU President Michael Crow correctly notes, “race-based admissions is not necessary if… everyone who’s qualified is admitted.”

So for the growing jilted multitude, affirmative action mattered. And its demise matters to high-potential, low-resource Black and Brown teens who were slammed by Covid closures and now will be denied the benefit of the doubt.

What problems was affirmative action trying to solve for these 68 or 200 schools? First and foremost, ensuring a diverse and representative student population. But although an admittedly blunt diversity instrument, affirmative action was useful in another way. While it’s debatable whether race is the right or even a good metric (and what its impact has been on Asians – see recent New Yorker critique of affirmative action: “a program designed for a racially binary America [that] never got meaningfully updated for today’s multiracial democracy”), what’s irrefutable is that it’s not subjective – and therefore a potentially useful shorthand or guidepost for handpicking a small class of admitted students from tens of thousands of qualified applicants.

After Covid wounded standardized testing and California killed it, selective college admissions offices have now lost another objective metric. What’s left? In his majority opinion, Chief Justice Roberts affirmed universities can still consider “an applicant’s discussion of how race affected his or her life.” He’s pointing to personal essay prompts and perhaps recommendations, which dramatically inflate the importance of the file’s first reader, and perhaps that reader’s mood and what she had for breakfast that morning. There’s no admissions rubric for grading personal essays or recommendations, particularly as there’s no way to really know how much help wealthy applicants receive from admissions consultants and advisors, and when applicants, as The Atlantic charges, are now likely to “write about their race through formulaic and belittling narrative tropes.” With affirmative action thrown overboard and with the establishment of a quantitative metric for distance traveled (i.e., adversity overcome) progressing at exactly zero knots, selective college admissions offices are hundreds of miles offshore, adrift in a sea of subjectivity.

Highly selective schools have long pointed to “holistic admissions” as a way of saying all factors are considered for every applicant. But in a compressed admissions calendar, it’s impossible for a few dozen people to deliberate (let alone investigate) every factor for tens of thousands of applicants. What holistic admissions has actually meant is that many factors are considered as long as objective criteria categorized an applicant into a desirable bucket. Now, all we’re left with are grades – a shaky foundation as they vary by school and teacher. Holistic admissions could work if humans could access, process, and properly weight all relevant data. But given the number of applicants to top schools and the amount of available data, it’s not possible to do so in a way that’s fair or likely to float for very long. Many qualified applicants may end up preferring to flip a coin, prompting interest groups to begin jockeying for lottery admissions – which would put the mockers on the entire selective admissions enterprise.

***

In terms of economic opportunity, selective college admissions is the VIP gate for a fortunate few. But hiring is the gate through which we all pass. And while many speculate about the Supreme Court decision’s impact on diverse hiring and corporate DEI, to my mind, the more important connection between the end of affirmative action and the world of work is that it’s merely the latest in a series of government actions pushing private actors towards subjective selection decisions.

The problem faced by employers with desirable – i.e., digital – jobs is similar to selective colleges: thanks to job boards and the ease of applying online, too many applicants to fully and fairly evaluate each one. Unlike holier-than-thou colleges, companies don’t claim to practice holistic hiring, sweating the details of every application. Instead, employers rely on applicant tracking systems: parsing keywords from résumés, matching them against job descriptions, and only passing through a small fraction for human evaluation.

In order to winnow down applicants to a reasonable number, employers would love to find more objective criteria than matching keywords. Online skills-based assessments are an obvious answer e.g., sending every applicant a short assessment to become visible to a human hiring manager. Unfortunately, developing valid skills-based assessments isn’t the hard part. The hard part is they’re probably illegal: the federal government prohibits employers from using any instrument between application and hire that has an “adverse impact” on a disadvantaged group.

Most of the adverse impact charges pursued each year by the Equal Employment Opportunity Commission (EEOC) involve the use of skills-based assessments in hiring. According to the CEO of a leading employment assessment provider, “the industrial-organization psychologist at one U.S. client described the last ten years of her career as being spent in court defending adverse impact cases brought by the EEOC.” Between 2015 and 2018, Target paid out almost $7M in fines and settlements over adverse impact in its hiring screens. As the Minneapolis EEOC office director said of Target’s case: “The tests were not sufficiently job-related… [and] screened out people in particular groups.” In 2020, Walmart settled an EEOC pre-employment testing suit for $20 million. One assessment provider executive told me that “what’s remarkable about the U.S. is how quickly lawyers and general counsel get involved.”

The only defense to an adverse impact claim is to demonstrate content validity, construct validity, and criterion validity i.e., that the skills test is predictive of job performance. The problem is that the process of demonstrating a priori validity for each and every assessment-position combination is so time- and resource-intensive that few companies ever do it; they’re busy with other things, like running their businesses (and matching keywords).

With assessments off-limits, employers are turning to new technology, specifically AI. Although no one is getting hired by AI – it’s simply a more data-informed way of filtering qualified candidates than parsing and matching keywords – here, again, government is getting in the way. So far, New York City has adopted legislation to limit the use of AI in the hiring process and California, Maryland, and Washington, are considering the same. New York’s law is similar to adverse impact enforcement, barring employers from using AI in hiring unless they first notify candidates and implement bias auditing. At the federal level, while the EEOC eyes AI as an opportunity to expand its regulatory ambit, Congressional Democrats have introduced the Algorithmic Accountability Act, which proposes a similar approach.

***

I’m not arguing AI is a hiring panacea. But it’s ironic that the authorities are prohibiting employers from using technology to solve a problem caused by technology i.e., how to get from thousands of online applicants to a more manageable number. What’s even more ironic is that in an era of digital transformation, we’re supposed to be making better, more data-driven, objective decisions. But thanks to public prohibitions, the opposite is occurring. For both selective admissions and selective hiring, it seems like government reaction to digital transformation is to run away from data and objective criteria, perhaps hoping the resulting decisions will be more human.

While subjective decisions are likely to have more of a human element, there’s a reason Alexander Pope wrote “to err is human” i.e., not just because it scans. In the context of admissions and hiring, subjective decisions mean thousands of false negatives and hundreds of false positives. So when government prohibits colleges from using standardized tests and race, and employers from using assessments and AI, it’s head-in-the-sand thinking to believe decisions are likely to be better. More likely, taking refuge from the cold calculus of digital transformation in warmhearted holistic and highly subjective processes means no data to support the legitimacy of decisions and further erosion of public confidence in both admissions and hiring. Because whether they’re trying to secure admission to a selective college or professional school, or get a great summer job or full-time position, millions of excluded Americans will join the ranks of the jaded Fenway Park coke vendor in saying of these decisions: “they’re all bad.”