Saving the Baby, Throwing Out the Bathwater

Volume III, #20

It was a head-scratching moment when I learned that community colleges are scrambling to offer housing to students. In an article in The Chronicle of Higher Education, Kim Linduska, Executive VP at Des Moines Area Community College, defended her institution’s effort to build housing: “As community colleges have grown and matured, many more of them have worked harder to attract students.” In Indiana, Ivy Tech Community College in South Bend recently completed a $4.5M dining hall and student center and the campus in Gary is developing a $17M fitness and wellness center. A performing arts center is also being contemplated. Meanwhile Los Angeles Community College District is spending over $25M building fitness centers, gyms, tracks and tennis courts.

The head scratching get more intense when you consider that 60% of entering community college students are relegated to remedial courses before being admitted to a degree program and over 70% of students in remedial math courses do not pass these courses. This means about 4 in 10 incoming community college students – almost 2 in 10 of all undergraduates, and mostly minority students – never have a fair shot. So rather than working harder to attract students, how about doing a better job with the students we have?

There is widespread agreement that current remedial or developmental courses are not passing muster. In response, organizations like Complete College America and Jobs for the Future have pushed to mainstream entering students immediately. These groups have persuaded Connecticut, Tennessee and other states to pass laws restricting relegation to remedial courses. But a more constructive answer would be to rethink how community colleges approach remedial education. And the best way to do this is to start with the ultimate goal: making as many students as job-ready as possible.

Fortunately, the research is fairly clear as to what predicts job readiness. Unfortunately, the baby comes with some fairly dirty bathwater. It’s an interesting and important story, and one that has the potential to materially improve outcomes for millions of prospective students over the coming years.

***

The best predictor of job readiness is a composite of cognitive skills tests.

M is the mean correlation coefficient for future on-the-job performance, as measured by supervisor ratings and work product. M=0 means no correlation, while M=1 means 100% correlation.

“Ability composite” is a combination of three scores on different cognitive skills tests – for example, Applied Mathematics, Reading for Information, and Locating Information. This composite is even more predictive of job performance and success than a job tryout, and over 5x more predictive than whether a student has a degree.

This may sound tautological – students who are better at math and reading are going to be better employees, so therefore what they need is more time in remedial math and English courses – but it’s not. Because cognitive skills are a different kettle of fish. Cognitive skills are assessed in a content neutral or independent manner – divorced from the typical learning progression in formal education (for example, where calculus builds on algebra). This makes sense as the road to success in most jobs isn’t dependent on arriving with any particular knowledge, but rather one’s ability to interpret, sort, organize and make sense of information on the job. This information processing facility is key to success in arriving at a conclusion, developing a product, addressing a customer complaint, or designing a new strategy. Individuals with high levels of cognitive skills are able to move from task to task with high fluidity and productivity. Ironically, they are the embodiment of what we think of when we think of a liberal arts education, because they are experts in learning how to lear

ACT, the leading assessment organization, has profiled over 16,000 jobs by sending experts to corporate worksites and found that over 95% of all jobs can be expressed as a combination of 3-5 fundamental skills, including the three in the ability composite. Companies that have utilized cognitive skills-based hiring to select job candidates have seen fairly remarkable performance improvements, such as 25-75% reductions in employee churn.

Sounds amazing, right? So amazing it must be illegal? Well, you’d be right about that. It is illegal.

***

From 1948 to 1981 the federal government employed some form of cognitive skills test as part of its hiring process. At one point called the Federal Service Entrance Examination, the test was found to be very predictive of job performance. But it was also highly discriminatory. Minority groups scored lower on the test and after a number of lawsuits, on November 19, 1981, in the early days of the Reagan Administration and guided by the Supreme Court’s 1971 decision in Griggs v. Duke Power Co. which outlawed testing that had a disparate impact on minorities, the federal government signed a consent decree in the case of Luevano et al v. Campbell that would prohibit any selection mechanism in hiring that produced “adverse impact” to minority groups – defined in the 1978 Uniform Guidelines on Employee Selection Procedures as a selection rate for any minority group that is less than 80% of the rate of the highest scoring group.

As a result, private sector employers have abandoned such testing. Employers who continue to utilize tests that produce adverse impact – not only cognitive skills tests, but also personality tests – are aggressively pursued by the Equal Employment Opportunity Commission (EEOC) until such activity ceases. The current hiring landscape reveals that the baby has been thrown out with the bathwater.

Interestingly, neither a federal court nor the EEOC has ever opined on whether using college degrees as a criterion for hiring is in violation of the Uniform Guidelines. Given the degree gap faced by minorities, it almost certainly is. But by employing a literalist reading of the Griggs decision, the EEOC is focused solely on use of assessments knowing full well that pushing the degree question would be opening Pandora’s Box.

***

The Uniform Guidelines have not been updated in 35 years. In this substantial period of time, we’ve learned a few important things.

1. Cognitive skills can be remediated reasonably quickly.
Utilizing specific curriculum designed to improve cognitive skills, 10 hours of training in each of the three key cognitive skills resulted in test results that were equivalent to a gain of 2-3 years of full-time education. 96 hours of training using the same curriculum yielded an improvement so significant that it allowed students to advance an entire job level (e.g., from entry-level to administrative jobs, or from administrative to white collar jobs). Based on available data, all racial and ethnic groups appear to improve at the same rate.

2. Technology can make cognitive skills tests much more accessible.
The federal government tests were 3 hours or longer. Through adaptive testing, key cognitive skills can be assessed in a very short period of time. By shortening the tests, they become repeatable so candidates can try as many times as they like

3. Technology can reduce the adverse impact of skills tests.
It has become easier to dynamically generate skills tests for individual jobs that are a combination of cognitive skills and job skills. These tests should produce less adverse impact

4. Technology can make remediation curriculum even more effective
Incorporating adaptive learning, gamification and on-demand help into remediation curriculum has the potential to make very dry and difficult curriculum much more accessible to the students our community colleges are currently failing to serve.

***

“He guessed as well as he could, and crawled along for a good way, till suddenly his hand met what felt like a tiny ring of cold metal lying on the floor of the tunnel. It was a turning point in his career, but he did not know it.” - J.R.R. Tolkien, The Hobbit

As I learned more about the saga of cognitive skills assessments, I imagined how Bilbo Baggins must have felt when he found the ring. Some thought the all-powerful ring could be used for good. But its power was so great that it corrupted anyone who slipped it on. In the end, it had to be destroyed, cast into the fires of Mount Doom in Mordor.

We think what we’ve learned about cognitive skills and the power of technology to help remediate cognitive skills means we don’t need to find a cognitive skills-bearer to carry assessments into Mordor alongside an oddly named companion. The baby can be saved! By allowing job seekers to repeat tests and by making remediation curriculum more accessible and powerful, adverse impact can be eliminated and assessments can be made safe and fair for employers and job seekers. And in the event adverse impact can’t be entirely eliminated, you can bet it will be much less than what we currently see from degrees, which will mean better prospects for all students.

We believe this approach represents an important opportunity for our community colleges – more worthwhile of their focus and resources than building housing and wellness centers.

Innovate + Educate, a not-for-profit organization focused on skills-based hiring, contributed a great deal to this Letter.

University Ventures (UV) is the premier investment firm focused exclusively on the global higher education sector. UV pursues a differentiated strategy of ‘innovation from within’. By partnering with top-tier universities and colleges, and then strategically directing private capital to develop programs of exceptional quality that address major economic and social needs, UV expects to set new standards for student outcomes and advance the development of the next generation of colleges and universities on a global scale.

Comments