Hacking Competency-based Education

Volume VI, #18

My college roommate Chris was notorious for a few things. Chris looked vaguely Nordic and enjoyed putting on a Viking helmet, taking off all his clothes, and venturing out to Cross Campus Lawn to ask unwitting pedestrians if they knew where the German Club was meeting. He was also renowned for going to great lengths to avoid school work. So in completing his foreign language requirement (German, naturally), rather than spend the time to complete weekly assignments on the tidy, efficient 3.5-inch floppy disk provided by the German Department, Chris took 10 hours one evening to hack into the program so he could mark all exercises completed.

In the ensuing years, hacking has gone from vice to something of a virtue. Hackathons refer to intensely productive coding sessions. Facebook’s Like button was developed at a hackathon and Mark Zuckerberg’s letter that accompanied Facebook’s IPO filing stated:

We have cultivated a unique culture and management approach that we call the Hacker Way. The word “hacker” has an unfairly negative connotation from being portrayed in the media as people who break into computers. In reality, hacking just means building something quickly or testing the boundaries of what can be done… The Hacker Way is an approach to building that involves continuous improvement and iteration.

***

By Zuck’s definition, much of higher education could use a good hacking. Competency-based education (CBE) is one such effort, and an important one. Rather than starting with what faculty want to teach, wouldn’t it make more sense to start with the competencies employers expect, then determine the assessments that demonstrate the requisite competencies, and only then turn to the question of the curriculum that best equips students with those competencies? By inverting the seat-time model, colleges and universities should be able to produce more competencies in less time and for significantly less money.

Hundreds of schools believe CBE is a worthwhile hack and have either launched or are preparing to launch CBE programs. Which is great, except for three inconvenient truths:

  • Students don’t care. Outside of Western Governor’s University, only a handful of CBE programs have enrolled more than 1,000 students. (I have argued elsewhere that these early CBE programs are insufficiently differentiated from existing online degree programs and that the real power of CBE will be the new competency-based “apps” that can and will be built atop a CBE platform.)
  • The CBE efforts underway at hundreds of institutions are for new programs only. I’m not aware of any institution looking to retrofit existing programs as CBE. This goes for programs that are delivered 100% online, and particularly for the many more certificate and degree programs delivered onground, for which a good model for delivering (by definition self-paced) CBE programs at scale has yet to be developed.

    This point is critical. Competency-based education is complicated and requires a partial if not complete disaggregation of the faculty role (design/teach/assess/advise). Does anyone believe faculty members will buy into retrofitting courses they’ve been teaching for decades? Not unless their jobs are at risk (and by then it will almost certainly be too late for CBE to be a viable solution).
  • CBE is under regulatory attack. An audit report issued by the Department of Education’s Office of Inspector General (OIG) found one accreditor had failed to evaluate whether CBE programs ensured “regular and substantive interaction” between faculty and students. Without such a finding, CBE programs should be reclassified as correspondence programs, which means students are classified as attending half-time (with a concomitant reduction in federal aid). Concurrently, several early leaders in CBE have supposedly received notices from OIG that their programs are properly correspondence courses, causing these schools to scramble to defend their programs and students.

Still, the potential benefits of CBE are undeniable. CBE can make competencies visible to employers and students, resulting in greater clarity as to what postsecondary education is required, and when it’s required i.e., not universally necessary or beneficial to do four consecutive years before the age of 25, and then stop for life. CBE can power co-requisite remediation: directing students to appropriate reinforcing or co-remedial materials when required, allowing them to keep swimming towards the finish rather than treading water in remedial academic limbo.

***

So what to do? Is there a way to hack the hack? One theory is that CBE will be hacked by competency-based assessments (CBA). Administering a range of short, specific competency-based assessments to students along their postsecondary education trajectory – and critically at the point of credential completion – could provide many of the benefits of CBE without any of the hassle of changing faculty roles or retrofitting programs and courses.

CBAs are different from familiar standardized tests – not because the questions are very different, but because they test for distinct competencies. Discrete 15-20-minute CBAs already exist for various key competencies: reading for information, critical reasoning, locating information, organization, writing effectiveness, writing mechanics, analysis and problem solving, critical reading and evaluation, and quantitative reasoning. If a competency or capability can be defined, it can be assessed. And while brief CBAs aren’t entirely conclusive, they are highly predictive. In addition, psychometricians are busy developing situational judgment tests (SJTs) that apply knowledge and skills to real world settings across a wide range of professions, and that also test soft skills like teamwork and empathy. These aren’t merely technical or vocational skills to be dismissed as the province of community colleges and workforce development agencies. These are “power skills” or “executive function skills” – the competencies that allow a graduate to get and succeed at the fifth job (if not the first). These competencies are the very raison d’ȇtre of a liberal arts education.

Countries with less developed higher education systems have already hacked CBE. In India, many mid-sized and large employers won’t look at university graduates unless they also provide relevant and compelling CBA and SJT scores. And while U.S. employers aren’t as skeptical about the value of a degree, they’re not exactly over the moon. So an increasing number have resorted to utilizing CBAs as a secondary screen – from the candidate pool to the hire.

Hacking CBE becomes powerful when CBAs become sufficiently popular among job seekers without significant work experience that employers are able to rely upon them as a primary screen i.e., filtering from the applicant pool to the candidate pool – determining which applicants make it through to a human hiring manager. How could this happen? Most likely as universities seek a leg up for their students. While many universities – particularly the most elite – will continue to maintain (as one Dean recently relayed to me) that universities don’t care whether students get jobs, faced with uncertain and declining enrollment as well as survey after survey demonstrating that employability is now students’ number one priority and expectation by far, most schools are already interested in providing their graduates with a competitive advantage in the job market.

In the next few years, CBAs will come to market as exactly this competitive advantage, with compelling evidence that brand-name employers are increasingly relying on them (albeit as secondary screens). Early adopters of CBAs (whether paying for them directly, or encouraging students to do so) will find they’ve opened a door for their graduates to human hiring managers at many desirable employers. CBA vendors will partner with Applicant Tracking Systems, allowing Applicant Tracking Systems to capture and consider CBA data on the primary applicant-to-candidate screen. And employers will utilize People Analytics technologies to better describe required competencies in job descriptions, further increasing the accuracy of the primary screen. In time, it may become clear to universities and students that candidates without the additional competency data provided by CBAs will never make it through the ATS filter to be reviewed by a human hiring manager (let alone actually hired). Candidate pools will seem better and better, employers will stop screaming about the “skills gap,” and the marketplace will gain further steam.

An alternative path to the CBA hack is accreditation. If accreditors – perhaps prompted by a Department of Education (ED) weary of a quality assurance system based on process rather than outcomes – were to suggest that student outcome data could be relevant to the question of continued accreditation and access to billions of dollars in Title IV funding, CBA adoption would explode. But regardless of what ED and accreditors do, farsighted trustees and college and university leaders will want to know how their courses and programs do at equipping students with these competencies. Such knowledge will provide an invaluable new resource to evaluate, improve and cull academic programs.

The coming rise of CBAs is likely to raise familiar questions in the world of education, albeit new to higher ed. Do these tests merely measure test-taking abilities? And how do we keep colleges and universities from teaching-to-the-test? In the spirit of Mark Zuckerberg’s definition of hacking, this is the constant iteration and improvement that will be required if CBAs are to prove transformational. The stakes are too high for colleges and universities not to try.

University Ventures (UV) is the premier investment firm focused exclusively on the global higher education sector. UV pursues a differentiated strategy of 'innovation from within'. By partnering with top-tier universities and colleges, and then strategically directing private capital to develop programs of exceptional quality that address major economic and social needs, UV is setting new standards for student outcomes and advancing the development of the next generation of colleges and universities on a global scale.

Comments

Three articles that tell us where the puck is going in higher education

1. Higher Education’s Real Crisis Third Way report by Tamara Hiler and Lanae Erickson Hatalsky on crisis of efficacy at America’s public colleges, including poor completion rates and wage outcomes. Only 80 out of 535 four-year public institutions—or 15.0% of these schools—graduate more than two-thirds of their first-time, full-time students each year. That means that if the remaining 455 schools were a part of our country’s K-12 system, they would be considered “dropout factories” under the recent reauthorization of the Elementary and Secondary Education Act. Read more 2. Focus On What Really Matters Trusteeship article by Gallup Executive Director Brandon Busteed on the emerging focus on outcomes – particularly behavioral outcomes – and the end of college rankings as we know them. The Real data revolution in higher education… will usher in a new era of rigorously tracking the expectations, experiences, emotions, and outcomes of students, alumni, staff, and faculty in the spirit of understanding how institutions of higher education are performing, and how they can improve. Boards, take heed. Read more 3. Show What You Know VentureBeat report on innovative (and scary) experiment at Indian e-commerce firm Flipkart: hire coders without interviewing them. Rather than waiting for experienced developers or those who have honed their interview techniques, they’re basing their hiring decisions purely on the potential demonstrated in a graduate’s coursework… “The conventional hiring process often comes down to the performance of the candidate on that specific day, which may not be a true reflection of their skills and temperament,” said Peeyush Ranjan, chief technology officer at Flipkart. Read more