In 1998, the University of Charleston implemented an academic transformation, moving to an outcomes-based learning environment in which the university awards academic credit based on competences mastered rather than courses completed. That fall, for the first time, students no longer took introductory courses in composition, speech, and computing. Rather, they used assignments from their first-year learning communities to assemble portfolios as evidence of their writing, speaking, and computing skills. However, some semesters later, few students were completing portfolios and therefore were at risk of not making sufficient progress toward graduation. This article describes the evaluation process undertaken by the University of Charleston to determine the success rate of its outcomes-based learning program. It also describes the steps made to improve the success rate of the program.