Previous research has shown that the use of clickers in the classroom enhances student engagement. However, few studies have investigated how the type of clicker question may influence learning outcomes. To explore this, we compared the effects of lower-order cognitive skill (LOCS) and higher-order cognitive skill (HOCS) clicker questions on later exam performance in a biology course. During class time, students were presented with clicker questions directly related to unit content. Half of the content units were taught with LOCS, the other half with HOCS. To ensure that type of content did not influence results, the cognitive level of the clicker questions was counterbalanced across two semesters. The exams included a mix of LOCS and HOCS for each content unit. We also investigated the possible moderating effects of student perceptions on the relationship between type of clicker question and exam performance using student surveys. We found that using HOCS clicker questions significantly affects student learning. Practice with HOCS clicker questions improved performance on LOCS exam questions but not on HOCS exam questions. Students ranked lecture with clickers as a preferred and most helpful teaching methodology. Overall, these results suggest that practice with HOCS questions is engaging and gives students practice recalling content to "solve" a problem, thereby encoding low-level information and preparing them for higher-order thinking activities.