Despite evidence on the interaction between cognitive individual differences (IDs) and task complexity, our knowledge of how affective IDs, such as foreign language enjoyment (FLE), interact with task complexity and other factors is limited. Since tasks and activities were found by Dewaele and MacIntyre (2014) to be most relevant to FLE, and since task complexity might interact with learners' perceptions of task difficulty, it is important to investigate how task complexity impacts FLE changes. Informed by the complex dynamic systems theory, this study employed a mixed-methods multiple case study design to study patterns and causes of high and low FLE arousals. The participants were four pairs of Taiwanese high-intermediate EFL university students who were engaged in simple or complex storytelling tasks with speech acts of refusals. The speakers' interactions were triangulated with an individual learner's rating of FLE on a per-second scale and stimulated recalls. Results revealed idiosyncratic patterns of FLE fluctuations of peer interlocutors and a high degree of overlap in sources of low and high FLE in both groups. Speakers reported high FLE as a result of interesting storylines inherent in task design and created by peers, the use of picture prompts, peer collaboration, and task performance. Performance problems, failure to retrieve appropriate vocabulary, task design, and lack of ideas led to low FLE arousals. The findings suggest that task complexity combined with other task-induced, social, and individual factors to affect the fluctuations of FLE. Implications for task design and oral communication instruction to promote FLE are discussed.