This paper investigates how an assessment roleplay task was being interpreted and performed by its student test-takers. The rationale for this is that we need feedback into whether the task is functioning as intended and to ensure that our interpretation of the scores from grading these students correctly informs our decision making about their abilities. This study focused on a group open roleplay task in which students must choose their own persona and showcase their socializing skills for an end-of-unit assessment task. Particularly, their choice of roles can affect the quality of interaction, which in turn can affect the scores they eventually received. This study analyzed 180 students' performances, grouped into 34 roleplays. As a mixed-methods study which seeks to utilize the information that we gathered on students' task uptake to improve the assessment practices for the next iteration of this task, this study first evaluated the personas students selected for their roleplay to identify features that distinguish appropriate roles for the task from inappropriate ones. Later, the task uptake in terms of the qualities of their roleplay interaction was also investigated through the lens of conversation analysis (CA). The results show very few incidences of roleplay personas deemed inappropriate for the task, showing a good level of alignment between student task uptake and the intended design of the assessment task. The CA results show that this roleplay provided opportunities for students to display interactional skills that were similar to that of real-life interactions. The findings highlight the unique aspects of communication skills that this open roleplay task required from the students specifically in the turn-taking organization, overall sequential organization, and topic management practices.