Mostly in web surveys, attention checks have been proposed to identify inattentive respondents in self-administered surveys as previous research has argued that low-quality answers may introduce severe biases in data analyses. The increasing popularity of mixing survey modes for conducting probability-based surveys amplifies the need for systematic research on the implementation of attention checks in these survey designs. Our study investigates whether an instructed response item (IRI) can be an adequate predictor of response quality in a self-administered mixed-mode panel survey with a probability-based German general population sample. By doing so, we compare the results of the attention check with those of other response quality indicators. However, while the results for the online respondents were somewhat promising, the mail respondents result suggests that this response quality indicator seems unreliable, so that we cannot recommend using IRIs in general population mixed-mode surveys, which include a mail option.