Subjective measures (SMs) of awareness assume (a) participants can accurately report the implicit/explicit status of their knowledge and (b) the act of reporting does not change that knowledge. However, SMs suffer from nonveridicality (e.g., overreporting of "guess" responses) and reactivity (e.g., prompting rule search). Attempting to improve the validity of "guess" responses, we conducted an exploratory mixed-methods replication of Rebuschat et al. (2013). Participants (N = 30) were randomly assigned to Traditional, True Guess, and NoSMs conditions. True Guess participants were led to believe the computer would replace "guess" responses with random answers. Confirming that SMs are reactive, Traditional and True Guess participants responded more slowly and accurately, with greater awareness of the linguistic target. Moreover, although True Guess participants responded "guess" less frequently, interviews revealed this was due not to greater veridicality, but rather to additional reactivity. We conclude with directions for further research to enhance the validity of SMs.