The current study aims to demonstrate how the lack of a conceptual framework and the inadequate use of confirmatory factor analysis (CFA) in second language (L2) acquisition validation studies can lead to misconceptions about the nature of data collected via different measurement instruments. To this end, we reanalyzed data from three factor-analytic validation studies on several measures of L2 explicit (EXK) and implicit (IMK) knowledge and demonstrated how an inadequate implementation of CFA in these studies has led to unwarranted validity claims. Following several criteria (e.g., presence or absence of time pressure, or drawing of attention to form or meaning), Ellis and Loewen (2007), Bowles (2011), and Zhang (2015) created test batteries that included different types of tests hypothesized to be "distinct" measures of EXK or IMK. Our re-analysis included the original CFA models retained in these studies, together with new theoretically and empirically plausible alternative models. Results demonstrated that the conclusions reached in the original studies were compromised by the existence of alternative or even equivalent CFA models that fit the date, and the measures included in these batteries were not actually distinct measures of EXK and IMK.