Addressing the lack of English tests that measure critical thinking (CT) abilities of EFL learners, this study aims to develop an English Critical Thinking Test (ECTT) designed to measure the consistency, analysis, and inference CT skills. It is a follow-up to a pilot study to accumulate validity evidence for the use of the ECTT. A total of 262 Japanese EFL learners who were first-year college students took the ECTT, a Japanese Critical Thinking Test (JCTT), and an English proficiency test (EPT). The result of the internal structures of ECTT and JCTT revealed that both CT tests fit two-factor models better than three-factor models, with consistency as one factor and analysis and inference as the other factor. This implies that it is difficult to measure the CT skills' hierarchical nature. However, the model that included the three tests fit the data well and indicated moderately high correlations between the ECTT and EPT (r = 0.64) and between the ECTT and JCTT (r = 0.55). In addition, we confirmed that the skills that test-takers needed most in solving questions in each of the three CT sections (i.e., consistency, analysis, and inference CT skills) of the ECTT were as intended. Based on these findings, the validation of the ECTT has been improved; ECTT can be used for EFL learners at upper secondary and university levels in Japan.