Although text readability has traditionally been measured based on simple linguistic features, recent studies have employed natural language processing techniques to develop new readability formulas that better represent theoretical accounts of reading processes. This study evaluated the construct validity of different readability formulas, including both traditional and newer formulas, by examining their ability to predict the processing effort involved during L2 reading as evidenced by eye movements. Two studies (an experimental study and a corpus-based study) were conducted in which the readability of target texts was calculated using different formulas and then utilized to develop models that predict particular eye movement patterns during reading. These studies revealed that although traditional formulas showed reliable performance in predicting particular eye movement patterns, in many cases, the newer formulas outperformed them. These findings support the newer readability formulas as more theoretically valid and accurate measures of the processing effort involved in L2 reading.