Processing efficiency theory can explain the relationship between anxiety and academic success; however, its application to adults with Specific Learning Disabilities (SLD) remains unclear, especially in a nonalphabetic language, such as Chinese. This study investigated the effects of working memory and processing speed on the relationships between state anxiety and academic performance of university students with and without SLD in Chinese. A sample of 223 s-year undergraduate students was recruited from universities in southern Taiwan; 123 were typical learners, while the remaining 100 were identified as having SLD. We found distinct profiles in the relationships between state anxiety, working memory, processing speed, and academic performance. The interaction between state anxiety and working memory was also predictive of the academic performance of university students with SLD, highlighting the negative impact of state anxiety on those students who performed poorly in working memory tasks. Our findings emphasize the importance of cognitive and psychological factors in contributing to the learning of students with SLD. Furthermore, the effects of working memory and state anxiety on academic performance, particularly in students with SLD, could inform the design of teaching materials and procedures, especially regarding the levels of difficulty and volumes of learning.