Background: Complex problem-solving competence is regarded as a key construct in science education. But due to the necessity of using interactive and intransparent assessment procedures, appropriate measures of the construct are rare. This paper consequently presents the development and validation of a computer-based problem-solving environment, which can be used to assess students' performance on complex problems in Chemistry. The test consists of four scales, namely, understanding and characterizing the problem, representing the problem, solving the problem, and reflecting and communicating the solution. Based on this four-dimensional framework, the computer-based assessment has been evaluated with the data of N = 395 10th grade high school students. Results: Result showed that students' complex problem-solving competence could be modelled by four related but empirically distinct factors with moderate to high intercorrelations. The construct showed substantial relations with fluid intelligence and prior domain knowledge in Chemistry, indicating that construct validity and domain specificity were given. Processes of understanding and characterizing the problem were substantially related to subsequent processes in complex problem solving. Conclusions: Due to the complexity of complex problem-solving processes in Chemistry, multidimensionality of the construct could be assumed. Consequently, science educators should take into account abilities of understanding, representing, solving the problem, and finally reflecting and communicating the solution when developing instructional approaches and valid computer-based assessments.