Despite problem solving being a core skill in chemistry, students often struggle to solve chemistry problems. This difficulty may arise from students trying to solve problems through memorising algorithms. Goldilocks Help serves as a problem-solving scaffold that supports students through structured problem solving and its elements, such as planning and evaluation. In this study, we investigated how first-year chemistry students solved problems, when taught with Goldilocks Help, and whether their problem-solving success and approaches changed over the course of one semester. The data comprised of student written problem-solving work, and was analysed using frequency analysis and grouped based on the problem-solving success and the extent of the demonstrated problem-solving elements. Throughout the course of semester, students exhibited increasingly consistent demonstration of structured problem solving. Nonetheless, they encountered difficulties in fully demonstrating such aspects of problem solving as understanding and evaluating concepts, which demand critical thinking and a firm grasp of chemistry principles. Overall, the study indicated progress in successful and structured problem solving, with a growing proportion of students demonstrating an exploratory approach as time progressed. These findings imply the need for incorporation of metacognitive problem-solving scaffolding, exposure to expert solutions, reflective assignments, and rubric-based feedback into wide teaching practice. Further research is required to extend the exploration of the effectiveness of metacognitive scaffolding, in particular "via" think-aloud interviews, which should help identify productive and unproductive uses of the problem-solving elements.