Successful problem solving is a complex process that requires content knowledge, process skills, developed critical thinking, metacognitive awareness, and deep conceptual reasoning. Teaching approaches to support students developing problem-solving skills include worked examples, metacognitive and instructional scaffolding, and variations of these techniques. In this report, we describe a classroom activity, which involves a combination of metacognitive scaffolding, problem-solving practice, and critiquing of ChatGPT-generated solutions. It was demonstrated that students engaged with the idea of metacognition as part of the problem-solving toolbox and showed appreciation for the collaborative nature of problem solving. They were also able to identify mistakes and flaws in the provided erroneous solutions, albeit to varying degrees. The results also revealed that incoming university students likely require scaffolding to develop sophisticated critical-thinking skills.