The "Next Generation Science Standards" calls for new assessments that measure students' integrated three-dimensional science learning. The National Research Council has suggested that these assessments utilize a combination of item formats including constructed-response and multiple-choice. In this study, students were randomly assigned three-dimensional assessments that contained either constructed-response or multiple-choice versions of items. Rasch analysis was used to compare the difficulty of these items on the same construct scale. We found that partial-credit constructed-response items were located at similar places on the scale as their multiple-choice counterpart but dichotomously-scored constructed-response items were considerably more difficult. This suggests that scoring method influences an item's difficulty, which has implications for what inferences are made from assessments.