This paper describes a preliminary validation study of the Instructional Material Motivational Survey (IMMS) derived from the Attention, Relevance, Confidence and Satisfaction motivational design model. Previous studies related to the IMMS, however, suggest its practical application for motivational evaluation in various instructional settings without the support of empirical data. Moreover, there is a lack of discussion regarding the validity of the instrument. Therefore, this study empirically examined the IMMS as a motivational evaluation instrument. A computer-based tutorial setting was selected owing to its wide application in teaching large entry-level college courses. Data collected from 875 subjects were subjected to exploratory and confirmatory factor analyses, and measurement modeling LISREL. Findings suggested that 16 original items should be excluded from the IMMS; the retained 20 items were found to fall into different constructs, indicating that instructional features of the tutorial may influence the validity of the survey items. The implication of the study supports the situational feature of the IMMS. Therefore, a preevaluation adjustment on the IMMS items is recommended to identify suitable items before the full motivational evaluation. Future research should focus on the further validation of the IMMS based on this preliminary evidence.