While dichotomous item dominates the application of computerized adaptive testing (CAT), polytomous item and set-based item hold promises for being incorporated in CAT. However, how to assemble a CAT containing mixed item formats is challenging. This study investigated: (1) how the mixed CAT works compared with the dichotomous-item-based CAT; (2) how to develop a procedure assembling a CAT with mixed item formats. Two test assembly designs, shadow test and bin-structured approach, were compared under a variety of test conditions. The item pool shape, IRT model used during CAT (modified three-parameter model and two-parameter model), and test constrains for exposure rate, cognitive skills and content balance were manipulated to simulate various real test situations. The effect of test assembly was evaluated in terms of ability measurement accuracy, content balance, test security, and item usage. The results showed that (1) compared with dichotomous-item-based CAT, the mixed-item-based CAT was more efficient but had higher exposure rate; (2) the effects of bin-structured method and shadow test were comparable in test accuracy and content balance, while bin-structure method can improve the item usage and lead to more even distribution of exposure rate.