To evaluate multidimensional factor structure, a popular method that combines features of confirmatory and exploratory factor analysis is Bayesian structural equation modeling with small-variance normal priors (BSEM-N). This simulation study evaluated BSEM-N as a variable selection and parameter estimation tool in factor analysis with sparse cross-loading structures, focusing on ordered categorical data. A sensitivity analysis was conducted by assigning eight choices of small-variance priors on all potential cross-loadings. Results indicated that variable selection was performed well in a sparse loading structure in which the number of essential cross-loadings was small and the magnitudes were relatively large. Characteristics of ordinal items did not seem to have a sizable impact on parameter estimation. If the number of cross-loading estimates were small and centered around zero, BSEM-N may serve more efficiently as a tool for parameter estimation.