Real data are unlikely to be exactly normally distributed. Ignoring non-normality will cause misleading and unreliable parameter estimates, standard error estimates, and model fit statistics. For non-normal data, researchers have proposed a distributionally-weighted least squares (DLS) estimator to combines the normal theory based generalized least squares estimation (GLS[subscript N]) and WLS. The key in DLS is to select an optimal weight a[subscript s] to compute a weighted average of GLS[subscript N] and WLS. To better estimate a[subscript s] in DLS, we propose a method based on the delta method and the empirical Bayesian method. When data were normal, DLS and GLS[subscript N] provided similar root mean square errors (RMSEs) and biases of the standard error estimates, and were smaller than those from WLS. When the data were elliptical or skewed, DLS generally provided the smallest RMSEs and biases of the standard error estimates. Additionally, the Type I error rates of Jiang-Yuan rank adjusted test statistic (T[subscript JY]) using DLS were generally around the nominal level.