The Video Engagement Scale (VES) is a quality indicator to assess engagement in experimental video-vignette studies, but its measurement properties warrant improvement. Data from previous studies were combined (N = 2676) and split into three subsamples for a stepped analytical approach. We tested construct validity, criterion validity, reliability, and measurement invariance. Confirmatory factor analysis (CFA) reiterated poor fit of the previously suggested subscales. An alternative 2-factor structure is presented, which was cross-validated and further shortened (VES-sf). Reliability of both subscales ("Immersion," "Emotional Impact") was very high ([alpha] > 0.86). Multi-group CFAs demonstrated partial and complete measurement invariance among male/female and differently educated participants respectively. Invariance between healthy versus ill participants was not established, but CFAs supported the VES-sf structure in both groups separately. To conclude, we provide the VES-sf with sound measurement properties and demonstrated the appropriateness of comparing certain subgroups. Overall, we recommend the VES-sf to assess engagement/ecological validity in video-vignette research.