Topic models are mathematical and statistical models used to analyze textual data. The objective of topic models is to gain information about the latent semantic space of a set of related textual data. The semantic space of a set of textual data contains the relationship between documents and words and how they are used. Topic models are becoming…

In secure web application development, the role of web services will not continue if it is not trustworthy. Retaining customers with applications is one of the major challenges if the services are not reliable and trustworthy. This article proposes a trust evaluation and decision model where the authors have defined indirect attribute, trust,…

The subject of the research is to build methodologies to evaluate the student knowledge by testing. The author points to the importance of feedback about the mastering level in the learning process. Testing is considered as a tool. The object of the study is to create the test system models for defence practice problems. Special attention is paid…

The online deployment of Technology Engineered online Student Ratings of Instruction (SRIs) by colleges and universities in the United States has dynamically changed the deployment of course evaluation. This research investigation is the fourth part of a post hoc study that analytically and psychometrically examines the design, reliability, and…

This annotated bibliography was developed upon noting a deficiency of information in the literature regarding the training of raters for establishing agreement. The ERIC descriptor, "Interrater Reliability", was used to locate journal articles. Some of the 33 resulting articles focus on mathematical concepts and present formulas for computing…

This report gives some simple examples of stability for one factor and 2 x 2 factorial analysis of variance, reliability and correlations. The findings are very different: from superstability (no transformation whatsoever can change the result) to almost total instability. This is followed by a discussion of applications to multivariate analysis,…

A large number of seemingly diverse coefficients have been proposed as indices of dependability, or reliability, for domain-referenced and/or mastery tests. In this paper, it is shown that most of these indices are special cases of two generalized indices of agreement: one that is corrected for chance, and one that is not. The special cases of…

The magnitude of statistical bias for the phi-coefficient was investigated, using computer simulated examinations in which all the students had equal knowledge. Several modifications of phi were tested, but when applied to real examinations, none succeeded in improving its reproducibility when items are re-used on equivalent student groups.…

Sixteen measures of reliability for two-category nominal scales are compared. Upon correcting for chance agreement, there are only five distinct indices: Fleiss's modification of A-sub-1, the phi coefficient, Cohen's kappa, and two intraclass coefficients. Recommendations for choosing an agreement index are made based on definitions, magnitude,…

This paper examines the properties of the Item Efficiency Index proposed by Neill and Jackson (1976; EJ 137 077) for minimum redundancy item analysis. (Author/BW)

A mathematical link between test reliability and test validity is derived, taking into account the correlation between error scores on a test and error scores on a criterion measure. When this correlation is positive, the "paradoxical" nonmonotonic relation between test reliability and test validity occurs universally. (Author/BW)

Limitations and extensions of Feldt's approach to testing the equality of Cronbach's alpha coefficients in independent and matched samples are discussed. In particular, this approach is used to test equality of intraclass correlation coefficients. (Author)

Large sample chi-square tests of the significance of the difference between two correlated kappas, weighted or unweighted, are derived. Cases are presented with one judge in common between the two kappas and no judge in common. An illustrative calculation is included. (Author/SLD)

A procedure for estimating the reliability of a factorially complex composite is considered. An application of its use with Scholastic Aptitude Test data is provided. (Author/JKS)