Studies aggressive automated word removal in text categorization in large databases based on corpus statistics to reduce the noise in free texts and to enhance the computational efficiency of categorization. Topics include stop word identification, categorization methods for comparison, tests on four document collections, and evaluation…

Describes a procedure for information retrieval testing that is based on the comparison of statistically independent methods of retrieval applied to the same database. The probability ranking principle is discussed, the statistical meaning of relevance is examined, and the methodology is illustrated on a large database of MEDLINE records. (19…

Studies the performance of probabilistic information retrieval systems using differing statistical dependence assumptions when estimating the probabilities inherent in the retrieval model. Experimental results using the Bahadur Lazarsfeld expansion on the Cystic Fibrosis database are discussed that suggest that incorporating term dependence…