Background: The world population is aging rapidly. However, no study exists that examines specifically the effect of sleep time on memory among the middle-aged and elderly. Purpose: This paper examines the effect of sleep time on memory for the middle-aged and elderly. Methods: Data from the "China Health and Retirement Longitudinal Study" (CHARLS) were analyzed using fixed-effects models for panel data. The final sample used for this study included 4,330 middle-aged and elderly adults, of which about 50.4% were male. Results: The effect of sleep time on memory is heterogeneous across gender. One more hour of sleep time per night was estimated to increase the probability of "good memory" by roughly 0.2% (p > 0.1) for men and 1.6% (p < 0.01) for women. The odds ratios of "good memory" for men and women were estimated to be around 1.05 (p > 0.1) and 1.6 (p < 0.01), respectively. Discussion: This study suggests that efforts aimed at improving sleep time can bring significant memory-related benefit to middle-aged and elderly women. Translation to Health Education Practice: Health care providers and health educators should play a role in raising middle-aged and elderly women's general awareness of the memory-related benefit of sleep.