This study tested the hypothesis that memory is a major source of variance in temporal processing. Participants categorized intervals as short or long. The number of base durations and interval types mixed within blocks of trials varied from 1 session to another. Results revealed that mixing 2 base durations within blocks increased categorization errors, but mixing 2 marker types did not. Results are attributed to the involvement of more than 1 memory representation, which is argued to show the critical role of memory in temporal processing. Because mixing modalities has no such effect, it was argued that modalities share a common representation in memory. Finally, there was no difference in the perceived duration of auditory and visually marked intervals, which is inconsistent with most reports on this effect.