Visual working memory (VWM) is traditionally assumed to be immune to proactive interference (PI). However, in a recent study (Endress & Potter, 2014), performance in a visual memory task was superior when all items were unique and hence interference from previous trials was impossible, compared to a standard condition in which a limited set of repeating items was used and stimuli from previous trials could interfere with the current trial. Furthermore, when all the items were unique, the estimated memory capacity far exceeded typical capacity estimates. Consequently, the researchers suggested the existence of a separate memory buffer, the "temporary memory," which has an unbounded capacity for meaningful items. However, before accepting this conclusion, methodological differences between the repeated-unique procedure and typical estimates of VWM should be considered. Here, we tested the extent to which the exceptional set of heterogeneous, complex, meaningful real-world objects contributed to the large PI in the repeated-unique procedure. Thus, the same paradigm was employed with a set of real-world objects and with homogenous sets (e.g., houses, faces) in which the items were meaningful, yet less visually distinct, and participants had to rely on subtle visual details to perform the task. The results revealed a large PI effect for real-world heterogeneous objects, but substantially smaller effects for the homogenous sets. These findings suggest that there is no need to postulate a new memory buffer. Instead, we suggest that VWM capacity and vulnerability to PI are highly influenced by task characteristics, and specifically, by the stimuli distinctiveness.