Children begin to learn abstract rules at an early age, in an implicit way, without access to rule descriptions. They rely on specific rule instances that they encounter. However, rule instances often co-occur with rule-inconsistent instances. One kind of inconsistent input, non-application instances, constitutes a learnability problem. For example, a child might hear many instances of the dative shift rule in English, such as "Mary gave the book to me ? Mary gave me the book," and partial cases such as "He will bring toys to you" or "He donated books to John," without being told which of the latter sentences (i.e., non-application instances) would be rule-possible or exceptions. We examined whether and how non-application instances may impact rule learning per se. Fourteen-month-old infants were passively exposed to an unfamiliar natural language. In Experiment 1 half of the training input supported an artificial word-order shift rule, and the other half were non-application singletons without shifting. During test, infants failed to generalize the rule to new instances, suggesting that the non-application cases in training were treated as non-rule cases and might have impeded rule learning. In Experiment 2 rule instances were dominant in type frequency relative to non-rule instances in the training input, and infants showed rule generalization, confirming that the non-rule instances in Experiment 1 indeed impeded learning. The token frequency of individual instances did not affect rule generalization. Experiments 3 and 4 replicated the same findings (of Experiments 1 and 2 respectively) with stimuli containing no morphological cues, demonstrating that the mechanism underlying abstract rule learning is robust, likely being one of the earliest learning mechanisms available to humans.