Speechreading is a visual communicative skill for perceiving speech. In this study, we tested the effects of speech experience and deafness on the speechreading neural network in normal hearing controls and in two groups of deaf patients who became deaf either before (prelingual deafness) or after (postlingual deafness) auditory language acquisition. Magnetic signals from the cerebral cortex were recorded using a 306-channel magnetoencephalographic system. During magnetoencephalographic measurements, subjects were asked to perform a speechreading task from video clips of a female speaker either pronouncing syllables (speechreading condition) or showing closed-mouth movement. The sources of the evoked fields were modelled using equivalent current dipoles, the origins of which were fitted to the intracranial space based on magnetic resonance imaging findings. During the speechreading condition, the latency of auditory cortex activation was shorter in the postlingual deafness group than in the normal hearing control group. This parameter negatively correlated with speechreading scores measured clinically. Furthermore, as the duration of deafness increased, the latency of auditory cortex activation decreased exponentially. However, no such correlation was found in the prelingual deafness group which differed significantly from the two other groups in this respect. The latency of auditory cortex activation was significantly longer in the prelingual deafness group than in the two other groups. Thus, auditory experience may be crucial for the development of a normal neural network for speechreading. The pre-existing speechreading network in the postlingual deafness group is made more efficient by speeding up the neural response.