Several studies on Virtual Exchange (VE) have highlighted positive learning experiences, increases in technological pedagogical and content skills (TPACK) and foreign language (FL) competence. However, most VE research to date use qualitative or descriptive case-studies of how VEs have been implemented, and what "might" have worked. In this large-scale quantitative two-study design, we explored how 622 pre-service teachers developed TPACK skills and (perceived) FL competence over time in 23 VEs across 34 institutions in 16 countries. In Study 1, we used a (quasi-) experimental design of 3 VEs in an experimental (n = 151) or control group (n = 77) to explore the impact on TPACK. In Study 2, we used a larger sample of 20 VEs and 394 participants to replicate and contrast the findings from Study 1 in a broader context. In contrast to our expectations, participants in the experimental condition did not have higher TPACK skills growth relative to the control condition in Study 1, which was further confirmed in Study 2. Nonetheless, in Study 2 pre-existing TPACK skills influenced the development of (perceived) FL competence over time, whereby those participants who further strengthened their TPACK skills during the VE were more likely to nurture FL competence. A major lesson from this large-scale implementation is that VEs do not generate TPACK skills and FL competence by osmosis. We encourage CALL researchers to carefully reflect on any positive or negative finding that something has "worked" when there is no comparison or control group included.