During face-to-face interactions typically developing individuals use gaze aversion (GA), away from their questioner, when thinking. GA is also used when individuals with autism (ASD) and Williams syndrome (WS) are thinking during question-answer interactions. We investigated GA strategies during face-to-face social style interactions with familiar and unfamiliar interlocutors. Participants with WS and ASD used overall typical amounts/patterns of GA with all participants looking away most while thinking and remembering (in contrast to listening and speaking). However there were a couple of specific disorder related differences: participants with WS looked away less when thinking and interacting with unfamiliar interlocutors; in typical development and WS familiarity was associated with reduced gaze aversion, however no such difference was evident in ASD. Results inform typical/atypical social and cognitive phenotypes. We conclude that gaze aversion serves some common functions in typical and atypical development in terms of managing the cognitive and social load of interactions. There are some specific idiosyncracies associated with managing familiarity in ASD and WS with elevated sociability with unfamiliar others in WS and a lack of differentiation to interlocutor familiarity in ASD. Regardless of the familiarity of the interlocutor, GA is associated with thinking for typically developing as well as atypically developing groups. Social skills training must take this into account. (Contains 3 tables.)