Non-verbal communication plays a major role in social interaction understanding. Using functional magnetic resonance imaging, we explored the development of the neural networks involved in social interaction recognition based on human motion in children (8-11), adolescents (13-17), and adults (20-41). Participants watched point-light videos depicting two actors interacting or moving independently and were asked whether these agents were interacting or not. All groups successfully performed the discrimination task, but children had a lower performance and longer response times than the older groups. In all three groups, the posterior parts of the superior temporal sulci and middle temporal gyri, the inferior frontal gyri and the anterior temporal lobes showed greater activation when observing social interactions. In addition, adolescents and adults recruited the caudate nucleus and some frontal regions that are part of the mirror system. Adults showed greater activations in parietal and frontal regions (part of them belonging to the social brain) than adolescents. An increased number of regions that are part of the mirror system network or the social brain, as well as the caudate nucleus, were recruited with age. In conclusion, a shared set of brain regions enabling the discrimination of social interactions from neutral movements through human motion is already present in 8-year-old children. Developmental processes such as refinements in the social brain and mirror system would help grasping subtle cues in non-verbal aspects of social interactions.