Human sexual dimorphism has been widely misunderstood. A large literature has underestimated the effect of differences in body composition and the role of male contest competition. It is often assumed that sexually dimorphic traits reflect a history of sexual selection, but natural selection frequently builds different phenotypes in males and females. The relatively small sex difference in stature (~7%) and its decrease during human evolution have been widely presumed to indicate decreased male contest competition for mates. However, females likely increased in stature relative to males in order to successfully deliver large-brained neonates through a bipedally-adapted pelvis. Despite the relatively small differences in stature and body mass (~16%), there are marked sex differences in body composition. Across multiple samples from groups with different nutrition, males typically have 36% more lean body mass, 65% more muscle mass, and 72% more arm muscle than women, yielding parallel sex differences in strength. These sex differences in muscle and strength are comparable to those seen in primates where sexual selection, arising from aggressive male mating competition, has produced high levels of dimorphism. Body fat percentage shows a reverse pattern, with females having ~1.6 times more than males and depositing that fat in different body regions than males. We argue that these sex differences in adipose arise mainly from natural selection on women to accumulate neurodevelopmental resources.