Ongoing ocean acidification affects marine calcification, although the scope and magnitude of this impact is essentially unknown. Here, we investigate the evolutionary origin of shell building in foraminifera to understand the long-term interplay between ocean carbon chemistry and calcification. Our analysis of shell chemical composition reveals multiple, independent origins for foraminiferal calcification throughout the Phanerozoic. Differences between orders reflect the different physiological controls employed by foraminifera to take up Ca²⁺ and inorganic carbon from seawater for CaCO₃ precipitation. With the long timespan involved, variability in seawater chemistry provided contrasting environments for calcification to arise, resulting in the diverse calcification strategies that exist today. This, in turn, explains the opposite responses of shell building to carbon perturbations. Our results call for adopting an evolutionary perspective when predicting the impact of perturbations on marine calcification and thereby, on the global carbon cycle.