The links between climate variability, depicted by time series of oceanic indices, and changes in total water and groundwater storage are investigated across nine large aquifer basins of the African continent. The Gravity Recovery and Climate Experiment (GRACE) mission’s observations represent a remarkable tool that can provide insight into the dynamics of terrestrial hydrology in areas where direct in situ observations are limited. In order to evaluate the impact of interannual and multidecadal climate variability on groundwater resources, this study assesses the relationship between synoptic controls on climate and total water storage estimates from (i) GRACE from 2002 to 2013 and (ii) a two-variable climate-driven model that is able to reconstruct past storage changes from 1982 to 2011. The estimates are then compared to time series of groundwater levels to show the extent to which total water storage covaries with groundwater storage. Results indicate that rainfall patterns associated with the El Niño Southern Oscillation (ENSO) are the main driver of changes in interannual groundwater storage, whereas the Atlantic MultiDecadal Oscillation (AMO) plays a significant role in decadal to multidecadal variability. The combined effect of ENSO and AMO could trigger significant changes in recharge to the aquifers and groundwater storage, in particular in the Sahel. These findings could help decision-makers prepare more effective climate-change adaptation plans at both national and transboundary levels.