We derive a model for the steady fluid flow at the top of Earth's core driven by thermal coupling with the heterogeneous lower mantle. The model uses a thermal wind balance for the core flow, and assumes a proportionality between the horizontal density gradients at the top of the core and horizontal gradients in seismic shear velocity in the lowermost mantle. It also assumes a proportionality between the core fluid velocity and its radial shear. This last assumption is validated by comparison with numerical models of mantle-driven core flow, including self-sustaining dynamo (supercritical) models and non-magnetic convection (subcritical) models. The numerical dynamo models show that thermal winds with correlated velocity and radial shear dominate the boundary-driven large-scale flow at the top of the core. We then compare the thermal wind flow predicted by mantle heterogeneity with the 150 year time-average flow obtained from inverting the historical geomagnetic secular variation, focusing on the non-zonal components of the flows because of their sensitivity to the boundary heterogeneity. Comparing magnitudes provides an estimate of the ratio of lower mantle seismic anomalies to core density anomalies. Comparing patterns shows that the thermal wind model and the time-average geomagnetic flow have comparable length scales and exhibit some important similarities, including an anticlockwise vortex below the southern Indian and Atlantic Oceans, and another anticlockwise vortex below Asia, suggesting these parts of the non-zonal core flow could be thermally controlled by the mantle. In other regions, however, the two flows do not match well, and some possible reasons for the dissimilarity between the predicted and observed core flow are identified. We propose that better agreement could be obtained using core flows derived from geomagnetic secular variation over longer time periods.