Despite the implication of aerosols for the radiation budget, there are persistent differences in data for the aerosol optical depth (τ) for 1998-2019. This study presents a comprehensive evaluation of the large-scale spatio-temporal patterns of mid-visible τ from modern data sets. In total, we assessed 94 different global data sets from eight satellite retrievals, four aerosol-climate model ensembles, one operational ensemble product, two reanalyses, one climatology and one merged satellite product. We include the new satellite data SLSTR and aerosol-climate simulations from the Coupled Model Intercomparison Project Phase 6 (CMIP6) and the Aerosol Comparisons between Observations and Models Phase 3 (AeroCom-III). Our intercomparison highlights model differences and observational uncertainty. Spatial mean τ for 60°N - 60°S ranges from 0.124 to 0.164 for individual satellites, with a mean of 0.14. Averaged τ from aerosol-climate model ensembles fall within this satellite range, but individual models do not. Our assessment suggests no systematic improvement compared to CMIP5 and AeroCom-I. Although some regional biases have been reduced, τ from both CMIP6 and AeroCom-III are for instance substantially larger along extra-tropical storm tracks compared to the satellite products. The considerable uncertainty in observed τ implies that a model evaluation based on a single satellite product might draw biased conclusions. This underlines the need for continued efforts to improve both model and satellite estimates of τ, for example, through measurement campaigns in areas of particularly uncertain satellite estimates identified in this study, to facilitate a better understanding of aerosol effects in the Earth system.