We announce the discovery of the Aquarius 2 dwarf galaxy, a new distant satellite of the Milky Way, detected on the fringes of the VLT Survey Telescope (VST) ATLAS and the Sloan Digital Sky Survey (SDSS) surveys. The object was originally identified as an overdensity of red giant branch stars, but chosen for subsequent follow-up based on the presence of a strong blue horizontal branch, which was also used to measure its distance of ∼110 kpc. Using deeper imaging from the Inamori-Magellan Areal Camera and Spectrograph camera on the 6.5m Baade and spectroscopy with DEep Imaging Multi-Object Spectrograph on Keck, we measured the satellite's half-light radius 5.1 ± 0.8 arcmin, or ∼160 pc at this distance, and its stellar velocity dispersion of 5.4^{+3.4}_{-0.9} km s^-1. With μ = 30.2 mag arcsec^-2 and M_V = -4.36, the new satellite lies close to two important detection limits: one in surface brightness; and one in luminosity at a given distance, thereby making Aquarius 2 one of the hardest dwarfs to find.