No, within a broad class of scenarios. Gravitational-wave (GW) astronomy will open a new window on compact objects such as neutron stars and black holes (BHs). It is often stated that large signal-to-noise detections of ringdown or inspiral waveforms can provide estimates of the masses and spins of compact objects to within fractions of a percent, as well as tests of general relativity. These expectations usually neglect the realistic astrophysical environments in which compact objects live. With the advent of GW astronomy, environmental effects on the GW signal will eventually have to be quantified. Here we present a wide survey of the corrections due to these effects in two situations of great interest for GW astronomy: the BH ringdown emission and the inspiral of two compact objects (especially BH binaries). We mainly focus on future space-based detectors such as eLISA, but many of our results are also valid for ground-based detectors such as aLIGO, aVirgo, and KAGRA. We take into account various effects such as electric charges, magnetic fields, cosmological evolution, possible deviations from general relativity, firewalls, and the effects related to various forms of matter such as accretion disks and dark matter halos. Our analysis predicts the existence of resonances dictated by the external mass distribution, which dominate the very late-time behavior of merger and ringdown waveforms. The mode structure can drastically differ from the vacuum case, yet the BH response to external perturbations is unchanged at the time scales relevant for detectors. This is because, although the vacuum Schwarzschild resonances are no longer quasinormal modes of the system, they still dominate the response at intermediate times. Our results strongly suggest that both parametrized and ringdown searches should use at least two-mode templates. Our analysis of compact binaries shows that environmental effects are typically negligible for most eLISA sources, with the exception of very few special extreme-mass-ratio inspirals. We show, in particular, that accretion and hydrodynamic drag generically dominate over self-force effects for geometrically thin disks, whereas they can be safely neglected for geometrically thick disk environments, which are the most relevant for eLISA. Finally, we discuss how our ignorance of the matter surrounding compact objects implies intrinsic limits on the ability to constrain strong-field deviations from general relativity.