The precise measurement of the masses and radii of stars in eclipsing binary systems provides a window into uncertain processes in stellar evolution, especially mixing at convective boundaries. Recently, these data have been used to calibrate models of convective overshooting in the cores of main sequence stars. In this study we have used a small representative sample of eclipsing binary stars with 1.25 ≤ M/M_⊙ < 4.2 to test how precisely this method can constrain the overshooting and whether the data support a universal stellar mass-overshooting relation. We do not recover the previously reported stellar mass dependence for the extent of overshooting and in each case we find there is a substantial amount of uncertainty, that is, the same binary pair can be matched by models with different amounts of overshooting. Models with a moderate overshooting parameter 0.013 ≤ f_os ≤ 0.014 (using the scheme from Herwig et al. 1997, A&A, 324, L81) are consistent with all eight systems studied. Generally, a much larger range of f is suitable for individual systems. In the case of main sequence and early post-main sequence stars, large changes in the amount of overshooting have little effect on the radius and effective temperature, and therefore the method is of extremely limited utility.