Pulsar wind nebulae (PWNe) are outstanding accelerators in Nature, in the sense that they accelerate electrons up to the radiation reaction limit. Motivated by this observation, this paper examines the possibility that young pulsar wind nebulae can accelerate ions to ultra-high energies at the termination shock of the pulsar wind. We consider here powerful PWNe, fed by pulsars born with ~ millisecond periods. Assuming that such pulsars exist, at least during a few years after the birth of the neutron star, and that they inject ions into the wind, we find that protons could be accelerated up to energies of the order of the Greisen-Zatsepin-Kuzmin cut-off, for a fiducial rotation period P ~ 1 msec and a pulsar magnetic field B_star ~ 10¹³ G, implying a fiducial wind luminosity L_p ~ 10⁴⁵ erg/s and a spin-down time t_sd ~ 3× 10⁷ s. The main limiting factor is set by synchrotron losses in the nebula and by the size of the termination shock; ions with Z>= 1 may therefore be accelerated to even higher energies. We derive an associated neutrino flux produced by interactions in the source region. For a proton-dominated composition, our maximum flux lies slightly below the 5-year sensitivity of IceCube-86 and above the 3-year sensitivity of the projected Askaryan Radio Array. It might thus become detectable in the next decade, depending on the exact level of contribution of these millisecond pulsar wind nebulae to the ultra-high energy cosmic ray flux.