It has recently been suggested that planets can form by dust coagulation in the tori of active galactic nuclei (AGN) with low luminosities of L _bol ≲ 10⁴² erg s^-1, constituting a new class of exoplanets orbiting supermassive black holes called blanets. However, large dust grains in the AGN torus may be rotationally disrupted by the radiative torque disruption (RATD) mechanism due to AGN radiation feedback, which would prevent blanet formation. To test this scenario, we adopt a simple smooth and a clumpy dust/gas distribution inside the torus region to study the effect of RATD on the evolution of composite dust grains in the midplane of the torus. We found that grain growth and then blanet formation are possible in the smooth torus model. However, in the clumpy torus model, grain growth will be strongly constrained by RATD, assuming the gas density distribution as adopted by Wada et al. We also found that icy grain mantles inside clumps are quickly detached from the grain cores by rotational desorption, reducing the sticking coefficient between icy grains and the coagulation efficiency. The grain rotational disruption and ice desorption occur on timescales much shorter than the growth time up to a factor of ~10⁴, which are the new barriers that grain growth must overcome to form blanets. Further studies with more realistic AGN models are required to constrain better the effect of RATD on grain growth and blanet formation hypothesis around low-luminosity AGN.