We present a systematic survey of numerical geodynamo simulations where the inner core is allowed to differentially rotate in the longitudinal direction with respect to the mantle. We focus on the long-term behaviour of inner core rotation, on timescales much longer than the overturn time of the fluid outer core, including the steady component of rotation. The inner core is subject to viscous and magnetic torques exerted by the fluid outer core, and a gravitational restoring torque exerted by the mantle. We show that the rate of steady inner core rotation is limited by the differential rotation between spherical surfaces that the convective dynamics can sustain across the fluid outer core. We further show that this differential rotation is determined by a torque balance between the resistive Lorentz force and the Coriolis force on spherical surfaces within the fluid core. We derive a scaling law on the basis of this equilibrium suggesting that the ratio of the steady inner core rotation to typical angular velocity within the fluid core should be proportional to the square root of the Ekman number, in agreement with our numerical results. The addition of gravitational coupling does not alter this scaling, though it further reduces the amplitude of inner core rotation. In contrast, the long-term fluctuations in inner core rotation remain proportional to the fluid core angular velocity, with no apparent dependency on the Ekman number. If the same torque balance pertains to the Earth's core conditions, the inner core rotation then consists in a very slow super rotation of a few degrees per million years, superimposed over large fluctuations (at about a tenth of a degree per year). This suggests that the present-day seismically inferred inner core rotation is a fragment of a time-varying signal, rather than a steady super rotation. For the inner core rotation fluctuations not to cause excessive variations in the length-of-day, the strength of the gravitational coupling between the inner core and the mantle must be smaller than previously published values. We finally explore how the torque balance which we observe in our models could be altered in planetary cores, yielding possibly larger values of the steady rotation.