Electromagnetic coupling between the Earth's core and mantle is one of the proposed mechanisms to explain length of day (LOD) variations on decadal timescales. Mantle conductivity, a determining parameter in this process, is only poorly known. Earlier work on core-mantle coupling mostly assumed laterally homogeneous mantle conductivities. However, the lower mantle is a highly inhomogeneous region. Seismic evidence is growing that many of the inhomogeneities must have chemical, compositional, or thermal origins. We consider the effect of different laterally varying conductivity models on electromagnetic mantle torques. Torque amplitudes for a specific epoch can depend strongly on the assumed conductivity distribution. However, when comparing time series of the electromagnetic torque to decadal LOD variations, none of the examined conductivity models improve the agreement significantly. As in the simplest case of a homogeneous mantle conductivity, a minimum average conductance of 10⁸ S is always required to make electromagnetic coupling efficient.