[1] The impact of the heterogeneous lower mantle on the geomagnetic field is under debate, especially the question of whether high‐latitude intense geomagnetic flux patches currently observed at the core surface are stationary, oscillating, or drifting on longer time scales. While the correlation between the location of these patches with that of similar patches found in the time‐averaged paleomagnetic field may suggest stationary behavior, their variability over archaeomagnetic time scales together with their weaker signature in the average paleomagnetic field relative to the present geomagnetic field precludes such a scenario. Here we use numerical dynamos with an imposed heat flux boundary condition based on lower mantle tomography to study the behavior of such intense magnetic flux patches. We design an algorithm to detect centers of intense flux patches and track their time evolution. We find that the time‐dependent nature of those patches comprises oscillatory motion about statistically preferred locations imposed by mantle control, with episodic drift from one preferred location to the other corresponding to an azimuthal migration of fluid downwelling structures that concentrate surface magnetic flux. This statistical behavior provides a possible explanation for both the observed variability of high‐latitude patches on the archaeomagnetic time scale and the similar locations of the current patches and the weaker patches seen in the paleomagnetic field. Our simulations also show that the patches exhibit more time dependence and less coherency in the southern hemisphere, leading to a weaker time‐averaged patch signature in that hemisphere.