Studying comets is believed to bring invaluable clues on the formation and evolution of our planetary system. In comparison to planets, they have undergone much less alteration, and should have therefore retained a relatively pristine record of the conditions prevailing during the early phases of the solar system. However, comets might not be entirely pristine. As of today, we have not been able to determine which of the observed physical, chemical and orbital characteristics of comets, after they have evolved for more than 4 Gyr in a time-varying radiative and collisional environment, will provide the best clues to their origin. Comet physical characteristics as inherited from their formation stage may be very diverse, both in terms of composition and internal structure. The subsequent evolution of comet nuclei involves some possible processing from radiogenic heating, space weathering and large- and small-scale collisions, which might have modified their primordial structures and compositions with various degrees. When comets enter the inner solar system and become active, they start to lose mass at a very high rate. The effects of activity on comet nuclei involve a layering of the composition, a substantial non-even erosion and modification of their size and shape, and may eventually result in the death of comets. In this review, we present the dominating processes that might affect comet physical and chemical properties at different stages of their evolution. Although the evolutionary track may be specific to each comet, we can focus on long-lasting modifications which might be common to all nuclei after their formation stage, during their storage in reservoirs in the outer solar system, and once comets enter the inner solar system and become active objects.