Since the late 1970s, successive satellite missions have been monitoring the sun's activity and recording the total solar irradiance (TSI). Some of these measurements have lasted for more than a decade. In order to obtain a seamless record whose duration exceeds that of the individual instruments, the time series have to be merged. Climate models can be better validated using such long TSI time series which can also help to provide stronger constraints on past climate reconstructions (e.g., back to the Maunder minimum). We propose a 3-step method based on data fusion, including a stochastic noise model to take into account short and long-term correlations. Compared with previous products scaled at the nominal TSI value of ∼1361 W/m², the difference is below 0.2 W/m² in terms of solar minima. Next, we model the frequency spectrum of this 41-year TSI composite time series with a Generalized Gauss-Markov model to help describe an observed flattening at high frequencies. It allows us to fit a linear trend into these TSI time series by joint inversion with the stochastic noise model via a maximum-likelihood estimator. Our results show that the amplitude of such trend is ∼−0.004 ± 0.004 W/(m²yr) for the period 1980-2021. These results are compared with the difference of irradiance values estimated from two consecutive solar minima. We conclude that the trend in these composite time series is mostly an artifact due to the colored noise.