The Sea Surface Height (SSH) is an important variable of the ocean state. It is currently estimated by satellites measuring the return time of a radar pulse. Due to this remote sensing technology, nadir-pointing altimeters take measures vertically, only along their ground tracks. Recovering fully gridded SSH fields involves a challenging spatiotemporal interpolation. The most widely used operational product, the Data Unification and Altimeter Combination System (DUACS), combines data from several satellites through linear optimal interpolation to estimate the SSH field. However several studies demonstrate that DUACS does not resolve mesoscale structures, motivating our interest in improving interpolation methods. Recently, Deep Learning has emerged as one of the leading methods to solve ill-posed inverse imaging problems. Deep Neural Networks can use multi-variate information to constrain the interpolation. Among them, Sea Surface Temperature (SST) data is based on a different remote-sensing technology, which leads to higher data coverage and resolution. Deep Learning methods have been proposed to interpolate SSH from track measurements, efficiently using SST contextual information. However, training neural networks usually requires either a realistic simulation of the problem on which we have access to SSH ground truth or a loss function that does not require it. Both solutions present limitations: the first is likely to suffer from domain gap issues once applied to real-world data, and training on observations only leads to lower performance than supervision on complete fields. We propose a hybrid method: a supervised pretraining on a realistic simulation, and an unsupervised fine-tuning on real-world observations. This approach was performed using a deep Attention-based Encoder-Decoder architecture. We compare the performances of the same neural network architecture trained in the three described settings: simulation-based training, observation-based training, and our hybrid approach. Preliminary results show an improvement of approximately 25% over DUACS in the interpolation task on the Ocean Data Challenge 2021 dataset. We further explore the ability of the architecture proposed to produce near real-time forecasts of SSH.