Radar data cleansing with deep learning
Résumé
Quality of radar data is critical for climatology, analysis and forecast. Actual algorithms present some drawbacks. For example, outer edges of rainy areas are often mistaken. Typing and cleansing of radar echoes is an ongoing challenge for over seventy years. It consists in telling which physical obstacles have been detected and how to remove irrelevant echoes. Typing refers to a pixel wise classification, and corresponds to a segmentation task. In the field of meteorological radar data processing, former algorithms were based on thresholding and decision trees. More objective approaches, based on fuzzy logic have been also developed. In the SERVAL operational algorithm of Meteo France, the pixel typing begins with a first basic decision tree followed by a fuzzy logic classification step and a contextual correction to avoid isolated pixels with a different type.
Deep convolutional neural networks (DCNN) have been chosen as a basis for several reasons. First for their high scoring in the domain of image processing, in particular, segmentation tasks has been successfully approached in biomedicine and satellite imagery. This success lies on DCNNs ability to exploit long range information and abstract representations on picture contents. Hence, long range patterns, associated with a meteorological context or specific non precipitating echoes could be handled by DCNN for typing or separation tasks.
In these study data have been recovered thanks to CMR (centre de météorologie radar, Météo France) and count 100k radar pictures. But these pictures lack human typing or cleaning by an expert. Pixels can belong to one of the following classes: pure precipitating echoes, non- precipitating echoes or mixed echoes. A database with pictures which contain only non- precipitating echoes collected during non-rainy periods (thanks to a collocated network of pluviometers) was set up. A second database corresponding to rainy periods was also built. However this database contains pictures with pure precipitating echoes mixed with some non- precipitating echoes. Training databases were built by merging these two databases. However, due to the presence of non-precipitating echoes in the second database some labeling errors exist and could be interpreted as label noise. Hence, part of this work is devoted to improvement of robustness w.r.t. label noise.
First networks were trained on cleansing and segmentation tasks. Once segmented, non- recipitating echoes could be removed and concerned pixels are inpainted with a network trained on inpainting task. It allows to develop an indirect cleansing method. Direct and indirect cleansing methods are evaluated and compared with Météo-France operational algorithm.