FOrecasting radiation foG by combining station and satellite data using Machine Learning (FOG-ML)
Final Report Abstract
The FOG-ML project aimed to improve the prediction of radiative fog formation and dissipation by combining machine learning (ML) techniques with various data sources, including ground station measurements and satellite data. The overall goal was to develop a powerful and robust fog prediction system that exploits the strengths of both types of data. In the first phase, the main challenges in training and validating ML models for fog prediction, in particular with the eXtreme Gradient Boosting (XGBoost) algorithm, were identified. It was found that problems such as temporal order and autocorrelation in the data lead to an overestimation of model performance. The introduction of baseline models and a more application-oriented evaluation method, the Expanding Window Approach (EWA), provided a more accurate assessment of the model's operational capabilities. In the next phase, the incorporation of pre-processed temporal data and smoothing techniques on the performance of the XGBoost model was investigated. This data preprocessing resulted in a significant increase in the accuracy of the model, with the F1 score increasing from 0.75 to 0.88. This demonstrates the importance of data pre-processing in improving short to medium term fog forecasts. The final phase evaluated the combination of ground-based station data with Meteosat Second Generation (MSG) satellite data for fog forecasting. The investigations showed that models using MSG data in combination with visibility measurements provide comparable forecast performance to models using only station data. The combined data set shows a balanced model performance, especially in the prediction of fog formation and resolution transitions. The use of MSG data thus represents a cost-effective alternative for large-scale fog forecasts. The project has successfully demonstrated the potential of ML in fog forecasting, in particular the combination of station and satellite data to improve the accuracy and efficiency of fog forecasting. The use of satellite data, which provides wide coverage, is a cost-effective alternative to relying on station data alone. The results lay the foundation for a comprehensive, nationwide fog forecasting system in Germany. Future work will focus on extending this approach to multiple stations and further refining the temporal dynamics of the model for longer term forecasts.
Publications
-
Forecasting radiation fog by combining station measurements and satellite data using machine learning. Presented at 38th AK Klima, Jesteburg.
Vorndran, M., Thies, B. & Bendix, J.
-
Training and validation weaknesses in pointwise classification-based radiation fog forecast using machine learning algorithms . Presented at 39th AK Klima, Passau.
Vorndran, M., Schütz, A., Bendix, J. & Thies, B.
-
Current Training and Validation Weaknesses in Classification-Based Radiation Fog Nowcast Using Machine Learning Algorithms. Artificial Intelligence for the Earth Systems, 1(2).
Vorndran, Michaela; Schütz, Adrian; Bendix, Jörg & Thies, Boris
-
The effect of filtering and preprocessed temporal information on a classification based machine learning model for radiation fog nowcasting. Presented at 40th AK Klima, Würzburg.
Vorndran, M., Schütz, A., Bendix, J. & Thies, B.
-
Improving classification‐based nowcasting of radiation fog with machine learning based on filtered and preprocessed temporal data. Quarterly Journal of the Royal Meteorological Society, 150(759), 577-596.
Schütz, Michaela; Schütz, Adrian; Bendix, Jörg & Thies, Boris
-
Machine learning based radiation fog nowcast with station data in Germany. Presented at 41st AK Klima, Tübingen.
Vorndran, M. Schütz, A., Bendix, J. & Thies, B.
-
Observation of low stratus and ground fog in Europe using Meteosat and AI. Invited talk at a workshop of the EUMETSAT Aviation Testbed Project, FMI Headquarters, Helsinki.
Bendix, J.
-
Pointwise Machine Learning Based Radiation Fog Nowcast with Station Data in Germany. Presented at 9th International Conference on Fog, Fog Collection, and Dew, Fort Collins, Colorado, USA.
Vorndran, M., Schütz, A., Bendix, J. & Thies, B.
-
A regression-based machine learning model for radiation fog nowcasting using station and satellite data. Presented at 42nd AK Klima 2024
Schütz, M., Schütz, A., Bendix, J., Müller J. & Thies, B.
-
A satellite-based analysis of fog and low stratus life cycle processes in the Po valley, Italy. Copernicus GmbH.
Pauli, Eva; Cermak, Jan; Andersen, Hendrik & Schütz, Michaela
-
Radiation fog nowcasting with XGBoost using station and satellite data. Copernicus GmbH.
Schütz, Michaela; Bendix, Jörg & Thies, Boris
