Diagnostic benchmarking of hybrid hydrological models in the UNITE framework (UNI-BENCH)

• Applicants: Dr.-Ing. Uwe Ehret; Dr. Anneli Guthke
• Subject Area: Hydrogeology, Hydrology, Limnology, Urban Water Management, Water Chemistry, Integrated Water Resources Management
• Term: since 2022
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 507884992

In the Earth Sciences, we increasingly rely on models to simulate the behaviour of the natural system, for hypothesis testing to deepen system understanding, operational forecasting, or predicting previously unobservable conditions as in the context of climate change. The hydrological community traditionally uses physics-based or conceptual models that represent our process understanding. In cases of data abundance, machine learning approaches have become popular, because they efficiently extract information from observed data and do not suffer from too rigid model structure assumptions as process-based models tend to do. Especially deep learning methods have shown impressive prediction skill even in extrapolation. Yet, they are criticized for their lack of interpretability and of physics consistency, which limits their acceptance and applicability to problems of societal relevance. To merge the best of two worlds, variants of physics-informed machine learning are being developed in an ever-increasing speed. The expectation of such hybrid approaches is to exploit the information extraction capability for interpolation skill, while gaining interpretability and (additional) extrapolation skill through the infusion of expert knowledge. In the currently funded UNITE project, we have addressed the research gap that hindered efficient learning from the various hybrid approaches: the hydrological community had lacked a formal basis to compare arbitrary modelling approaches. We have developed such a unified framework to systematically evaluate the quality of hybrid models based on information theory. We have identified reliable sample-based estimates of information-theoretic quantities for performance and uncertainty evaluation. Further, we have developed an entropy-based method that quantifies the relative contribution of the data-driven component and the conceptual constraint in hybrid models, supported by a diagnostic routine that helps exploring how much physics is actually left in hybrids after training. Our findings highlight the urgent need to better understand the internal functioning of hybrids. The goal of this project extension is therefore to focus explicitly on diagnosing hydrological consistency of hybrid models through mapping between model components and physical processes, and a comparison of information flows, enabled by an efficient estimate of transfer entropy. Ultimately, we will bring together three axes that are pivotal for model evaluation: performance, interpretability, and process consistency. Equipped with universal anchors for benchmarking, the proposed model evaluation space "UNI-BENCH" shall help ensure that hybrid models perform well for the right reasons, such that we can confidently promote their use in science and practice. While demonstrated on a case study of rainfall-runoff modelling, we expect UNI-BENCH to be valuable for a wide range of disciplines and societal challenges that rely on trust in dynamical models.

• DFG Programme: Research Grants
• International Connection: Austria, USA
• Cooperation Partners: Professor Dr. Hoshin Vijai Gupta; Professor Dr. Daniel Klotz