Workshop: Verification of AI models in Operational Centers
AI-based weather prediction has advanced rapidly over the past decade, with AI systems increasingly being adopted and integrated with physically based Numerical Weather Prediction (NWP) systems in operational environments. Robust, fair, and transparent verification methods are essential to understand the characteristics and capabilities of these new systems in comparison to traditional NWP, to guide their effective use, and to pave the way toward optimal hybrid approaches to weather forecasting.
This workshop brings together international scientists and researchers from the WWRP Joint Working Group on Forecast Verification Research (JWGFVR), from the WCRP Working Group on Numerical Experiment (WGNE), and from the Meteorological Research Division of Environment and Climate Change Canada (ECCC) to address emerging challenges in the operational verification of AI models alongside traditional NWP systems. We aim to examine how established verification practices can be adapted to account for the distinct characteristics of AI-based forecasts, explore metrics and novel diagnostics for meaningful comparisons, and promote best practices to ensure that WMO Members and forecast users can trust and interpret outputs from both paradigms with confidence. Building on discussions from the two preceding (2024, 2025) VerAI workshops hosted at the University of Reading (UK), this event also aims to establish a baseline for verification activities within the WGNE Weather Prediction Model Intercomparison Projet (WP-MIP), which in the future could feed into recommendations for the WIPPS standard verification exchange.
Contributions and Registration
Contributions on verification approaches for comparing AI, hybrid and traditional NWP models are welcome. Topics of interest include (but are not limited to) the following verification research topics:
- Observation uncertainty and incestuousness of verification against own analysis
- Spatial methods: neighborhood, filtering and scale-separation approaches; field morphing and displacement methods
- Process diagnostics and physical coherence assessment
- Verification of extreme and rare events
Contributions and registration deadline is the 26th of September. To register and/or to propose a contribution, please write an email to barbara.casati@ec.gc.ca. Please note that solely registered participants will receive the link for online attendance.
For more information: https://hpfx.collab.science.gc.ca/~bca851/VerAIops2025/index.html