Weather Radar Data Exchange Development

Following the establishment and work of the CBS Task Team on Weather Radar Data Exchange (TT-WRDE) and its first session, the Inter-Programme Expert Team on Operational Weather Radars (IPET-OWR) and it successor Joint Expert Team on Opetational Weather Radars (JET-OWR), as a component of its work program, has been advancing the activity started on developing a data model and standardized format for international exchange of weather radar data.

The data model for WRDE has now been fully developed by the IPET and will be documented within the developing Guide to Operational Weather Radar Best Practices within Volume VII. The current draft of Vol VII is available here. A brief outline of the WRDE model is provided below.

The next steps to be taken by the WMO expert team on operational weather radars will be:

Finalise the provisions and practices for maintenance and operation of the WRDE within the WMO Manual on Codes.
Promote the international exchange of weather radar data.
Develop and establish more tools and guidance on WRDE and assist WMO members and program managers to implement data exchange.

Scope

This material presents outputs by IPET-OWR related to:

Weather radar data representation covering
- Information model
- Data models
- File format representation using CfRadial 2
Adoption of weather radar data representation as a single global WMO standard
Supporting WMO members in exchanging weather radar data

Information model

This document describes an information model for the representation of weather radar and scanning lidar data, metadata, and products. Representations are defined for the types of data and metadata that are relevant to international exchange of weather radar data, also supporting processing of such data by recipients. Weather radar data in original polar/spherical coordinates, as they are produced by the radar systems, are the data of primary interest. A set of standard representations have been defined for individual pieces of information, like integers, floating point values, and strings. So-called radar “moment” data, also known as “quantities” and “parameters”, are defined to be represented using arrays of different depths, mostly 8 and 16 bits. Each piece of information is defined using a unique numerical Information Model Identifier (IMID).

The organization of the data with this information model defines a volume of acquired operational weather radar data as a collection of two-dimensional “sweeps” of data, meaning a collection of radials or rays of data that scan the atmosphere either horizontally or vertically. While the information model recognizes the radial as the basic rational representation of data, the sweep is the representation starting point identified as relevant for international data exchange. Calibration metadata are defined to enable processing of quantitative outputs.

Figure 1. Object Model Hierarchy. Horizontal sweep-based example shown.

Data model

Given the information model, the next step is to define the digital representations for each of the payload types. Thus, the data model defines strings as being of Unicode type, integers of different sizes (8, 16, 32, and 64-bits) using native endianness, real values (32 and 64-bits) using the IEEE 754-2008 standard, and absolute and relative times, both of which use the ISO 8601 standard time representation. There is a 1:1 correspondence between the information model and the data model through the use of the IMID.

Where the information model defines relevant types of information and their representation type, the data model is specific about the how each type of information is represented and presented to the user. In this regard the data model can be considered as describing the interface between data users and the object model. The information and data models together provide a basis for the design of APIs for working with polar radar and lidar data.

Figure 2. Object Model Hierarchy as refined for efficient storage by the Data Model.

CfRadial 2

The CfRadial 2 format is intended to provide a standard, modern, self-describing way to store radar and lidar data in native radial (polar) coordinates. It is an extension to the Climate and Forecast (CF) Metadata Conventions which provide a standard approach for the storage and description of scientific datasets using NetCDF files.

CfRadial 2 extends the base CF Conventions by adding support for the complex polar geometries typically used by operational weather radar datasets. It also provides for the hierarchical structure of radar data (volume, sweep, dataset) defined by the WMO Information Model for Radial Radar and Lidar Data. CfRadial 2 has reached the stage of maturity where its implementation in software is possible, and in some places either planned or underway.

A large amount of freedom is given to the user of CfRadial 2 regarding the use of metadata and the naming of datasets. Further guidance on the use of CfRadial 2 is provided by Part G of the WMO Best Practices Guide on Operational Weather Radar. Users are strongly encouraged to heed this guidance to ensure compatibility of files intended for data exchange.

Work is underway to have CfRadial 2 potentially annexed to the WMO Manual on Codes. This is being conducted with the WMO Inter-Programme Expert Team on Data Representation (IPET-DD), and together with the NetCDF and CF communities to achieve durable and sustainable outcomes.

WMO member guidance

Where the information model defines the relevant information and how to organize it, the data model defines how to represent it digitally, and the CfRadial 2 format defines how to represent it in digital files, Part G of the WMO Operational Weather Radar Best Practices Guide (OWR-BPG) provides WMO’s members with guidance on how to apply all of this for the purposes of international exchange of weather radar data. The end result is intended to be a single global standard representation that can serve the emerging global resource that is weather radar.

Experience has shown that defining a well-working standard for the representation of weather radar data greatly facilitates its international exchange. However, this could be even further facilitated though explicit guidance on how such a standard should be applied by WMO’s members. With this in mind, Part G of the OWR-BPG guides members on which data and metadata are mandatory for exchange purposes between or among members, yielding harmonized weather radar data representation independently of origin.

Recommendations on weather radar data exchange

The approach outlined above recognizes that weather radar datasets in native polar coordinates are becoming increasingly important to international users and their application requirements. The ability for these datasets and their metadata to be exchanged internationally in a way that facilitates downstream processing has been vitally important to how IPET-OWR has designed the data representation. We also recognize the importance of being able to continue exchanging derived weather radar products, such as radar reflectivity or precipitation intensity fields on regular Cartesian grids.

Data exchange comprises the combination of information and data models, a file format (in our case CfRadial 2), a so-called “envelope” that is an extra header announcing/describing the payload, and a protocol that is a mechanism for communicating over a network. There is a rich and ongoing legacy of combining different methods and technologies to exchange weather radar data and products. With the WMO push towards operational implementation of WIGOS and WIS solutions, it is becoming increasingly important that exchange mechanisms be harmonized. IPET-OWR highlights the following recommendations:

FTP should be avoided due to its security vulnerabilities. This is particularly sensitive, and potentially impactful, to Members who experience challenges with capacity related to IT expertise. While it may not be reasonable to expect current data exchange solutions using FTP to be replaced in the short term, future WIS mechanisms for real-time data exchange should not be built on top of FTP.
WMO data exchange mechanisms should be designed to accommodate weather-radar data payloads using the data representations outlined above.
Members wishing to exchange weather-radar data should use WMO data exchange mechanisms that are designed to accommodate weather-radar data payloads.
Private (secure) networks can be used as an alternative to the Internet for additional safety and stability. Such networks, e.g. the Regional Meteorological Data Communication Network (RMDCN) in Europe, are private TCP/IP networks with regulated bandwidth designed for peer-to-peer data transfer. The GTS is used by many European Members over the RMDCN.