Satellite Data Formats and Standards

This page explains the different formats used for dissemination and processing of Satellite Data.

HRIT/LRIT

High Rate Information Transmission (HRIT) and Low Rate Information Transmission (LRIT) are the CGMS standards agreed upon by satellite operators for the dissemination of digital data originating from geostationary satellites to users via direct broadcast. The distinction between the two standards, as their names suggest, is the data rate (bandwidth) necessary to convey the data content. LRIT data are typically disseminated at speeds up to around 256 Kbps while HRIT data are typically disseminated at speeds up to 10 Mbps. Commonly the content of LRIT data streams are subsets of the equivalent HRIT data, with subsampling and lossy compression applied.

Global format specifications for these digital data transmission standards, based on the application and presentation layers of the OSI reference model are described in HRIT/LRIT Global Format Specifications. Individual satellite operators have discretion to make mission-specific implementations of the lower layers of the OSI reference (to implement such elements as data compression methods, data encryption, etc) and these are described for EUMETSAT's Meteosat Second Generation (MSG) and for JMA's Multifunction Transport Satellite (MTSAT) for JMA-HRIT and JMA-LRIT.

It should be noted that the file structure of HRIT/LRIT data are sometimes used to distribute data via other mechanisms, for example in the EUMETCast data dissemination service and as retrieval formats for archived data.

HRPT/LRPT

High Rate Picture Transmission (HRPT) and Low Rate Picture Transmission (LRPT) are the CGMS standards agreed upon by satellite operators for the dissemination of digital data originating from low earth orbit satellites to users via direct broadcast. In a very similar way to HRIT/LRIT, described above, the distinction between the two standards is the data rate (bandwidth) necessary to convey the data content. LRPT data are typically disseminated at speeds less than 150 Kbps while HRPT data are typically disseminated at speeds greater than 0.5 Mbps.

Global format specifications for these digital data transmission standards, based on the application and presentation layers of the OSI reference model are described in HRPT/LRPT Global Format Specifications.

WMO binary data exchange formats - BUFR, GRIB

The WMO Binary Universal Form for the Representation of meteorological data (BUFR) is a binary code designed to represent any meteorological data set employing a continuous binary stream. It has been designed to achieve efficient exchange and storage of meteorological and oceanographic data. It is self defining, table driven and very flexible data representation system, especially for huge volumes of data.

Similarly, another widely used bit-oriented data exchange scheme is the WMO GRIddedBinary (GRIB) format. GRIB is an efficient vehicle for transmitting large volumes of gridded data to automated centers over high-speed telecommunication lines using modern protocols. An updated version of GRIB, commonly abbreviated to GRIB-2, is currently being introduced and is most relevant for use with satellite data.

These two WMO Table Driven Code Forms have been widely adopted for the distribution of meteorological satellite products, especially those processed to level 2 or beyond (see the Imagery and Derived Products section). They are described in the Operational Codes and Manual on Codes. By packing information into the BUFR or GRIB code, data records can be made more compact than character oriented bulletins, resulting in faster computer-to-computer transmissions. The formats can equally well serve as a data storage formats, generating the same efficiencies relative to information storage and retrieval devices.

Software for encoding and decoding data in the BUFR and GRIB formats is freely available for download from the ECMWF web site.

McIDAS

The Man computer Interactive Data Access System (McIDAS) is not simply a satellite data format, it is rather a suite of applications for analyzing and displaying meteorological data for research and education. McIDAS has been in use and under continual development by the University of Wisconsin-Madison Space Science and Engineering Center (SSEC) since 1972. The Unidata McIDAS software (a superset of SSEC McIDAS) has been under development since 1985 and in distribution since 1988. The software can be used with conventional observational, satellite, and grid-point data. Unidata distributes a version of McIDAS (McIDAS-X) for a variety of platforms running Unix, Linux, and MacOS-X.

NetCDF, HDF, XML

These three formats are widely used in the area of exchanging and storing meteorological data. They are emerging industry standards and each offers benefits of effectiveness and efficiency. Over the coming years it is possible that one will assume a primary role with respect to the processing of satellite data.

NetCDF (Network Common Data Form) is a machine-independent, self-describing, binary data format standard for exchanging scientific data. The project homepage is hosted by the Unidata program at the University Corporation for Atmospheric Research (UCAR). They are also the chief source of netCDF software, standards development, updates etc. The format is an open standard. The data format is "self-describing". This means that there is a header which describes the layout of the rest of the file, in particular the data arrays, as well as arbitrary file metadata in the form of name/value attributes. The format is platform independent, with issues such as endianness being addressed in the software libraries. The data arrays are rectangular, not ragged, and stored in a simple and regular fashion that allows efficient subsetting.

Hierarchical Data Format, commonly abbreviated HDF, HDF4, or HDF5 is a library and multi-object file format for the transfer of graphical and numerical data between computers. It is created and maintained by the NCSA. The freely available HDF distribution consists of the library, command-line utilities, test suite source, Java interface, and the Java-based HDF Viewer (HDFView). HDF supports several different data models, including multidimensional arrays, raster images, and tables. Each defines a specific aggregate data type and provides an API for reading, writing, and organizing the data and metadata. New data models can be added by the HDF developers or users. HDF is self-describing, allowing an application to interpret the structure and contents of a file without any outside information. One HDF file can hold a mixture of related objects which can be accessed as a group or as individual objects.

The Extensible Markup Language (XML) is a general-purpose markup language. Its primary purpose is to facilitate the sharing of data across different information systems, particularly via the Internet. More details of XML and its potential role in the exchange of binary data may be found in the web pages of the W3C organisation