World Meteorological Organization

Past Intercomparisons

WMO Intercomparison of Pyrheliometer
Davos, Switzerland (September - October 2010)
Final Report  (2011)

The 11th international Pyheliometer Comparison (IPC-XI) was held together with Regional Pyrheliometer Comparisons (RPCs) of all WMO Regional Associations (RA | to RA IV) from 27 September through 15 October 2010 at the Physikalisch-Meteorologisches Observatorium Davos/World Radiation Centre (PMOD/WRC) in Davos, Switzerland.

The results presented in this report are based on the measurements carried out during the three weeks assigned to the IPC-XI. The favorable weather conditions allowed to acquire a large number of calibration points for most participating instruments. Cloudy and overcast days were used for technical preparations and training of participants as well as for a the IPC-XI symposium and Course on Radiation Measurement. A Saharan Dust Event (SDE) affected the measurements during several days starting October 8th. Analyzing the effect of the SDE on different types of instruments led to interesting findings which are summaries dedicated section of this report.

WMO Intercomparison of High Quality  Radiosonde Systems
Yangjiang (July - August 2010)
Final Report  (2011)

The 8th WMO Intercomparison of High Quality Radiosonde Systems held in China in July
2010 has produced a large data set on the performance of new operational radiosonde
designs, backed up by measurements from Scientific Sounding Instrumentation. This was the result of China supporting this test very actively, and very good collaboration with the
various manufacturers.

This allows recommendations as to the radiosonde designs that are potentially suitable for the GRUAN network operations and those which are suitable for routine operations, together
with recommendations to improve systems without excessive development expenditure.

At night, most radiosonde systems can provide temperature measurements of suitable quality for both weather and climate work. In the day, many designs need improvement to the sensor exposure to improve the reproducibility of measurements near 10 hPa, and some further testing are needed to reduce the systematic bias between the various radiosondes at these upper levels. With more documentation, several systems have potential for use in GRUAN operations.

The relative humidity sensors tested in Yangjiang had good reproducibility, but several types had large systematic errors at all heights in the troposphere, and the origin of these needs to be identified and rectified as soon as possible. Several systems showed potential for observing relative humidity in the upper troposphere in the tropics, and the new correction schemes seem to have good potential for future observations. It was possible to check the measurements in cloud, using cloud radar (up to 15 km) and ceilometer observations (up to 12 km) to identify some of the clouds. Thus, with more documentation several systems have potential for use in GRUAN operations.

GPS height measurements are very reproducible, and are clearly suitable for all radiosonde
operations, given that the equipment is initiated correctly. On the other hand, cheaper
radiosondes may be used by exploiting systems such as the Chinese secondary radar for good quality operational results.

Thus, where GPS radiosondes are set up correctly, pressure can be deduced from the
geopotential heights and the radiosonde measurements of temperature and relative humidity
profile.

As a result of the test, many errors in the various quality radiosonde systems were identified and subsequently rectified ensuring improved accuracy for future radiosondes.

The Radiosonde Comparison was performed as a first collaboration between CIMO and a
range of climate scientists associated with GCOS and GRUAN, as part of a WIGOS Pilot
Project.

WMO Field Intercomparison of Thermometer Screens/Shields and Humidity Measuring Instruments
Ghardaïa (November 2008 -  October 2009)
Final Report  (2011)

The WMO Field intercomparison of thermometer screens and humidity measuring
instruments was held from the 1st of November 2008 to the 31st of October 2009, at the
meteorological station of Ghardaïa, Algeria.

The need of a combined intercomparison of thermometer screens/shields and humidity
measuring instruments in hot desert conditions was identified in 2003. The site of
Ghardaïa, Algeria, was proposed by the Algerian National Weather Service (ONM) and
accepted by the ET & IOC in 2006.

This intercomparison hosted:

          − 18 different types of screens/shields both ventilated (7) and non-ventilated (11),
most of them installed in pairs (the total number being 29);
          − 2 wind sensors from the manufacturer Thies (Germany) for evaluating ultrasonic
temperature measurement (proposed by DWD);
          − 8 different types of humidity sensors, most of them installed in pairs (the total
number being 17)

Météo-France supplied calibrated Pt100 probes for most of the screens. All humidity
sensors were delivered to Trappes for calibration in agreement with the manufacturers. An
on-site calibration was also performed for a subset of the hygrometers.

The ONM prepared the experimental field and installed 36 platforms for the selected
screens/shields and the ancillary sensors (radiation sensors, 2-meter wind, ground
temperature…).

All data were filtered with quality control procedures. Over the 12 months period of the
intercomparison, more than 500 000 minutes of data are available for the majority of the
screens and hygrometers, allowing a deep data analysis.

Generally the intercomparison was successful. It experienced problems in its schedule,
due to customs constraints and electrical grounding problems at the beginning.

WMO Field Intercomparison of Rainfall Intensity Gauges
Vigna di Valle (October 2007 - April 2009)
Final Report  (2009)

The WMO Field Intercomparison of Rainfall Intensity (RI) Gauges was conducted from
1 October 2007 to 30 April 2009, in the Centre of Meteorological Experimentations (ReSMA) of the Italian Meteorological Service, in Vigna di Valle, Italy. It was organized following the request of users and the recommendation of CIMO-XIV.

Heavy rainfall is generally the origin of flash floods. In view of the very high variability of the rainfall intensity, measurements at a 1-minute time scale are crucial to enable proper measures be taken to mitigate the impact of such events and save lives, property and infrastructures. As the return period of heavy rainfall events is large, long-term records of rainfall intensity data are needed to estimate the probability of occurrence of heavy rainfall at a given location and time. Such measurements would also be used for better design of structures (building, construction works) and infrastructure (drainage) to mitigate severe weather impact.

This intercomparison hosted 25 different rainfall intensity gauges. The majority of these instruments were catching type gauges comprising tipping-bucket gauges, weighing gauges and one water level gauge. Non-catching rain gauges were represented by optical and impact disdrometers, one optical/capacitive gauge and one microwave radar gauge. This intercomparison was unique as to the number of instruments and variability of techniques used.

The main objective of this intercomparison was to intercompare the performance of in-situ
rainfall intensity instruments of different measuring principles, with special consideration given to high rainfall intensities. Further objectives were to offer advice on improvements of instruments and precipitation measurements.

Prior to installation in the field all reference gauges and the catching type instruments were calibrated in the WMO recognized laboratory at the University of Genoa. Calibration procedures were based on recommendations of the previous WMO Laboratory Intercomparison of RI Gauges (2004-2005) which were further developed to allow an assessment of the 1-minute measurement uncertainty under constant flow conditions. Additionally all catching gauges were subject to a step response test to assess their dynamic response.

In the field, all gauges were compared with a RI composite working reference consisting of
a set of three reference rain gauges in a standard pit.

The results of the intercomparison confirmed the feasibility to measure and compare rainfall
intensities on a one minute time scale and provided information on the achievable measurement uncertainties. Due to the very high variability of rainfall intensity, the time synchronization of the instruments is crucial to compare their measurements and to design the measurement systems, as two successive 1-minute rainfall intensity measurements can differ much more than the measurements of two well synchronized instruments.

The uncertainty of the RI composite working reference in the pit was evaluated to be 4.3
mm/h, leading to a relative uncertainty below 5% above 90 mm/h and higher than the 5%
measurement uncertainty required by WMO below 90 mm/h.

It is recommended that rainfall intensity measurements be further standardized at an
international level and based on knowledge obtained from this intercomparison to allow the users to obtain homogeneous and compatible data sets. The procedure adopted for performing calibration tests in the laboratory should become a standard method to be used for assessing the instruments’ performance. Acceptance tests could be based on the adopted laboratory procedures and standards. A classification of instrument performance should also be developed to help users in selecting the proper instrument for their applications.

The intercomparison results confirmed that uncorrected tipping bucket rain gauges should
be corrected. Very good results can be achieved by software correction methods. Catching gauges that do not use a funnel are sensitive to environmental factors, affecting the measurements of some instruments. It was found that proper techniques can be used to reduce the noise in the measurements.

It was found that manufacturers need to improve their documentation, and users should
contact manufacturers for additional guidance, as how to best operate their instruments for various applications.

The Intercomparison high quality data set (1-min rainfall intensity data) constitutes an
important scientific resource that should be further exploited beyond the objectives of the present data analysis.

It is also recommended that the developed expertise and the infrastructure of the sites, both the field and the laboratory facilities, be further exploited within WMO.

The WMO Laboratory Intercomparison of Rainfall Intensity (RI) Gauges was held from September 2004 to September 2005 in the laboratories of the Royal Netherlands Meteorological Institute (The Netherlands), Météo-France (France) and the Department of Environmental Engineering of University of Genoa (Italy) in collaboration with the Italian Meteorological Service.

No intercomparisons of instruments for the measurement of RI had been organized so far. Following the recommendations of the Expert Meeting on Rainfall Intensity Measurements, Bratislava, Slovakia, April 2001, it was proposed, as the first and necessary step, to organize an intercomparison of RI instruments in the laboratory conditions. Some laboratory tests of rain gauges were done and reported in the literature, however no intercomparison of RI instruments, in one or several laboratories, had been conducted.

The main objective of the Intercomparison was to test the performance of catchment type rainfall intensity gauges of different measuring principles under documented conditions. Other objectives were to define a standardized procedure for laboratory calibration of catchment type rain gauges, and to provide information relevant to improving the homogeneity of rainfall time series with special consideration given to high rainfall intensities. Finally, a comment on the need to proceed with a field intercomparison of catchment type of RI gauges was required as well as to identify and recommend the most suitable method and equipment for reference purposes within the field intercomparison of catching and non-catching types of gauges.

The CIMO Project Team consisted of the Chair of the Expert Team (ET) and the International Organizing Committee (IOC) on Surface-Based Instrument Intercomparisons, the Project Leader and three Site Managers, coordinated the work of the laboratories involved in the intercomparison. The 19 pairs of participating instruments from 18 manufacturers were divided into three groups, with each group being tested for a period of about three to six months in each of the laboratories, in order to obtain a high degree of confidence in the results. The first phase of tests had successfully concluded by 15 February, the second by 15 May 2005 and the third by September 2005. All the cost related to laboratory intercomparisons was born by the laboratories and the manufacturers involved.

The majority of the participating instruments were tipping-bucket gauges, which are the most widely used in operational networks. Second group of instruments were weighing gauges; the third group consisted of two participating instruments only using a non-common measuring principle, namely, a water level based on conductivity measure.

A general methodology was adopted for the tests, based on the generation of a constant water flow from a suitable hydraulic device within the range of operational use declared by the instrument’s manufacturer. The water was conveyed to the funnel of the instrument under test in order to simulate constant rainwater intensity. The flow was measured by weighing the water over a given period of time. The relative difference between the measured and generated rainfall intensity was assumed as the relative error of the instrument for the given reference flow rate. In addition to measurements based on constant flow rates, the step response of each instrument was checked based on the suitable devices developed by each laboratory.

Each laboratory developed its own testing device, with some differences in the principle and technology used to generate a constant water flow, as well as in the way the water is weighed in the device. These provided a basis for the development of a standardized procedure for generating consistent and repeatable precipitation flow rates for possible adoption as a laboratory standard for calibration of catchment type RI gauges.

The results of the Intercomparison showed that the tipping-bucket rain gauges that were equipped with proper correction software provided good quality rainfall intensity measurements. The gauges where no correction was applied had larger errors. In some cases problems of water storage in the funnel occurred that could limit the usable range for rain intensity measurement.

The uncertainty of the rainfall intensity is generally less for weighing gauges than for the tipping-bucket rain gauges under constant flow rate condition, provided the instrument is properly stabilized. The measurement of rainfall intensity is affected by the response time of the acquisition system. Significant delays were observed in “sensing” time variation of the RI by weighing gauges. The delay is the result of the internal software which is intended to filter the noise. Only one instrument had a delay that met the WMO 1-minute rainfall intensity requirement.

The two gauges using a conductivity measurement for determining water level showed good performances in terms of uncertainty under controlled laboratory conditions. Siphoning problems for one gauge limits its ability to measure a wide range of rainfall intensity. For the other one, a limitation is related to the emptying mechanism, in which case 2-minute delay was observed. These gauges are potentially sensitive to the water conductivity, with no demonstrated problems during the laboratory tests.

The laboratory tests were performed under controlled conditions and constant flow rates (rainfall intensities) so as to determine the intrinsic counting errors. It must be considered that RI is highly variable in time, thus catching errors may have a strong influence on the overall uncertainty of the measurement. The need to combine the assessment of both counting and catching errors for the instrument analyzed in the laboratory is paramount. Provided the instrument is properly installed in the field, according to the WMO specifications, the question to be answered is what kind of instrument (measuring principle, manufacturer, model) is the most suited to the specific requirements of the user. This question cannot be answered based on the laboratory Intercomparison alone, although the results obtained can provide preliminary information to manufacturers and the first-step selection criterion for the user.

Therefore, it is necessary to proceed with the quality assessment procedure initiated in the laboratory by organizing a follow-up Intercomparison in the field where the instruments tested in the laboratory should have a priority. This would allow continuity in the performance assessment procedure and result in the estimation of the overall operational error to be expected in the measurement of RI in the field. Also other instruments would be included in the field intercomparison, even if not tested in the laboratory phase, with a priority given to the non-catching type of instruments.

For the Field Intercomparison a working reference rain gauge(s) should be inserted in a pit according to the EN-13798 standard Reference Raingauge Pit, adopted by ISO, in order to minimize the effect of weather related errors on the measured rainfall intensities. According to the results of the laboratory intercomparison, it is recommended to select the best performing dynamically corrected tipping bucket rain gauge(s) and the weighing gauge(s) showing the shortest step response and the lowest uncertainty as reference gauges. The combined analysis of the reference gauges allows the best possible estimation of the rainfall intensity in the field, given their demonstrated performance in the laboratory. The use of one reference instrument alone is not recommended.

Finally, the improvement of the uncertainty of RI gauges introduces the risk of affecting the homogeneity of rainfall time series. The improvement of the measurement of RI may produce a discontinuity of the historical rainfall intensities records, which could influence especially the studies of extreme events. The bias introduced by non-corrected records propagates through any rainfall-runoff model down to the statistics of flow rates in water courses, with non negligible effects on the study of floods and flash floods. An example of the correction of the historical rainfall series was demonstrated using the result of the laboratory intercomparison.

Sub-Regional Pyranometer Intercomparison of the RA VI members from South-Eastern Europe
Split , Croatia (July - August 2007)
Final Report  (2009)

With the great support of WMO Secretariat and World Radiation Centre, MHSC has
conducted the first international pyranometer intercomparison in the south-eastern part of Europe.

Favourable, sunny weather enabled acquiring a large amount of data for a reasonable
evaluation of the pyranometers participated in the Intercomparison.

Reference value obtained by three recently calibrated pyranometers represented a reliable
basis for further calculations.

Large number of data used for statistics assured reasonable confidence in the calculated
calibration factors. Available auxiliary data could be linked to the calibration factors and considered during future comparisons of the calibration factors for similar conditions.
Generally, slight underestimation of the global solar radiation by the participating
instruments could be addressed also to the degradation of their sensitivity (some of them haven’t been calibrated for more than 15 years) or to different type of sensors. Further detailed analyses with the use of available auxiliary data and comparison with previous calibrations and corresponding condition could enable better understanding of calibration factors behaviour.

A larger standard deviation of few pyranometers was a consequence of the larger
dispersion of the measurements.

Although there were some unexpected organisational and infrastructural irregularities,
generally it is believed that the Intercomparison could provide some useful experiences and results for the participants, as well as the community involved in this issue from the sub-region, especially in the field of improving accuracy and assuring traceability of the global solar radiation measurement.

top of this page

WMO Intercomparison of Pyrheliometer
Davos, Switzerland (September-October 2005)
Final Report  (January 2006)

Under the auspices of the Commission for Instruments and Methods of Observation (CIMO), the Tenth International Pyrheliometer Comparison (IPC-X) was held together with the Regional Pyrheliometer Comparisons of all WMO Regions from 26 September through 14 October 2005 at the Physikalisch-Meteorologisches Observatorium Davos/World Radiation Centre (PMOD/WRC) in Davos, Switzerland.

The results presented in this report are based on the measurements carried out during the  three weeks assigned to the IPC-X. The favorable weather conditions allowed to acquire a record number of calibration points for most participating instruments. Cloudy and overcast days were used for technical preparations and training of participates as well as for a the IPC-X symposium.

The WMO Intercomparison of High Quality Radiosonde Systems, held in Vacoas, Mauritius, 2-25 February 2005, was organized because a new generation of radiosondes is being introduced into most of the global upper air network. Six operational radiosonde systems (Vaisala, Sippican, Modem, MEISEI Electric Co., Graw Radiosondes and Meteolabor) participated in the intercomparison, which consisted of 62 successful comparison flights. In addition Sippican MKII, 3 thermistor radiosondes were flown to provide a daytime “working reference” for temperature and the Snow-white chilled mirror hygrometer as a “working reference” for dewpoint/relative humidity. The working references were flown within the intercomparison to provide additional evidence on the accuracy of the operational radiosondes. The intercomparison was intended to identify any significant flaws in the new radiosonde designs, so that these could be rectified before use became widespread in the operational radiosonde networks. The following is the summary of results.

Measurements of wind by the GPS radiosonde systems were of good availability and quality. Preparing the GPS radiosondes for flight is now much easier than in the intercomparison in Brazil in 2001. The GPS heights measured by the GPS radiosondes were so accurate that in most situations there is no longer any need to use a pressure sensor on a GPS radiosonde. The Intercomparison in Mauritius demonstrated that errors identified in the WMO Intercomparison of GPS Radiosondes in Brazil have mostly been rectified. Temperature, pressure and relative humidity measurements by the radiosondes agreed more closely than in any of the earlier WMO Radiosonde Intercomparisons.

Thus, all radiosondes in this Intercomparison were judged to merit the designation of high quality radiosonde and were of better quality than in the previous WMO Radiosonde Intercomparison in Brazil. Whilst much progress has followed from major investments by the main manufacturer, the smaller manufacturers have also contributed with significant innovations generating a more competitive environment.

Some problems remain in most systems tested. When rectified these will further improve radiosonde measurement quality (including long term stability of measurement quality). This should produce a stability in radiosonde measurements that has not been present in earlier generations of radiosondes. 

Manufacturers need to consider from the evidence in the report whether their radiosonde measurements need:

• ·     Hydrophobic coatings on temperature sensors;

• ·     Improved sensor exposure for daytime temperature measurements in the stratosphere;

• ·     Prevention of chemical contamination of relative humidity sensors during storage;

• ·     Procedures to minimize the effects of water contamination on humidity sensors after emerging from cloud;

• ·     Procedures to minimize the effects of ice contamination on humidity sensors after emerging from cloud at all  temperatures in the middle and upper troposphere;

• ·     Improved procedures to eliminate low bias in daytime relative humidity caused by the humidity sensor observing at a higher temperature than the reported temperature (software correction or direct measurement of humidity sensor temperature?).

The working reference systems proved very useful in interpreting results from the Intercomparison, but were not developed to the stage that they could be used as stand alone references in the type of conditions experienced in Mauritius.

Recommendations on suitable radiosondes for future climate observing networks are presented. Most of the radiosondes tested in Mauritius could be brought up to a standard suitable for this work. Thus, it is suggested that the best traceable upper air measurement record might be obtained by successive measurements by two of the best operational radiosonde types at one observing site. The use of two different types launched with relatively small time separations would allow possible changes in the measurement quality of either radiosonde type to be identified. For temperature measurements there is a large range of suitable sensors. For water vapour/ relative humidity the range of sensor type is much more limited and further development of new sensors could be beneficial.

The two radiosonde systems currently closest to the standards required for climate measurements to the highest levels in the tropics (pressures as low as 5 hPa) were Vaisala RS92-SGP and Sippican LMS-5. However, it is expected that several of the other systems will be close to this standard shortly, and could possibly offer cheaper consumable costs in the long term.

The use of GPS radiosondes without a pressure sensor, tested in 2001 for MODEM and Sippican in the WMO GPS Radiosonde Comparison in Brazil, and also used by Meisei in Mauritius, was found to offer a reliable method of reducing consumable costs.

The system developed by Graw demonstrates how GPS radiosonde systems can be developed to make setting up and operations extremely easy for unskilled staff.

The SRS digital radiosonde system proved a very reliable method of flying the Snow White chilled mirror hygrometer, with much less operational trouble than was experienced with the Sippican/Snow White combination in Brazil intercomparison.

An extended list of recommendations and conclusions can be found within the relevant sections of the report.

This Intercomparison was performed in Mauritius under the management of a small WMO Project Team. The team was responsible for the conduct of the test and for training of local staff from Mauritius Meteorological Services. The Intercomparison demonstrated that the local staff was experienced enough and could be trained quickly in more advanced test procedures. This resulted in an improved morale and knowledge within the Mauritius Meteorological Services. The deployment of a WMO Project Team to supervise the test was successful, however, for future tests it should be recognized that the supervisory work involves a lot of real time consultation with the participants and is much more than purely managing the test procedures.  This consultation period extended for several months after the completion of the test and was not just limited to the time during the intercomparison.

At the invitation of INMET/Brazil, between 20 May and 10 June 2001, under tropical weather conditions, took place the WMO Intercomparison of GPS Radiosondes, at the Alcantara Air Force Satellite/Rocket Launch Centre. The trials were carried out in accordance with the recommendations developed by the International Organizing Committee (IOC) (Final Report). During the project, led by Dr. Reinaldo da Silveira (INMET), the performance of the main types of GPS-radiosonde currently used for operational measurements was tested in over 40 comparison flights. The GPS-radiosonde involved were: Vaisala RS80-15G and Vaisala RS90-AG (Finland); Sippican MKII (USA); Geolink GPSonde GL98 (France); and Dr Graw Messgeräte DFM-97(Germany).

All the GPS wind measurement systems produced high quality wind data (with errors much less than 1 ms-1) when working correctly. Some systems had more malfunctions during flight than others. Another objective was to obtain more detailed information on the performance of humidity sensors, so it was highly appreciated that "Snow-White" chilled mirror hygrometers (Meteolabor/Switzerland) were coupled to the Sippican radiosondes on 20 flights. Preliminary results of humidity comparisons revealed that significant differences between the two commonly used radiosonde relative humidity sensors have still not been resolved. On most flights relative humidity varied a great deal with height, with many ascents passing through upper cloud. The difference in sensitivity and absolute calibration in upper cloud between the older humidity sensors and some of the newer designs can readily be quantified. Results for both wind and relative humidity will be significant in improving the operational reliability of radiosonde designs. Following the completion of the data analysis, the Executive Summary Report has been published under IOM No. 76, TD-1153.

The facilities and support provided by the Brazilian Air Force at Alcantara to participants from Brazil, France, Finland, Germany, UK and USA were excellent and underpinned the success of the experiment.

WMO Ninth International Pyrheliometer Comparison

(IPC-IX)Davos, Switzerland (September-October 2000)

The Ninth International Pyrheliometer Comparison (IPC-IX), conjointly held with Regional Pyrheliometer Comparisons (RPCs), was carried out at the World Radiation Center (WRC/PMOD) Davos, Switzerland, from 25 September to 13 October 2000, led by Dr Isabelle Ruedi. In total 65 radiation experts from 39 WMO Member countries participated and 85 pyrheliometers were calibrated successfully despite unfavourable weather conditions. It can be noted with appreciation that representatives from 18 out of the 21 available Regional Radiation Centres could arrange for their participation. Participants particularly appreciated the opportunity to attend the several symposia and workshops carried out during the IPC-IX. The lectures provided and the discussions held significantly contributed to the transfer of knowledge to participants, especially to those from developing countries. The Final Report of the IPC-IX has already been published and distributed to the participants and to al NMHSs. It can be accessed through the Web-Site of the WRC Davos.

The technical facilities available, the excellent arrangements made and the support provided by Dr W. Schmutz, Director of the WRC, and his staff were essential to the success of the IPC-IX and were highly appreciated by all participants.

top of this page