Verification is the key to providing reliable information for improving weather forecasts. As part of ESRL's involvement with the Federal Aviation Administration (FAA) Aviation Weather Research Program (AWRP), the Forecast Verification Section develops verification techniques, mainly focusing on aviation weather forecasts and tools that allow forecasters, researchers, developers, and program leaders to generate and display statistical information in near real time using the Real Time Verification System (RTVS).

In adhering to related goals in the ESRL Strategic Plan, the section strives to maintain a strong verification program by working closely with other agencies, such as the National Centers for Environmental Prediction (NCEP), National Weather Service (NWS), and the National Center for Atmospheric Research (NCAR) Research Applications Laboratory. The technology developed through these close interactions benefit all agencies by building and strengthening the verification programs.

The Real Time Verification System (RTVS)

In support of ESRL's verification efforts, a Real-Time Verification System (RTVS) is being developed as a tool for assessing the quality of weather forecasts. RTVS is designed to provide a statistical baseline for weather forecasts and model-based guidance products, support real-time forecast operations, model-based algorithm development, and case study assessments. To this end, RTVS was designed to ingest weather forecasts and observations in near real time and store the relevant information in a relational database management system (RDBMS). A flexible easy-to-use Web-based graphical user interface allows quick and easy access to the data stored in the RDBMS. Users can compare various forecast lengths and issue times, over a user-defined time period and geographical area, for a variety of forecast models and algorithms.

The RTVS has become an integral part of the AWRP by providing a mechanism for monitoring and tracking improvements in AWRP-sponsored forecast products. RTVS provides displays and statistical information to forecasters and managers at the Aviation Weather Center (AWC), Air Traffic Systems Control Command Center (ATCSCC), and to the developers of the AWRP Weather Research Product Development Teams. The RTVS will run operationally at the NWS providing feedback directly to forecasters and managers in near real time.

Verification Methods

The section is an active participant, in collaboration with NCAR, in developing and testing state-of-the-art verification methods, with an emphasis mainly on aviation and precipitation forecast problems. New techniques have been developed for convection, icing, turbulence, ceiling and visibility, cloud top heights, and precipitation. Many of these techniques are applied to aviation forecasts that have been deemed "unverifiable." Nevertheless, the development and implementation of these verification methods are leading to a better understanding and improvement in aviation forecast quality.

• Completed verification activities supporting the transition of Current Icing Potential for Alaska (CIP-AK), CIP severity index, and NCWF-2 from test phase to experimental phase. The verification reports for these products are available in the Publications section.


• In support of the Coastal Storms Initiative (CSI) project, RTVS was modified to include verification of temperature, relative humidity, and wind forecasts. This verification tool can be viewed at http://www-ad.fsl.noaa.gov/fvb/rtvs/csi/.


• A new verification methodology for defining a convective impact field used for evaluating convective forecasts was developed (Mahoney et al. 2004). This new field, termed the Convective Constrained Area (CCA), accounts for the distance aircraft must deviate from convection.


• RTVS continued to be developed over the past year. The relational database management system (RDBMS) was extended to all aspects of RTVS and the Web-based graphical user interface was also modified to allow easy and flexible access to the data stored in the RDBMS. Access to RTVS can be obtained through the web at http://www-ad.fsl.noaa.gov/fvb/rtvs/.


• Three real-time objective intercomparison exercises, including turbulence, convection, and icing were executed in support of the work of the AWRP Product Development Teams.


• In support of Weather Research and Forecasting (WRF) model development, the Forecast Verification Section and NCEP collaborated to develop a web-based verification system to analyze model results in comparison with point observations for select periods. Results of these eight model runs over a four-month period can be viewed at http://www-ad.fsl.noaa.gov/fvb/rtvs/wrf/retro_runs/.

The Forecast Verification Section anticipates the following future projects:

• Continuation of real-time objective intercomparison exercises for turbulence, icing, convection, and ceiling and visibility in support of the AWRP.


• Develop new verification capabilities for NWS aviation forecasts.


• Provide ongoing support for FAA and NWS activities, such as providing input into the development of a national verification program, transition of RTVS to the NWS, and support for the development of the WRF numerical modeling effort.


• Explore verification techniques that address questions pertaining to flight operations.


• Enhanced RTVS to include advanced diagnostic verification approaches that will provide users with the ability to partition errors that are associated with the phase, orientation, and displacement of the forecasts as compared to the observations.


• Perform extensive statistical evaluations of the Forecast Icing Potential for Alaska (FIP-AK) algorithm and the Cloud Top Height and provided written reports to the FAA/NWS Aviation Weather Technology Transfer Board.


• Continue to develop verification tools, through RTVS, that provide immediate and useful feedback to forecasters.