2007 Convective Forecast Scientific Evaluation

Abstract

The goal of this scientific evaluation is to assess the performance of several convective forecasts with respect to their application to operational air traffic flow planning. Five forecasts, including Collaborative Convective Forecast Product (CCFP) Preliminary and Final, Rapid Update Cycle (RUC) Convective Probability Forecast (RCPF), RUC Reflectivity, and the North American Mesoscale model (NAM) Reflectivity were evaluated using the National Convective Weather Detection (NCWD) product from 11 June – 31 August 2007. This Executive Summary highlights the main aspects of the scientific evaluation, but further detail and analyses are summarized in a comprehensive report. The main verification approach applied in the study was used to intercompare the forecast quality of the five products at strategic flight planning time periods and within impacted sectors. This unique measure of forecast quality was linked directly to the application of the convective forecasts to the operational flight planning process. Relevant results from the study indicated: • Nearly identical forecast performance from the CCFP Preliminary and the CCFP Final • The CCFP and RCPF performed similarly for nearly all time periods, except at the 2-h outlook period where CCFP performed slightly better. • At early valid times and for shorter outlook periods, CCFP performed slightly better than RCPF. • At later valid times and longer output periods (i.e., when convective weather has the potential to severely impact air traffic), the RCPF performed as well as the CCFP. • Analysis of the top ten high-impact air traffic days indicated that the performance of the RCPF at the 8-h outlook period for the afternoon shows some promise for planning purposes. • On high-coverage, high-impact days neither CCFP nor RCPF performed significantly different from the other. • The CCFP, for every valid time of interest, better identified the sectors that were impacted by convection than did the RCPF. • The reflectivity products (NAM and RUC) showed virtually no skill at forecasting hazardous convection, but did provide some guidance at long lead periods for areas of concern. • The probabilistic aspects of CCFP and the RCPF indicated low reliability. Recommendations that were identified from the results include: • Meteorologically, the forecast skill of the CCFP Final and Preliminary perform similarly, thus we recommend that the meteorological collaboration and its relationship to the planning process be further evaluated before possible elimination. • In the near term, RCPF should be used as input to the CCFP generation process. • In the longer term, we recommend adoption of a gridded probabilistic forecast as meteorological input to the traffic planning process. • Since the radar reflectivity products have in some cases alerted planners to course areas of hazardous weather 8-24 h in future, we recommend further resources be devoted to the development of these products to improve their ability to better forecast convective intensity and structure. This may also benefit other automated convective forecasts.