Quality Assessment Report National Convective Weather Forecast 2 (NCWF-2)

Abstract

This report summarizes a statistical evaluation of the quality of the National Convective Weather Forecast (NCWF-2) product, developed by the Convective Weather Product Development Team under the Federal Aviation Administration’s (FAA) Aviation Weather Research Program. The document provides an assessment of the NCWF-2 forecasting capability, including quantitative verification of 1-h and 2-h probabilistic forecasts. The results of this evaluation will be provided to the FAA and National Weather Service (NWS) Aviation Weather Steering Group (AWSG) for its consideration of whether NCWF-2 is ready for transition to operations. The components of the NCWF-2 considered in this evaluation include all forecast lead times and the probabilistic capability. An evaluation of the NCWF-2 motion vectors is presented in a supplemental report. The NCWF-2 is evaluated using the operational National Convective Weather Detection product (NCWD). For a standard of comparison, the quality of the NCWF-2 is compared to the quality of the operational 1-h NCWF, the 1-h NCWD Persistence, the 1- and 2-h C-SIGMET, and the 2-h Collaborative Convective Forecast Product (CCFP). Overall statistics for the NCWF-2 were computed from 1 April – 30 September 2005. The statistical methodology is consistent with the approach used in previous evaluations of the NCWF such as Brown and Mahoney (2000). The results indicate that: • Overall, the NCWF-2 shows modest reliability and resolution. The reliability and resolution decrease with lead time, with the statistics for the 30-min forecast being distinctly better than the statistics for the longer lead times. • The 1-h probabilistic comparison shows that NCWF-2 performs better than the operational NCWF and C-SIGMET with respect to reliability and resolution. • The 1-h deterministic comparison shows that NCWF-2 at the 0.4 probability threshold performs similarly to the operational NCWF and better than the CSIGMET, as indicated by CSI statistics. • The 2-h probabilistic comparison shows that NCWF-2 performs better than the CCFP and C-SIGMET with respect to reliability and resolution. The reliability of the CCFP approaches that of the NCWF-2 at higher probabilities. • The 2-h deterministic comparison shows that NCWF-2 at the 0.4 probability threshold performs slightly better than the CCFP and C-SIGMET, as indicated by CSI statistics.