An Update On MADIS Observation Ingest, Integration, Quality Control, and Distribution Capabilities

Abstract

NOAA’s Forecast Systems Laboratory (FSL) has established the Meteorological Assimilation Data Ingest System (MADIS) to make integrated, quality-controlled datasets available to the greater meteorological community. The goals of MADIS are to promote the comprehensive data collection and distribution of operational and experimental observation systems, and to decrease the cost and time required to move new observing systems and products from research to operations. MADIS users have access to a reliable and easy-to-use database containing real-time and saved real-time datasets available via ftp, Local Data Manager (LDM), or through the use of web-based OPen source project for Network Data Access Protocol (OPeNDAP) clients. Observational datasets currently available via MADIS include radiosonde soundings, automated aircraft reports, NOAA and non-NOAA wind profilers, non-NOAA experimental microwave radiometer observations, operational and experimental NOAA Geostationary Operational Environmental Satellite (GOES) system winds, and several types of surface datasets. The latter includes water vapor observations from geo-positioning satellites (GPS) and a unique, national collection of over 14,500 mesonet stations from local, state, and federal agencies, and private firms. MADIS also supports NOAA with ingest, quality control (QC), and distribution of surface observations from modernized Cooperative Observer (COOP-M) stations, and has been tasked to contribute to the development of a national transportation mesonet consisting of integrated Road Weather Information System (RWIS) data from state Departments of Transportation (DOTs) as an integral part of the NOAA/ Federal Highway Administration (FHWA) surface transportation partnership. MADIS data files are available in uniform formats with uniform quality control structures within the data files,and are compatible with the National Weather Service (NWS) Advanced Weather Interactive Processing System (AWIPS) systems, and with data assimilation systems such as the Weather Research and Forecasting 3Dvariational (WRF 3D-VAR) system. Software support is also provided for the datasets through the use of an Application Program Interface (API) that provides users with easy access to the data and QC information. The API allows each user to specify station and observation types, as well as QC choices, and domain and time boundaries. Many of the implementation details that arise in data ingest programs are automatically performed, greatly simplifying user access to the disparate datasets, and effectively integrating the database by allowing, for example, users to access COOP-M, maritime, and non- NOAA mesonets through a single interface. MADIS datasets were first made publicly available in July 2001, and have proven to be popular within the meteorological community. FSL now supports hundreds of MADIS users, including the majority of NWS forecast offices, the National Climatic Data Center (NCDC), the National Centers for Environmental Prediction (NCEP), and many universities and private companies. Additionally, MADIS supplies non-NOAA data providers with QC and station monitoring information, which have proven useful in their maintenance activities. This paper provides a general overview of MADIS capabilities. For more information, see Barth et. al (2002), Miller and Barth (2003), and the MADIS web pages (FSL 2005).