S4: An O2r/r2o Infrastructure For Optimizing Satellite Data Utilization In NOAA Numerical Modeling Systems: A Step Toward Bridging The Gap Between Research and Operations

Abstract

In 2011, the National Oceanic and Atmospheric Administration (NOAA) began a cooperative initiative with the academic community, to help address a vexing issue that has long been known as a disconnection between the operational and research realms for weather forecasting and data assimilation. The issue is the gap, or more exotically referred to as the ‘valley of death’, between efforts within the broader research community and NOAA’s activities, which are heavily driven by operational constraints. With the stated goals of leveraging research community efforts to benefit NOAA’s mission and offering a path to operations for the latest research activities which support the NOAA mission, satellite data assimilation in particular, this initiative aims to enhance the linkage between NOAA’s operational systems and the research efforts. A critical component is the establishment of an efficient Operations-To-Research (O2R) environment on the Supercomputer for Satellite Simulations and data assimilation Studies (S4). This O2R environment is critical for successful Research-To-Operations (R2O) transitions because it allows rigorous tracking, implementation and merging of any changes necessary (to operational software codes, scripts, libraries, etc.) to achieve the scientific enhancement. So far, the S4 O2R environment, with close to 4700 computing cores (60 TFLOPs) and 1700 TB disk storage capacity, has been a great success, and consequently was recently expanded to significantly increase its computing capacity. The objective of this article is to highlight some of the major achievements and benefits of this O2R approach, and some lessons learned, with the ultimate goal of inspiring other O2R/R2O initiatives in other areas and for other applications.