The Grid: An It Infrastructure For NOAA In The 21st Century

Abstract

This paper highlights the need to build a grid infrastructure to meet the challenges facing NOAA in the 21st century. Given the enormous expected demands for data, and increased size and density of observational systems, current systems will not be scalable for future needs without incurring enormous costs. NOAA’s IT infrastructure is currently a set of independent systems that have been built up over time to support its programs and requirements. NOAA needs integrated systems capable of handling a huge increase in data volumes from expected launches of GOES-R, NPOESS, new observing systems being proposed or developed, and to meet requirements of the Integrated Earth Observation System. Further, NOAA must continue moving toward integrated compute resources to reduce costs, to improve systems utilization, to support new scientific challenges and to run and verify increasingly complex models using next generation high-density data streams. Finally, NOAA needs a fast, well-managed network capable of meeting the needs of the organization: to efficiently distribute data to users, to provide secure access to IT resources, and be sufficiently adaptable and scalable to meet unanticipated needs in the future. Just as the NOAA Observing System Architecture (NOSA) was developed to define, coordinate and integrate data sources, NOAA needs to build an IT counterpart (NITSA) to define, coordinate and integrate its IT resources. We propose the construction of a grid infrastructure, centered around a fast network, which will permit resources including computers, data storage, software systems and services to be managed and shared more effectively. All three types of grids identified in this paper, can be utilized at NOAA: compute grids to provide access to underutilized cycles and to link super-computing centers, data grids to promote sharing and better utilization of data, and service grids to provide the reliability and redundancy demanded by operational processes. Grid extends the power and success of the World Wide Web (WWW) by providing robust mechanisms for data access, discovery and integration that can transform static methods of data distribution into dynamic demand driven delivery systems. We have also described a process by which secure grids can be built using token-based authentication, VPN communications between secure NOAA systems, and restricted external access through heavily monitored firewalls. NOAA currently spends hundreds of millions of dollars on observing systems, supercomputing facilities, data centers, and the network. The development of a NITSA, enabled by Grid, can potentially reap huge cost savings for the organization.