The IDEAS Project has specific use case requirements, an ambitious effort to improve compatibility and usability of important DOE libraries, improve the practices, processes, and tools for scientific software development, and improve community knowledge.
However, the most important goal of the project is to help move scientific software development toward an approach of building new applications as a composition of reusable, robust, and scalable software components and libraries, using the best available practices and tools. We want to develop, demonstrate, use and make possible for others the ability to create and support new applications better, faster and cheaper.
The scientific software community is facing a crisis created by the confluence of disruptive changes in computing architectures and new opportunities for greatly improved simulation capabilities. New architectures are demanding fundamental algorithm and software refactoring, while at the same time enabling new multiscale and multiphysics modeling, simulation, and analysis. This crisis brings with it a unique opportunity to fundamentally change how scientific software is designed, developed, and supported.
We have assembled an interdisciplinary team that spans leadership of several critical scientific libraries (including hypre, PETSc, SuperLU and Trilinos); world-class software engineering programs; developers of critical tools for program understanding, refactoring, transformation, and code generation; and flagship science programs in BER, including three Science Focus Area programs, the Next Generation Ecosystem Experiment (NGEE) Arctic program, and projects funded by the Water Sustainability and Climate program at NSF. This team has both the breadth and the skill to meet this challenge and realize the full potential of emerging extreme-scale systems.
In our project, Interoperable Design of Extreme-scale Application Software (IDEAS), we are developing and demonstrating new approaches for producing, using, and supporting scientific software. We are establishing methodologies and tools that facilitate delivery of software as reusable, interoperable components. We are defining software lifecycle models that are both flexible and rigorous, acknowledging that scientific software inception occurs in a research environment but must ultimately result in reliable, well-designed, and well-documented software with a full test suite, so that future use and refactoring can be done with confidence.
This approach will address the historical yet unsustainable trend that science applications make modest use of libraries but are otherwise self-contained. Specifically, by supporting suitable levels of autonomy for applications developers within a rich ecosystem of interoperable components, productivity gains can be demonstrated across the software ecosystem—with many opportunities to develop and leverage common capabilities. Any consensus in policies, practices, and tools can only improve reusability, reduce education costs for new team members, and foster the use of best practices across all application teams. Finally, forums for sharing plans across the community at all phases of development will enable informed decision making from the earliest stages of application development.
We are defining community collaboration models that provide opportunities and incentives for application developers to create and adopt emerging tools and best practices. We are also providing community forum models that enable communication of requirements and plans so that application teams are informed about complementary activities as early as possible. All of the community-oriented infrastructure will enhance collaboration, while at the same time protect and foster small-team autonomy as much as possible, recognizing that local control enhances creativity, fosters broader design exploration, and generally provides a dynamic exchange of ideas across the community.
Improving Software Productivity
Scientific productivity is one of the top ten exascale research challenges, and software productivity (the effort, time, and cost for software development, maintenance, and support) is one critical aspect of scientific productivity. Therefore, although general and actionable productivity metrics are very difficult to define, an overarching focus on productivity is essential to making clear decisions in this time of highly disruptive architectural changes. In this project we are focusing on improving productivity by defining and taking steps toward a new scientific software ecosystem that emphasizes the use and development of high-quality tools, adaptation and adoption of modern software engineering methodologies, and the development and use of high-quality software components in the composition of next-generation applications.
The IDEAS project focuses on the development of an extreme-scale scientific software ecosystem that will provide a foundation for next-generation scientific simulations. To ensure the effectiveness of our efforts, we are specifically addressing the modeling and simulation needs of three important use cases: (i) climate impacts on the upper Colorado river system; (ii) the hydrology and soil carbon dynamics of the Arctic tundra, and (iii) hydrologic, land surface, and atmospheric process coupling over the continental United States. At the same time, because many scientific application areas face similar challenges in multiphysics and multiscale modeling, and because many ASCR project participants pursue ongoing work with diverse science groups, our approach to the software ecosystem will also serve the broader DOE extreme-scale computational science community.