Software Engineering for Science 1st edition by Jeffrey Carver ISBN 1498743854 9781498743854

Introduction

1 Software Process for Multiphysics Multicomponent Codes

1.1 Introduction

1.2 Lifecycle

1.2.1 Development Cycle

1.2.2 Verification and Validation

1.2.3 Maintenance and Extensions

1.2.4 Performance Portability

1.2.5 Using Scientific Software

1.3 Domain Challenges

1.4 Institutional and Cultural Challenges

1.5 Case Studies

1.5.1 FLASH

1.5.1.1 Code Design

1.5.1.2 Verification and Validation

1.5.1.3 Software Process

1.5.1.4 Policies

1.5.2 Amanzi/ATS

1.5.2.1 Multiphysics Management through Arcos

1.5.2.2 Code Reuse and Extensibility

1.5.2.3 Testing

1.5.2.4 Performance Portability

1.6 Generalization

1.7 Additional Future Considerations

2 A Rational Document Driven Design Process for Scientific Software

2.1 Introduction

2.2 A Document Driven Method

2.2.1 Problem Statement

2.2.2 Development Plan

2.2.3 Software Requirements Specification (SRS)

2.2.4 Verification and Validation (V&V) Plan and Report

2.2.5 Design Specification

2.2.6 Code

2.2.7 User Manual

2.2.8 Tool Support

2.3 Example: Solar Water Heating Tank

2.3.1 Software Requirements Specification (SRS)

2.3.2 Design Specification

2.4 Justification

2.4.1 Comparison between CRAN and Other Communities

2.4.2 Nuclear Safety Analysis Software Case Study

2.5 Concluding Remarks

3 Making Scientific Software Easier to Understand, Test, and Communicate through Software Engineerin

3.1 Introduction

3.2 Case Studies

3.3 Challenges Faced by the Case Studies

3.3.1 Intuitive Testing

3.3.2 Automating Tests

3.3.3 Legacy Code

3.3.4 Summary

3.4 Iterative Hypothesis Testing

3.4.1 The Basic SEIR Model

3.4.2 Experimental Methodology

3.4.3 Initial Hypotheses

3.4.3.1 Sanity Checks

3.4.3.2 Metamorphic Relations

3.4.3.3 Mathematical Derivations

3.4.4 Exploring and Refining the Hypotheses

3.4.4.1 Complexities of the Model

3.4.4.2 Complexities of the Implementation

3.4.4.3 Issues Related to Numerical Precision

3.4.5 Summary

3.5 Testing Stochastic Software Using Pseudo-Oracles

3.5.1 The Huánglóngbìng SECI Model

3.5.2 Searching for Differences

3.5.3 Experimental Methodology

3.5.4 Differences Discovered

3.5.5 Comparison with Random Testing

3.5.6 Summary

3.6 Conclusions

3.7 Acknowledgments

4 Testing of Scientific Software: Impacts on Research Credibility, Development Productivity, Maturat

4.1 Introduction

4.2 Testing Terminology

4.2.1 Granularity of Tests

4.2.2 Types of Tests

4.2.3 Organization of Tests

4.2.4 Test Analysis Tools

4.3 Stakeholders and Team Roles for CSE Software Testing

4.3.1 Stakeholders

4.3.2 Key Roles in Effective Testing

4.3.3 Caveats and Pitfalls

4.4 Roles of Automated Software Testing in CSE Software

4.4.1 Role of Testing in Research

4.4.2 Role of Testing in Development Productivity

4.4.3 Role of Testing in Software Maturity and Sustainability

4.5 Challenges in Testing Specific to CSE

4.5.1 Floating-Point Issues and Their Impact on Testing

4.5.2 Scalability Testing

4.5.3 Model Testing

4.6 Testing Practices

4.6.1 Building a Test Suite for CSE Codes

4.6.2 Evaluation and Maintenance of a Test Suite

4.6.3 An Example of a Test Suite

4.6.4 Use of Test Harnesses

4.6.5 Policies

4.7 Conclusions

4.8 Acknowledgments

5 Preserving Reproducibility through Regression Testing

5.1 Introduction

5.1.1 Other Testing Techniques

5.1.2 Reproducibility

5.1.3 Regression Testing

5.2 Testing Scientific Software

5.2.1 The Oracle and Tolerance Problems

5.2.1.1 Sensitivity Testing

5.2.2 Limitations of Regression Testing

5.3 Regression Testing at ESG

5.3.1 Building the Tools

5.3.1.1 Key Lesson

5.3.2 Selecting the Tests

5.3.2.1 Key Lessons

5.3.3 Evaluating the Tests

5.3.3.1 Key Lessons

5.3.4 Results

5.4 Conclusions and Future Work

6 Building a Function Testing Platform for Complex Scientific Code

6.1 Introduction

6.2 Software Engineering Challenges for Complex Scientific Code

6.3 The Purposes of Function Unit Testing for Scientific Code

6.4 Generic Procedure of Establishing Function Unit Testing for Large-Scale Scientific Code

6.4.1 Software Analysis and Testing Environment Establishment

6.4.2 Function Unit Test Module Generation

6.4.3 Benchmark Test Case Data Stream Generation Using Variable Tracking and Instrumentation

6.4.4 Function Unit Module Validation

6.5 Case Study: Function Unit Testing for the ACME Model

6.5.1 ACME Component Analysis and Function Call-Tree Generation

6.5.2 Computational Characteristics of ACME Code

6.5.3 A Function Unit Testing Platform for ACME Land Model

6.5.3.1 System Architecture of ALM Function Test Framework

6.5.3.2 Working Procedure of the ALM Function Test Framework

6.6 Conclusion

7 Automated Metamorphic Testing of Scientific Software

7.1 Introduction

7.2 The Oracle Problem in Scientific Software

7.3 Metamorphic Testing for Testing Scientific Software

7.3.1 Metamorphic Testing

7.3.2 Applications of MT for Scientific Software Testing

7.4 MRpred: Automatic Prediction of Metamorphic Relations

7.4.1 Motivating Example

7.4.2 Method Overview

7.4.3 Function Representation

7.4.4 Graph Kernels

7.4.4.1 The Random Walk Kernel

7.4.5 Effectiveness of MRpred

7.5 Case Studies

7.5.1 Code Corpus

7.5.2 Metamorphic Relations

7.5.3 Setup

7.6 Results

7.6.1 Overall Fault Detection Effectiveness

7.6.2 Fault Detection Effectiveness across MRs

7.6.3 Effectiveness of Detecting Different Fault Categories

7.7 Conclusions and Future Work

8 Evaluating Hierarchical Domain-Specific Languages for Computational Science: Applying the Sprat Ap

8.1 Motivation

8.2 Adapting Domain-Specific Engineering Approaches for Computational Science

8.3 The Sprat Approach: Hierarchies of Domain-Specific Languages

8.3.1 The Architecture of Scientific Simulation Software

8.3.2 Hierarchies of Domain-Specific Languages

8.3.2.1 Foundations of DSL Hierarchies

8.3.2.2 An Example Hierarchy

8.3.3 Applying the Sprat Approach

8.3.3.1 Separating Concerns

8.3.3.2 Determining Suitable DSLs

8.3.3.3 Development and Maintenance

8.3.4 Preventing Accidental Complexity

8.4 Case Study: Applying Sprat to the Engineering of a Coupled Marine Ecosystem Model

8.4.1 The Sprat Marine Ecosystem Model

8.4.2 The Sprat PDE DSL

8.4.3 The Sprat Ecosystem DSL

8.4.4 The Ansible Playbook DSL

8.5 Case Study Evaluation

8.5.1 Data Collection

8.5.2 Analysis Procedure

8.5.3 Results from the Expert Interviews

8.5.3.1 Learning Material for DSLs

8.5.3.2 Concrete Syntax: Prescribed vs. Flexible Program Structure

8.5.3.3 Internal vs. External Implementation

8.6 Conclusions and Lessons Learned

9 Providing Mixed-Language and Legacy Support in a Library: Experiences of Developing PETSc

9.1 Introduction

9.2 Fortran-C Interfacing Issues and Techniques

9.3 Automatically Generated Fortran Capability

9.4 Conclusion

10 HydroShare – A Case Study of the Application of Modern Software Engineering to a Large Distribu

10.1 Introduction to HydroShare

10.2 Informing the Need for Software Engineering Best Practices for Science

10.3 Challenges Faced and Lessons Learned

10.3.1 Cultural and Technical Challenges

10.3.2 Waiting Too Long between Code Merges

10.3.3 Establishing a Development Environment

10.4 Adopted Approach to Software Development Based on the Lessons Learned

10.4.1 Adopting Best Practices in Modern Software Engineering

10.4.2 Iterative Software Development

10.4.3 Virtual Machines

10.4.4 Code Versioning

10.4.5 Code Reviews

10.4.6 Testing and Test-Driven Development

10.4.7 Team Communication

10.4.8 DevOps

10.5 Making Software Engineering More Feasible and Easier to Integrate into One’s Research Activit

10.6 Conclusion

References

Index