An Innovative Method to Hypothesis Testing for System Safety Assessment

Robert Thomas; Marilyn Eichelberger; Missey Lee; Joel Haan

doi:10.56094/jss.v51i3.147

Authors

Dr. Robert W. L. Thomas AECOM
Marilyn J. Eichelberger Naval Surface Warfare Center
Missey Lee Naval Surface Warfare Center
Joel Haan Naval Surface Warfare Center

DOI:

https://doi.org/10.56094/jss.v51i3.147

Keywords:

system safety, hypothesis, probability, weapons system, failure rate

Abstract

The way forward in system safety engineering will be quantitative, and this paper proposes an innovative method for generating a uniform way to understand the composite of testing and experience. In recent years, new approaches to exact hypothesis testing have been developed without a Gaussian probability distribution for success or failure rates. These techniques eliminate errors introduced by the Gaussian assumption, which is important for the small failure rates that are common in modern systems development, and offer considerable promise as a basis for the new direction.

This paper presents a theory for exact hypothesis testing and combines two 18th-century theorems to derive an equation for the probability distribution of failure rate employing only the number of tests and the observed count of failures. The concept is expanded to demonstrate the combination of operational experience and expert opinion to update test results. The objective in this work is to derive the general likelihood distribution of failure rate given any set of test results, and then to examine the implications regarding testing requirements, design and interpretation. The particular application considered here is safety assessment for a military weapons system. While the theory developed is for deriving the exact failure rate distribution for system safety applications, it is equally valid for investigating success rates and/or for interpreting performance evaluation tests.

Author Biographies

Dr. Robert W. L. Thomas, AECOM

Dr. Robert W. L. Thomas is a scientist at AECOM working on safety issues pertaining to electromagnetic environments near to high-power emitters. He has experience with the verification and validation of combat systems software, with a particular emphasis on critical timing issues, and has also developed models used by the National Aeronautics and Space Administration (NASA) to scope the independent verification and validation of satellite and spacecraftborne software. He has also designed optical instrumentation for space measurements to characterize planetary atmospheres.

Marilyn J. Eichelberger, Naval Surface Warfare Center

Marilyn J. Eichelberger is currently a member of the Combat Systems Safety Branch at the Naval Surface Warfare Center, Dahlgren Division. She has 21 years of experience supporting U.S. Navy, U.S. Marine Corps and foreign military programs, of which eight years have been in the role of Principal for Safety.

Missey Lee, Naval Surface Warfare Center

Missey Lee is currently a member of the Combat Systems Safety Branch at the Naval Surface Warfare Center, Dahlgren Division. She has more than 11 years of experience practicing system safety engineering for the U.S. Navy. Previously, she developed and implemented code for Weapon Systems of the U.S. Navy.

Joel Haan, Naval Surface Warfare Center

Joel Haan is currently a member of the Engagement System Safety Branch at the Naval Surface Warfare Center, Dahlgren Division. He has five years of experience practicing system safety engineering for U.S. Navy and U.S. Marine Corps safety programs. Prior to this, he worked on test engineering for both developmental and in-service weapon systems.