Accounting for Realties When Estimating the Field Failure Rate of Software

A realistic estimate of the field failure rate of software is essential in order to decide when to release the software while maintaining an appropriate balance between reliability, time-to-market and development cost. Typically, software reliability models are applied to system test data with the hope of obtaining an estimate of the software failure rate that will be observed in the field. Unfortunately, test environments are usually quite different from field environments. In this paper, we use a calibration factor to characterize the mismatch between the system test environment and the field environment, and then incorporate the factor into a widely used software reliability model. For projects that have both system test data and field data for one or more previous releases, the calibration factor can be empirically evaluated and used to estimate the field failure rate of a new release based on its system test data. For new projects, the calibration factor can be estimated by matching the software to related projects that have both system test data and field data.