Propose-specific information related to prediction level at x and mean magnitude of relative error: A case study of software effort estimation

Abstract

The prediction level at x (PRED(x)) and mean magnitude of relative error (MMRE) are measured based on the magnitude of relative error between real and predicted values. They are the standard metrics that evaluate accurate effort estimates. However, these values might not reveal the magnitude of over-/under-estimation. This study aims to define additional information associated with the PRED(x) and MMRE to help practitioners better interpret those values. We propose the formulas associated with the PRED(x) and MMRE to express the level of scatters of predictive values versus actual values on the left (sig(Left)), on the right (sig(Right)), and on the mean of the scatters (sig). We depict the benefit of the formulas with three use case points datasets. The proposed formulas might contribute to enriching the value of the PRED(x) and MMRE in validating the effort estimation.