A comparative evaluation of validation techniques in software effort estimation using eSOMCOCOMO

Abstract

This study investigates the impact of different validation techniques on the performance evaluation of software effort estimation models. Specifically, it compares k-fold cross-validation, leave-one-out cross-validation (LOOCV), and hold-out validation using the eSOMCOCOMO approach, which enhances COCOMO model predictions through the Self-Organizing Migrating Algorithm (SOMA). The evaluation was conducted on three benchmark datasets (NASA18, Kemerer, and Miyazaki94) and assessed using standard evaluation metrics (MMRE, PRED(25), MMER, MAE, MSE, RMSE, and R2). Statistical hypothesis testing revealed significant differences among most validation techniques, except in the comparison conducted on the NASA18 dataset. LOOCV demonstrates superior stability across multiple runs, whereas hold-out validation showed high variance.