Project similarity measures for collaborative filtering-based effort estimation: Review and empirical study

Abstract

As software project development becomes increasingly complex, accurate effort estimation is essential for successful delivery. This study investigates the impact of similarity measures on estimation accuracy within the Neighborhood-Based Collaborative Filtering for Effort Estimation (NCFEE) context. We analyzed the performance of 17 similarity measures using benchmark datasets, specifically fpa_china and fpa_isbsg. Effectiveness was assessed through Root Mean Squared Error (RMSE) to quantify prediction accuracy, supplemented by effect size analysis to gauge the practical significance of observed differences. The results demonstrate that Jaccard-based measures (JAC, DiceJAC, and TanimotoJAC) consistently achieved the lowest RMSE values, indicating their strong ability to capture effort-related similarities by focusing on overlapping project features. Effect size analysis confirmed that these performance advantages are highly practically significant. Furthermore, the optimal number of nearest neighbors varied between datasets, with effect sizes highlighting the substantial impact of dataset characteristics on model performance. These findings underscore the importance of selecting appropriate similarity measures, particularly Jaccard-based approaches, to enhance the effectiveness of NCFEE.