Mitigating the Risk of Defects and Improving Knowledge Distribution with Code Reviewer Recommenders

Defects are inevitable in software projects, leading to increased maintenance costs, user dissatisfaction, and a diminished software reputation. Code review is one of the most critical software quality assurance activities that reduces software defects and improves software quality. Prior works have quantified the impact of reviewer recommenders on the risk of inducing new defects based on the highest level of expertise among the developers in the reviewer team. However, our analysis shows that prior work overestimates the safety of a change and ignores the defect-finding effectiveness of the diverse knowledge of reviewers. In this study, we incorporate the knowledge of the entire reviewer team into the author’s level of expertise and introduce the novel Contribution-aware Changeset Safety Ratio (CCSR) outcome to assess the impact of code reviewer recommenders on the risk of inducing defects more accurately.

When a pull request is risky, a natural mitigation is to add an expert reviewer. We are unaware of any works that have quantified the impact of adding a reviewer to risky PRs. We propose the novel AddExpertRec(𝐷𝑡) strategy that recommends an additional expert reviewer for defect-prone pull requests to reduce the likelihood of introducing new defects when the risk is above the threshold 𝐷𝑡. The simulation results show that AddExpertRec(𝐷𝑡) can enhance the defect finding effectiveness of existing recommenders while still balancing reviewer workload and spreading knowledge to reduce the impact of turnover. Ultimately, our results give managers the ability to select a recommender strategy that best suits their project needs based on their resource constraints. The scripts and data are available in our replication package.