Ask the engineers: Exploring repertory grids and personal constructs for software data analysis

Maturity in software projects is often equated with data-driven predictability. However, data collection is expensive and measuring all variables that may correlate with project outcome is neither practical nor feasible. In contrast, a project engineer can identify a handful of factors that he or she believes influence the success of a project. The challenge is to quantify engineers' insights in a way that is useful for data analysis. In this exploratory study, we investigate the repertory grid technique for this purpose. The repertory grid technique is an interview-based procedure for eliciting 'constructs' (e.g., Adhering to coding standards) that individuals believe influence a worldly phenomenon (e.g., What makes a high-quality software project) by comparing example elements from their past (e.g., Projects they have worked on). We investigate the relationship between objective metrics of project performance and repertory grid constructs elicited from eight software engineers. Our results show correlations between the engineers' subjective constructs and the objective project outcome measures. This suggests that repertory grids may be of benefit in developing models of project outcomes, particularly when project data is limited.