Building a body of knowledge from families of software empirical studies: from statistical analysis to cluster-impact approach for manipulating experimental data using fuzzy sets

The explosive growth of the software industry in recent years has focused the attention on the problems long associated with software development. Empirical evaluations in software engineering are important for building a body of knowledge, which is missing for several software engineering areas nowadays. Transferring such body of knowledge to industry is one of the main objectives in research. In order to empirically achieve such an objective, it is our conviction that empirical studies should be considered from two viewpoints: one comes from the effect, the outcomes from as explicit as specific input variables or factors in traditional experimentation, where we will be able to provide studies with an initial benchmark of knowledge of software (“Effect Viewpoint”), while the other comes from the integral cause, where we will be able to understand empirical software studies as an integrated body of knowledge, based on the analysis of all possible input variables - including factors, parameters, and blocking variables - through families of experiments and other empirical studies (“Cause Viewpoint”). Along with current study, we present both Effect and Cause viewpoints. Current dissertation is an attempt to study the state of the art of experimentation in software engineering, aiming to understand variables that cause effects when empirical investigations are targeting to Object Oriented (OO) software applications and utilities, including OO software frameworks, which are designed and adopted from a set of OO applications in a specific domain. In the first part of current dissertation, we focus on the Effect Viewpoint; in particular, rules of empiricism are applied for investigating software-reading techniques. Two different, but related in goal, empirical packages are considered; the first package aims at evaluating the effectiveness of reading techniques for defect detection in OO C++ software frameworks, while the last package aims at evaluating the effectiveness of software testing strategies (static and dynamic techniques) for defect detection in OO event driven Java software. In the second part of current dissertation, we focus on the Cause Viewpoint. An approach, so called Cluster-impact Approach, is defined and eventually implemented, which spotlights on sets of variables that impact on out-coming responses when conducting empirical studies. With Cluster-impact Approach, collected empirical data from families of experiments (i.e. one or more experiments, each with one or more replications), can be significantly schematized and eventually engaged into a process that aims to build a body of knowledge for any OO software. Such a process is based on integration and formalization mechanisms: Fuzzy sets are used for transforming data from being quantitative to qualitative; transformed data are analyzed qualitatively for drawing the study conclusions.