An ontology to support empirical studies in software engineering

Ontologies are semantically organized collections of information pieces. Ontologies provide a way of organizing and encoding the collected knowledge for a given domain. Formalizing the accumulated knowledge in such a framework enables all sorts of automated analysis. We present an ontology for analyzing empirical studies of software engineering, in particular the design of software engineering experiments. The design of such experiments consists of assigning human subjects to apply treatments, such as techniques or tools, to artifacts such as code or specifications. The particular design and available treatments depend on the goals of the experiment. Provisions for addressing various threats to validity constrain the available design space. Furthermore, the assignments have to be consistent with the available resources. By encapsulating the existing knowledge on designing experiments, we posit that it is possible to check a given design for validity and consistency. We present a case study encoding software inspection experiments into an ontology and show how we can use it for checking a proposed design for a new inspection experiment.