Variability Bugs:: Program and Programmer Perspective

Many modern software systems are highly configurable. They embrace variability to increase adaptability and to lower cost. To implement configurable software, developers often use the C preprocessor (CPP), which is a well-known technique, mainly in industry, to deal with variability in code. Although many researchers suggest that preprocessor-based variability amplifies maintenance problems, there is little to no hard evidence on how actually variability affects programs and programmers. Specifically, how does variability affect programmers during maintenance tasks (bug finding in particular)? How much harder is it to debug a program as variability increases? How do developers debug programs with variability? In what ways does variability affect bugs?
In this Ph.D. thesis, I set off to address such issues through different perspectives using empirical research (based on controlled experiments) in order to understand quantitatively and qualitatively the impact of variability on programmers at bug finding and on buggy programs.
From the program (and bug) perspective, the results show that variability is ubiquitous. There appears to be no specific nature of variability bugs that could be exploited. Variability bugs are not confined to any particular type of bug, error-prone feature, or location. In addition to introducing an exponential number of program variants, variability increases the complexity of bugs due to unintended feature interactions, hidden features, combinations of layers (code, mapping, model), many function calls, etc.
From the programmer (and bug-finding) perspective, I find that the time needed for finding bugs increases linearly with variability, while finding bugs in the first place is relatively independent of variability. In fact, most developers correctly identify bugs, yet many fail to identify the exact set of erroneous configurations. I also observe that developers navigate much more between definitions and uses of program objects when interleaved with variability.
Overall, this Ph.D. thesis shows that variability complicates the complexity of bugs and bug finding, but not terribly so. This is positive and consistent with the existence of highly-configurable software systems with hundreds, even thousands, of features, testifying that developers in the trenches are able to deal with variability.