A Quantitative Analysis of Variability Warnings in Linux

In order to get insight into challenges with quality in highly-configurable software, we analyze one of the largest open source projects, the Linux kernel, and quantify basic properties of configuration-related warnings. We automatically analyze more than 20 thousand valid and distinct random configurations, in a computation that lasted more than a month. We count and classify a total of 400,000 warnings to get an insight in the distribution of warning types, and the location of the warnings. We run both on a stable and unstable version of the Linux kernel. The results show that Linux contains a significant amount of configuration-dependent warnings, including many that appear harmful. In fact, it appears that there are no configuration-independent warnings in the kernel at all, adding to our knowledge about relevance of family based analyses.