Beyond Continuous Delivery: An Empirical Investigation of Continuous Deployment Challenges

Context: A growing number of software organizations have been adopting Continuous DElivery (CDE) and Continuous Deployment (CD) practices. Researchers have started investing significant efforts in studying different aspects of CDE and CD. Many studies refer to CDE (i.e., where an application is potentially capable of being deployed) and CD (i.e., where an application is automatically deployed to production on every update) as synonyms and do not distinguish them from each other. Despite CDE being successfully adopted by a large number of organizations, it is not empirically known why organizations still are unable or demotivated to have automatic and continuous deployment (i.e., CD practice). Goal: This study aims at empirically investigating and classifying the factors that may impact on adopting and implementing CD practice. Method: We conducted a mixed-method empirical study consisting of interviewing 21 software practitioners, followed by a survey with 98 respondents. Results: Our study reveals 11 confounding factors that limit or demotivate software organizations to push changes automatically and continuously to production. The most important ones are "lack of automated (user) acceptance test", "manual quality check", "deployment as business decision", "insufficient level of automated test coverage", and "highly bureaucratic deployment process". Conclusion: Our findings highlight several areas for future research and provide suggestions for practitioners to streamline deployment process.