ABSTRACT
Feature flags are a popular method to control functionality in released code. They enable rapid development and deployment, but can also quickly accumulate technical debt. Complex interactions between feature flags can go unnoticed, especially if interdependent flags are located far apart in the code, and these unknown dependencies could become a source of serious bugs. Testing all possible combinations of feature flags is infeasible in large systems like Microsoft Office, which has about 12000 active flags. The goal of our research is to aid product teams in improving system reliability by providing an approach to automatically discover feature flag interdependencies. We use probabilistic reasoning to infer causal relationships from feature flag query logs. Our approach is language-agnostic, scales easily to large heterogeneous codebases, and is robust against noise such as code drift or imperfect log data. We evaluated our approach on real-world query logs from Microsoft Office and are able to achieve over 90% precision while recalling non-trivial indirect feature flag relationships across different source files. We also investigated re-occurring patterns of relationships and describe applications for targeted testing, determining deployment velocity, error mitigation, and diagnostics.
- Mona Attariyan and Jason Flinn. 2010. Automating Configuration Troubleshooting with Dynamic Information Flow Analysis. In OSDI. 10, 1–14. Google ScholarDigital Library
- Muffy Calder, Mario Kolberg, Evan H. Magill, and Stephan Reiff-Marganiec. 2003. Feature interaction: a critical review and considered forecast. Computer Networks, 41, 1 (2003), 115–141. issn:1389-1286 https://doi.org/10.1016/S1389-1286(02)00352-3 Google ScholarDigital Library
- Qingrong Chen, Teng Wang, Owolabi Legunsen, Shanshan Li, and Tianyin Xu. 2020. Understanding and Discovering Software Configuration Dependencies in Cloud and Datacenter Systems. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2020). Association for Computing Machinery, New York, NY, USA. 362–374. isbn:9781450370431 https://doi.org/10.1145/3368089.3409727 Google ScholarDigital Library
- Istehad Chowdhury and Mohammad Zulkernine. 2010. Can Complexity, Coupling, and Cohesion Metrics Be Used as Early Indicators of Vulnerabilities? In Proceedings of the 2010 ACM Symposium on Applied Computing (SAC ’10). Association for Computing Machinery, New York, NY, USA. 1963–1969. isbn:9781605586397 https://doi.org/10.1145/1774088.1774504 Google ScholarDigital Library
- Martin Fowler. 2010. FeatureToggle. https://martinfowler.com/bliki/FeatureToggle.html Google Scholar
- Thomas M. J. Fruchterman and Edward M. Reingold. 1991. Graph Drawing by Force-Directed Placement. Softw. Pract. Exper., 21, 11 (1991), nov, 1129–1164. issn:0038-0644 https://doi.org/10.1002/spe.4380211102 Google ScholarDigital Library
- Luke Harries, Rebekah Storan Clarke, Timothy Chapman, Swamy V. P. L. N. Nallamalli, Levent Özgür, Shuktika Jain, Alex Leung, Steve Lim, Aaron Dietrich, José Miguel Hernández-Lobato, Tom Ellis, Cheng Zhang, and Kamil Ciosek. 2020. DRIFT: Deep Reinforcement Learning for Functional Software Testing. CoRR, abs/2007.08220 (2020), arXiv:2007.08220. arxiv:2007.08220 Google Scholar
- Pete Hodgson. 2017. Feature Toggles (aka Feature Flags). https://martinfowler.com/articles/feature-toggles.html Google Scholar
- M. Jackson and P. Zave. 1998. Distributed feature composition: a virtual architecture for telecommunications services. IEEE Transactions on Software Engineering, 24, 10 (1998), 831–847. https://doi.org/10.1109/32.729683 Google ScholarDigital Library
- Flávio Medeiros, Márcio Ribeiro, Rohit Gheyi, Larissa Braz, Christian Kästner, Sven Apel, and Kleber Santos. 2020. An Empirical Study on Configuration-Related Code Weaknesses. In Proceedings of the 34th Brazilian Symposium on Software Engineering (SBES ’20). Association for Computing Machinery, New York, NY, USA. 193–202. isbn:9781450387538 https://doi.org/10.1145/3422392.3422409 Google ScholarDigital Library
- Jens Meinicke, Juan Hoyos, Bogdan Vasilescu, and Christian Kästner. 2020. Capture the Feature Flag: Detecting Feature Flags in Open-Source. In Proceedings of the 17th International Conference on Mining Software Repositories (MSR ’20). Association for Computing Machinery, New York, NY, USA. 169–173. isbn:9781450375177 https://doi.org/10.1145/3379597.3387463 Google ScholarDigital Library
- Jens Meinicke, Chu-Pan Wong, Bogdan Vasilescu, and Christian Kästner. 2020. Exploring Differences and Commonalities between Feature Flags and Configuration Options. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP ’20). Association for Computing Machinery, New York, NY, USA. 233–242. isbn:9781450371230 https://doi.org/10.1145/3377813.3381366 Google ScholarDigital Library
- Steve Neely and Steve Stolt. 2013. Continuous Delivery? Easy! Just Change Everything (Well, Maybe It Is Not That Easy). 2013 Agile Conference, 121–128. https://doi.org/10.1109/AGILE.2013.17 Google ScholarDigital Library
- Md Tajmilur Rahman, Louis-Philippe Querel, Peter C. Rigby, and Bram Adams. 2016. Feature Toggles: Practitioner Practices and a Case Study. In Proceedings of the 13th International Conference on Mining Software Repositories (MSR ’16). Association for Computing Machinery, New York, NY, USA. 201–211. isbn:9781450341868 https://doi.org/10.1145/2901739.2901745 Google ScholarDigital Library
- Doug Seven. 2014. Knightmare: A DevOps Cautionary Tale. https://dougseven.com/2014/04/17/knightmare-a-devops-cautionary-tale/ Google Scholar
- Chunqiang Tang, Thawan Kooburat, Pradeep Venkatachalam, Akshay Chander, Zhe Wen, Aravind Narayanan, Patrick Dowell, and Robert Karl. 2015. Holistic Configuration Management at Facebook. In Proceedings of the 25th Symposium on Operating Systems Principles (SOSP ’15). Association for Computing Machinery, New York, NY, USA. 328–343. isbn:9781450338349 https://doi.org/10.1145/2815400.2815401 Google ScholarDigital Library
- Jialu Zhang, Ruzica Piskac, Ennan Zhai, and Tianyin Xu. 2021. Static detection of silent misconfigurations with deep interaction analysis. PACM on Programming Languages, 5, OOPSLA (2021), 1–30. https://doi.org/10.1145/3485517 Google ScholarDigital Library
Index Terms
- Discovering feature flag interdependencies in Microsoft office
Comments