CISC 834 is a grad course given each Fall by Dr. Bram Adams.
It aims to introduce graduate students to the wonderful, yet challenging world of software release
engineering,
DevOps and MLOps. Software release engineering is the discipline of integrating, building, testing,
deploying and
releasing high-quality software releases to the end user. These activities form a vital link in today's
DevOps
world,
which builds on agile development processes to add a feedback loop between developers and operators. The
advent of
revolutionary AI technology has pushed the established concepts of
DevOps even further, leading to the recent fields of
MLOps or even LLMOps.
While many people still think of software as cardboard boxes bought in their local electronics shop, the
advent of
agile
methodologies, the web and AI has changed the landscape drastically. Deployment of modern applications often
includes
coordinating the release of applications on multiple mobile platforms, web platforms with centralized
backend
services,
app stores, and native desktop clients. Furthermore, concepts like continuous delivery of software are no
longer
curiosities, but essential to retain a competitive edge. Did you know that lean start-ups like IMVU release up to 50 times per
day, while modern companies like Intuit, Google and Mozilla only take
a couple of weeks in between releases? Browsers like Google
Chrome and Mozilla Firefox even update
silently to
new
releases, without the user noticing it. Similar notions are being developed for today's AI systems.
How can a modern software organization achieve such a highly performant release engineering process? How can
delays
and
inefficiencies be identified and monitored, and how can they be resolved? As a methodology towards dealing
with
these
questions, this course relies on techniques from the area of mining software
repositories, which leverages a wide range
of readily available databases such as version control repositories, bug repositories, mailing lists and
other data
gathered during or following the development process, in order to extract actionable data that can help
stakeholders
like release engineers or gatekeepers to improve integration or to make the right decision.
The following conference and journal papers started as an assignment or project in CISC 834, keep an eye on
this section
for more ;-)
-
Lin Geng (*), Hao Li (*), Sidney Givigi and Bram Adams (2026). SwarmUpdate: Hierarchical Software Updates and Deep Learning Model Patching for Heterogeneous UAV Swarms, Transactions on Software Engineering and Methodology (TOSEM), ACM, to appear.
-
Mohammed Mehedi Hasan (*), Hao Li (*), Emad Fallahzadeh (*), Gopi Krishnan Rajbahadur, Bram Adams and Ahmed E. Hassan (2026). An Empirical Study of Testing Practices in Open Source AI Agent Frameworks and Agentic Applications, Empirical Software Engineering (EMSE), 31(5), Springer.
-
Hao Li (*), Hicham Masri (*), Filipe Roseiro Côgo, Abdul Ali Bangash (*), Bram Adams and Ahmed E. Hassan (2026). Understanding Prompt Management in GitHub Repositories: A Call for Best Practices, IEEE Software, IEEE, to appear.
-
Aaditya Bhatia (*), Foutse Khomh, Bram Adams and Ahmed E. Hassan (2026). An Empirical Study of Self-Admitted Technical Debt in Machine Learning Software, Transactions on Software Engineering and Methodology (TOSEM), ACM, to appear.
-
Jaskirat Singh (*), Emad Fallahzadeh (*), Bram Adams and Ahmed E. Hassan (2025). On the Impact of Black-box Deployment Strategies for Edge AI on Latency and Model Performance - Empirical Study on ONNX, Empirical Software Engineering (EMSE), Springer, to appear.
-
Mateus Karvat, Bram Adams and Sidney Givigi (2026). From Leaderboard to Deployment: Code Quality Challenges in AV Perception Repositories, in Proceedings of the 29th IEEE International Conference on Intelligent Transportation Systems (ITSC) (Naples, Italy), IEEE.
-
Adekunle Ajibode (*), Abdul Ali Bangash (*), Filipe Roseiro Côgo, Bram Adams and Ahmed E. Hassan (2025). Towards Semantic Versioning of Open Pre-trained Language Model Releases on Hugging Face, Empirical Software Engineering (EMSE), 30(3), Springer.
-
Jerin Yasmin, Jiale Wang, Yuan Tian and Bram Adams (2025). An Empirical Study of Developers' Challenges in Implementing Workflows as Code: A Case Study on Apache Airflow, Journal of Software and Systems (JSS), 219, Elsevier.
-
Ernesto Lang Oreamuno (*), Rohan Faiyaz Khan, Abdul Ali Bangash (*), Catherine Stinson and Bram Adams (2024). The State of Documentation Practices of Third-party Machine Learning Models and Datasets, IEEE Software, 41(5), p. 52-59, IEEE.
-
Divya M. Kamath (*), Eduardo Fernandes (*), Bram Adams and Ahmed E. Hassan (2024). On Combining Commit Grouping and Build Skip Prediction to Reduce Redundant Continuous Integration Activity, Empirical Software Engineering (EMSE), 29(6), Springer.
-
Shenyu Zheng (*), Bram Adams and Ahmed E. Hassan (2024). Does Using Bazel Help Speed Up Continuous Integration Builds?, Empirical Software Engineering (EMSE), 29(5), Springer.
-
Youssef Esseddiq Ouatiti, Mohammed Sayagh, Noureddine Kerzazi, Bram Adams and Ahmed E. Hassan (2024). The impact of Concept drift and Data leakage on Log Level Prediction Models, Empirical Software Engineering (EMSE), 29(5), Springer.
-
Harsh Patel (*), Bram Adams and Ahmed E. Hassan (2024). Post Deployment Recycling of Machine Learning Models - Don't Throw Away Your Old Models!, Empirical Software Engineering (EMSE), 29(4), Springer.
-
Zhimin Zhao (*), Yihao Chen (*), Abdul Ali Bangash (*), Bram Adams and Ahmed E. Hassan (2024). An Empirical Study of Challenges in Machine Learning Asset Management, Empirical Software Engineering (EMSE), 29(4), Springer.
-
Akshat Malik (*), Bram Adams and Ahmed E. Hassan (2024). Towards Graph-Anonymization of Software Analytics Data: Empirical Study on JIT Defect Prediction, Empirical Software Engineering (EMSE), 29(4), Springer.
-
Rahul Bajaj (*), Eduardo Fernandes (*), Bram Adams and Ahmed E. Hassan (2024). Unreproducible builds: Time to fix, causes, and correlation with external ecosystem factors, Empirical Software Engineering (EMSE), 29(1), Springer.
-
Yihao Chen (*), Eduardo Fernandes (*), Bram Adams and Ahmed E. Hassan (2023). On practitioners' concerns when adopting service mesh frameworks, Empirical Software Engineering (EMSE), 28(5), Springer.
-
Mahmoud Alfadel, Diego Elias Costa, Emad Shihab and Bram Adams (2023). On the Discoverability of npm Vulnerabilities in Node.js Projects, Transactions on Software Engineering and Methodology (TOSEM), 32(4), p. 1-27, ACM.
-
Kazi Amit Hasan, Marcos Macedo (*), Yuan Tian, Bram Adams and Steven Ding (2023). Understanding the Time to First Response In GitHub Pull Requests, in Proceedings of the 20th International Conference on Mining Software Repositories (MSR) (Melbourne, Australia), p. 1-11. (Acceptance ratio: 43/115=37.4%)
-
Benjamin Rombaut, Filipe Roseiro Côgo, Bram Adams and Ahmed E. Hassan (2022). There's no Such Thing as a Free Lunch: Lessons Learned from Exploring the Overhead Introduced by the Greenkeeper Dependency Bot in Npm, Transactions on Software Engineering and Methodology (TOSEM), 32(1), p. 1-40, ACM.
-
Suhaib Mujahid, Diego Elias Costa, Rabe Abdalkareem, Emad Shihab, Mohamed Aymen Saied and Bram Adams (2022). Toward Using Package Centrality Trend to Identify Packages in Decline, Transactions on Engineering Management (TEM), 69(6), p. 3618-3632, IEEE.
-
Md Atique Reza Chowdhury, Rabe Abdalkareem, Emad Shihab and Bram Adams (2022). On the Untriviality of Trivial Packages: An Empirical Study of npm JavaScript Packages, Transactions on Software Engineering (TSE), 48(8), p. 2695-2708, IEEE.
-
Shivashree Vysali (*), Shane McIntosh and Bram Adams (2022). Quantifying, Characterizing, and Mitigating Flakily Covered Program Elements, Transactions on Software Engineering (TSE), 48(3), p. 1018-1029, IEEE.
-
Amine Barrak (*), Ellis E. Eghan (*), Bram Adams and Foutse Khomh (2021). Why do Builds Fail? - A Conceptual Replication Study, Journal of Software and Systems (JSS), 177, 110939 pages, Elsevier.
-
Sophia Quach, Maxime Lamothe, Bram Adams, Yasutaka Kamei and Weiyi Shang (2021). Evaluating the impact of falsely detected performance bug-inducing changes in JIT models, Empirical Software Engineering (EMSE), 26(97), Springer.
-
Arghavan Sanei, Jinghui Cheng and Bram Adams (2021). The Impacts of Sentiments and Tones in Community-Generated Issue Discussions, in Proceedings of the 14th International Conference on Cooperative and Human Aspects of Software Engineering (CHASE) (Madrid, Spain), p. 1-10. (Acceptance ratio: 10/38=26.32%)
-
Moses Openja, Bram Adams and Foutse Khomh (2020). Analysis of Modern Release Engineering Topics - A Large-Scale Study using StackOverflow, in Proceedings of the 36th International Conference on Software Maintenance and Evolution (ICSME) (Adelaide, Australia), p. 104-114, IEEE. (Acceptance ratio: 58/201=28.9%)
-
Marc Lanovaz and Bram Adams (2019). A Comparison of Communication Tone and Responding Across Users and Developers in Two R Mailing Lists, IEEE Software (Special Issue on Sentiment and Emotion in Software Engineering), 36(5), p. 46-50, IEEE.
-
Houssem Ben Braiek, Foutse Khomh and Bram Adams (2018). The Open-Closed Principle of Modern Machine Learning Frameworks, in Proceedings of the 15th International Conference on Mining Software Repositories (MSR) (Gothenburg, Sweden), p. 353-363. (Acceptance ratio: 37/113=32.7%)
-
Mahdis Zolfagharinia (*), Bram Adams and Yann-Gaël Guéhéneuc (2017). Do Not Trust Build Results at Face Value - An Empirical Study of 30 Million CPAN Builds, in Proceedings of the 14th International Conference on Mining Software Repositories (MSR) (Buenos Aires, Argentina), p. 312-322. (Acceptance ratio: 37/121=30.6%)
-
Foundjem Tita Armstrong (*), Foutse Khomh and Bram Adams (2017). Broadcast vs. Unicast Review Technology: Does it Matter?, in Proceedings of the 10th International Conference on Software Testing, Verification and Validation (ICST) (Tokyo, Japan), p. 219-229, IEEE. (Acceptance ratio: 36/135=27%)
-
Abderrahmane Benbachir, Isnaldo Francisco De Melo Jr., Michel Dagenais and Bram Adams (2017). Automated Performance Deviation Detection Across Software Versions Releases [short], in Proceedings of the 2017 International Conference on Software Quality, Reliability and Security (QRS) (Prague, Czech Republic), p. 450-457, IEEE.
-
Pooya Musavi, Bram Adams and Foutse Khomh (2016). Experience Report: An Empirical Study of API Failures in OpenStack Cloud Environments, in Proceedings of the 27th International Symposium on Software Reliability Engineering (ISSRE) (Ottawa, ON), p. 424-434. (Acceptance ratio: 45/130=34.6%)
-
Md Tajmilur Rahman, Louis-Philippe Querel, Peter C. Rigby and Bram Adams (2016). Feature Toggles: A case study and survey, in Proceedings of the 13th Working Conference on Mining Software Repositories (MSR) (Austin, TX), p. 201-211. (Acceptance ratio: 36/103=27%)
-
Vincent Boisselle (*) and Bram Adams (2015). The Impact of Cross-Distribution Bug Duplicates, Empirical Study on Debian and Ubuntu, in Proceedings of the 15th International Working Conference on Source Code Analysis and Manipulation (SCAM) (Bremen, Germany), p. 131-140, IEEE. (Acceptance ratio: 24/68=35%)
-
Mohammed Sayagh (*) and Bram Adams (2015). Multi-layer Software Configuration - Empirical Study on Wordpress, in Proceedings of the 15th International Working Conference on Source Code Analysis and Manipulation (SCAM) (Bremen, Germany), p. 31-40, IEEE. (Acceptance ratio: 24/68=35%)
-
Parastou Tourani (*), Yujuan Jiang (*) and Bram Adams (2014). Monitoring Sentiment in Open Source Mailing Lists - Exploratory Study on the Apache Ecosystem, in Proceedings of the 2014 Conference of the Center for Advanced Studies on Collaborative Research (CASCON) (Markham, ON, Canada), p. 34-44. (Acceptance ratio: 19/56=33.9%)
-
Le An, Foutse Khomh and Bram Adams (2014). Supplementary Bug Fixes vs. Re-opened Bugs, in Proceedings of the 14th International Working Conference on Source Code Analysis and Manipulation (SCAM) (Victoria, BC, Canada), p. 205-214, IEEE. (Acceptance ratio: 26/82=31.7%)
-
Mini Shridhar (*), Bram Adams and Foutse Khomh (2014). A Qualitative Analysis of Software Build System Changes and Build Ownership Styles, in Proceedings of the 8th International Symposium on Empirical Software Engineering and Measurement (ESEM) (Torino, Italy), article nr. 29. (Acceptance ratio: 23/123=19%)
