Publications    

   

2024

1. Incivility detection in open source code review and issue discussions Isabella Ferreira, Ahlaam Rafiq, and Jinghui Cheng Journal of Systems and Software [Abstract] [arXiv]

   Given the democratic nature of open source development, code review and issue discussions may be uncivil. Incivility, defined as features of discussion that convey an unnecessarily disrespectful tone, can have negative consequences to open source communities. To prevent or minimize these negative consequences, open source platforms have included mechanisms for removing uncivil language from the discussions. However, such approaches require manual inspection, which can be overwhelming given the large number of discussions. To help open source communities deal with this problem, in this paper, we aim to compare six classical machine learning models with BERT to detect incivility in open source code review and issue discussions. Furthermore, we assess if adding contextual information in the previous email/comment improves the models’ performance and how well the models perform in a cross-platform setting. We found that BERT performs better than classical machine learning models, with a best F1-score of 0.95. Furthermore, classical machine learning models tend to underperform to detect tone-bearing and civil discussions. Our results show that adding the previous email/comment to BERT did not improve its performance and that none of the analyzed classifiers had an outstanding performance in a cross-platform setting. Finally, we provide insights into the tones that the classifiers misclassify and lessons learned for using automated techniques in incivility detection. Editor’s note: Open Science material was validated by the Journal of Systems and Software Open Science Board.

2023

1. SUMMIT: Scaffolding Open Source Software Issue Discussion Through Summarization Saskia Gilmer, Avinash Bhat, Shuvam Shah, Kevin Cherry, Jinghui Cheng, and Jin L.C. Guo Proc. ACM Hum.-Comput. Interact. [Abstract] [arXiv]

   Issue Tracking Systems (ITS) often support commenting on software issues, which creates a space for discussions centered around bug fixes and improvements to the software. For Open Source Software (OSS) projects, issue discussions serve as a crucial collaboration mechanism for diverse stakeholders. However, these discussions can become lengthy and entangled, making it hard to find relevant information and make further contributions. In this work, we study the use of summarization to aid users in collaboratively making sense of OSS issue discussion threads. Through an empirical investigation, we reveal a complex picture of how summarization is used by issue users in practice as a strategy to help develop and manage their discussions. Grounded on the different objectives served by the summaries and the outcome of our formative study with OSS stakeholders, we identified a set of guidelines to inform the design of collaborative summarization tools for OSS issue discussions. We then developed SUMMIT, a tool that allows issue users to collectively construct summaries of different types of information discussed, as well as a set of comments representing continuous conversations within the thread. To alleviate the manual effort involved, SUMMIT uses techniques that automatically detect information types and summarize texts to facilitate the generation of these summaries. A lab user study indicates that, as the users of SUMMIT, OSS stakeholders adopted different strategies to acquire information on issue threads. Furthermore, different features of SUMMIT effectively lowered the perceived difficulty of locating information from issue threads and enabled the users to prioritize their effort. Overall, our findings demonstrated the potential of SUMMIT, and the corresponding design guidelines, in supporting users to acquire information from lengthy discussions in ITSs. Our work sheds light on key design considerations and features when exploring crowd-based and machine-learning-enabled instruments for asynchronous collaboration on complex tasks such as OSS development.

2. GUILGET: GUI Layout GEneration with Transformer Andrey Sobolevsky, Guillaume-Alexandre Bilodeau, Jinghui Cheng, and Jin L. C. Guo In Proceedings of the 36th Canadian Conference on Artificial Intelligence [Abstract] [arXiv]

   Sketching out Graphical User Interface (GUI) layout is part of the pipeline of designing a GUI and a crucial task for the success of a software application. Arranging all components inside a GUI layout manually is a time-consuming task. In order to assist designers, we developed a method named GUILGET to automatically generate GUI layouts from positional constraints represented as GUI arrangement graphs (GUI-AGs). The goal is to support the initial step of GUI design by producing realistic and diverse GUI layouts. The existing image layout generation techniques often cannot incorporate GUI design constraints. Thus, GUILGET needs to adapt existing techniques to generate GUI layouts that obey to constraints specific to GUI designs. GUILGET is based on transformers in order to capture the semantic in relationships between elements from GUI-AG. Moreover, the model learns constraints through the minimization of losses responsible for placing each component inside its parent layout, for not letting components overlap if they are inside the same parent, and for component alignment. Our experiments, which are conducted on the CLAY dataset, reveal that our model has the best understanding of relationships from GUI-AG and has the best performances in most of evaluation metrics. Therefore, our work contributes to improved GUI layout generation by proposing a novel method that effectively accounts for the constraints on GUI elements and paves the road for a more efficient GUI design pipeline.

3. Machine learning application development: practitioners’ insights Md Saidur Rahman, Foutse Khomh, Alaleh Hamidi, Jinghui Cheng, Giuliano Antoniol, and Hironori Washizaki Software Quality Journal [Abstract] [arXiv]

   Nowadays, intelligent systems and services are getting increasingly popular as they provide data-driven solutions to diverse real-world problems, thanks to recent breakthroughs in artificial intelligence (AI) and machine learning (ML). However, machine learning meets software engineering not only with promising potentials but also with some inherent challenges. Despite some recent research efforts, we still do not have a clear understanding of the challenges of developing ML-based applications and the current industry practices. Moreover, it is unclear where software engineering researchers should focus their efforts to better support ML application developers. In this paper, we report about a survey that aimed to understand the challenges and best practices of ML application development. We synthesize the results obtained from 80 practitioners (with diverse skills, experience, and application domains) into 17 findings outlining challenges and best practices for ML application development. Practitioners involved in the development of ML-based software systems can leverage the summarized best practices to improve the quality of their system. We hope that the reported challenges will inform the research community about topics that need to be investigated to improve the engineering process and the quality of ML-based applications.

4. “Finding the Magic Sauce”: Exploring Perspectives of Recruiters and Job Seekers on Recruitment Bias and Automated Tools Mitra Lashkari, and Jinghui Cheng In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems [Abstract] [arXiv] [Video]

   Automated recruitment tools are proliferating. While having the promise of improving efficiency, various risks, including bias, challenges the potential of these tools. An in-depth understanding of the perceived risk factors and needs from the perspective of both recruiters and job seekers is needed. We address this through an interview study in the high-tech industry to compare and contrast the concerns of these two roles. We found that the importance of clarifying position requirements and assessing candidates as “whole individuals” are commonly discussed by both recruiters and job seekers. In contrast, while recruiters tended to be more aware of cognitive bias and desired more tool support during interviews, job seekers voiced more desire towards a healthy candidate-company relationship. Additionally, both roles considered the uncertainty of the current technology capability and reduced human contact as concerns for using automated tools. Based on these results, we provided design implications for automated recruitment tools and related decision-support technologies.

2022

1. “Inconsistent Performance”: Understanding Concerns of Real-World Users on Smart Mobile Health Applications Through Analyzing App Reviews Banafsheh Mohajeri, and Jinghui Cheng In Adjunct Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology [Abstract] [arXiv]

   While smart mobile health apps that adapt to users’ progressive individual needs are proliferating, many of them struggle to fulfill their promises due to an inferior user experience. Understanding the concerns of real-world users related to those apps, and their smart components in particular, could help advance the app design to attract and retain users. In this paper, we target this issue through a preliminary thematic analysis of 120 user reviews of six smart health apps. We found that accuracy, customizability, and convenience of data input are primary concerns raised in real-world user reviews. Many concerns on the smart components are related to the trust issue of the users towards the apps. However, several important aspects such as privacy and fairness were rarely discussed in the reviews. Overall, our study provides insights that can inspire further investigations to support the design of smart mobile health apps.

2. Understanding and supporting the design systems practice Yassine Lamine, and Jinghui Cheng Empirical Software Engineering [Abstract] [arXiv]

   Design systems represent a user interaction design and development approach that is currently of avid interest in the industry. However, little research work has been done to synthesize knowledge related to design systems in order to inform the design of tools to support their creation, maintenance, and usage practices. This paper represents an important step in which we explored the issues that design system projects usually deal with and the perceptions and values of design system project leaders. Through this exploration, we aim to investigate the needs for tools that support the design system approach. We found that the open source communities around design systems focused on discussing issues related to behaviors of user interface components of design systems. At the same time, leaders of design system projects faced considerable challenges when evolving their design systems to make them both capable of capturing stable design knowledge and flexible to the needs of the various concrete products. They valued a bottom-up approach for design system creation and maintenance, in which components are elevated and merged from the evolving products. Our findings synthesize the knowledge and lay foundations for designing techniques and tools aimed at supporting the design system practice and related modern user interaction design and development approaches.

3. Characterizing User Behaviors in Open-Source Software User Forums: An Empirical Study Jazlyn Hellman, Jiahao Chen, Md. Sami Uddin, Jinghui Cheng, and Jin L. C. Guo In Proceedings of the 15th International Conference on Cooperative and Human Aspects of Software Engineering [Abstract] [arXiv] [Video]

   User forums of Open Source Software (OSS) enable end-users to collaboratively discuss problems concerning the OSS applications. Despite decades of research on OSS, we know very little about how end-users engage with OSS communities on these forums, in particular, the challenges that hinder their continuous and meaningful participation in the OSS community. Many previous works are developer-centric and overlook the importance of end-user forums. As a result, end-users’ expectations are seldom reflected in OSS development. To better understand user behaviors in OSS user forums, we carried out an empirical study analyzing about 1.3 million posts from user forums of four popular OSS applications: Zotero, Audacity, VLC, and RStudio. Through analyzing the contribution patterns of three common user types (end-users, developers, and organizers), we observed that end-users not only initiated most of the threads (above 96% of threads in three projects, 86% in the other), but also acted as the significant contributors for responding to other users’ posts, even though they tended to lack confidence in their activities as indicated by psycho-linguistic analyses. Moreover, we found end-users more open, reflecting a more positive emotion in communication than organizers and developers in the forums. Our work contributes new knowledge about end-users’ activities and behaviors in OSS user forums that the vital OSS stakeholders can leverage to improve end-user engagement in the OSS development process.

4. GANSpiration: Balancing Targeted and Serendipitous Inspiration in User Interface Design with Style-Based Generative Adversarial Network Mohammad Amin Mozaffari, Xinyuan Zhang, Jinghui Cheng, and Jin L.C. Guo In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems [Abstract] [arXiv] [Video]

   Inspiration from design examples plays a crucial role in the creative process of user interface design. However, current tools and techniques that support inspiration usually only focus on example browsing with limited user control or similarity-based example retrieval, leading to undesirable design outcomes such as focus drift and design fixation. To address these issues, we propose the GANSpiration approach that suggests design examples for both targeted and serendipitous inspiration, leveraging a style-based Generative Adversarial Network. A quantitative evaluation revealed that the outputs of GANSpiration-based example suggestion approaches are relevant to the input design, and at the same time include diverse instances. A user study with professional UI/UX practitioners showed that the examples suggested by our approach serve as viable sources of inspiration for overall design concepts and specific design elements. Overall, our work paves the road of using advanced generative machine learning techniques in supporting the creative design practice.

5. Understanding the Characteristics of Visual Contents in Open Source Issue Discussions: A Case Study of Jupyter Notebook Vishakha Agrawal, Yong-Han Lin, and Jinghui Cheng In Proceedings of the International Conference on Evaluation and Assessment in Software Engineering 2022 [Abstract] [arXiv] [Video]

   Most issue tracking systems for open source software (OSS) development include features for community members to embed visual contents, such as images and videos, to enhance the discussion. Although playing an important role, there is little knowledge on the characteristics of the visual contents to support their use. To address this gap, we conducted an empirical study on the Jupyter Notebook project. We found that more than a quarter of the issues in the Jupyter Notebook project included visual contents. Additionally, issues that had visual contents in the comments but not in the issue posts tended to have a longer discussion and to involve a larger number of discussants. In our qualitative analysis, we identified eight types of visual contents in our sample, with about 60% including screenshots or mockups of the main product. We also found that visual contents served diverse purposes, touching both problem and solution spaces of the issues. Our effort serves as an important step towards a comprehensive understanding of the characteristics of visual contents in OSS issue discussions. Our results provided several design implications for issue tracking systems to better facilitate the use of visual contents.

6. How Heated is It? Understanding GitHub Locked Issues Isabella Ferreira, Bram Adams, and Jinghui Cheng In Proceedings of the 19th International Conference on Mining Software Repositories [Abstract] [arXiv] [Video]

   Although issues of open source software are created to discuss and solve technical problems, conversations can become heated, with discussants getting angry and/or agitated for a variety of reasons, such as poor suggestions or violation of community conventions. To prevent and mitigate discussions from getting heated, tools like GitHub have introduced the ability to lock issue discussions that violate the code of conduct or other community guidelines. Despite some early research on locked issues, there is a lack of understanding of how communities use this feature and of potential threats to validity for researchers relying on a dataset of locked issues as an oracle for heated discussions. To address this gap, we (i) quantitatively analyzed 79 GitHub projects that have at least one issue locked as too heated, and (ii) qualitatively analyzed all issues locked as too heated of the 79 projects, a total of 205 issues comprising 5,511 comments. We found that projects have different behaviors when locking issues: while 54 locked less than 10% of their closed issues, 14 projects locked more than 90% of their closed issues. Additionally, locked issues tend to have a similar number of comments, participants, and emoji reactions to non-locked issues. For the 205 issues locked as too heated, we found that one-third do not contain any uncivil discourse, and only 8.82% of the analyzed comments are actually uncivil. Finally, we found that the locking justifications provided by maintainers do not always match the label used to lock the issue. Based on our results, we identified three pitfalls to avoid when using the GitHub locked issues data and we provide recommendations for researchers and practitioners.

2021

1. The "Shut the F**k up" Phenomenon: Characterizing Incivility in Open Source Code Review Discussions Isabella Ferreira, Jinghui Cheng, and Bram Adams Proc. ACM Hum.-Comput. Interact. [Abstract] [arXiv] [Video] [Blog]

   Code review is an important quality assurance activity for software development. Code review discussions among developers and maintainers can be heated and sometimes involve personal attacks and unnecessary disrespectful comments, demonstrating, therefore, incivility. Although incivility in public discussions has received increasing attention from researchers in different domains, the knowledge about the characteristics, causes, and consequences of uncivil communication is still very limited in the context of software development, and more specifically, code review. To address this gap in the literature, we leverage the mature social construct of incivility as a lens to understand confrontational conflicts in open source code review discussions. For that, we conducted a qualitative analysis on 1,545 emails from the Linux Kernel Mailing List (LKML) that were associated with rejected changes. We found that more than half (66.66%) of the non-technical emails included uncivil features. Particularly, frustration, name calling, and impatience are the most frequent features in uncivil emails. We also found that there are civil alternatives to address arguments, while uncivil comments can potentially be made by any people when discussing any topic. Finally, we identified various causes and consequences of such uncivil communication. Our work serves as the first study about the phenomenon of in(civility) in open source software development, paving the road for a new field of research about collaboration and communication in the context of software engineering activities.

2. Facilitating Asynchronous Participatory Design of Open Source Software: Bringing End Users into the Loop Jazlyn Hellman, Jinghui Cheng, and Jin L.C. Guo In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems [Abstract] [arXiv] [Video]

   As open source software (OSS) becomes increasingly mature and popular, there are significant challenges with properly accounting for usability concerns for the diverse end users. Participatory design, where multiple stakeholders collaborate on iterating the design, can be an efficient way to address the usability concerns for OSS projects. However, barriers such as a code-centric mindset and insufficient tool support often prevent OSS teams from effectively including end users in participatory design methods. This paper proposes preliminary contributions to this problem through the user-centered exploration of (1) a set of design guidelines that capture the needs of OSS participatory design tools, (2) two personas that represent the characteristics of OSS designers and end users, and (3) a low-fidelity prototype tool for end user involvement in OSS projects. This work paves the road for future studies about tool design that would eventually help improve OSS usability.

3. The Impacts of Sentiments and Tones in Community-Generated Issue Discussions Arghavan Sanei, Jinghui Cheng, and Bram Adams In 2021 IEEE/ACM 13th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE) [Abstract] [arXiv] [Video]

   The diverse community members who contribute to the discussions on issue tracking systems of open-source software projects often exhibit complex affective states such as sentiments and tones. These affective states can significantly influence the effectiveness of the issue discussions in elaborating the initial ideas into actionable tasks that the development teams need to address. In this paper, we present an extended empirical study to investigate the impacts of sentiments and tones in community-generated issue discussions. We created and validated a large dataset of sentiments and tones in the issues posts and comments created by diverse community members in three popular open source projects. Our analysis results drew a complex picture of the relationships between, on the one hand, the sentiments and tones in the issue discussions, and on the other hand, various discussion and development-related measures such as the discussion length and the issue resolution time. We also found that when factors such as the issue poster roles and the issue types were controlled, sentiments and tones had varied associations with the measures. Insights gained from these findings can support open source community members in making and moderating effective issue discussions and guide the design of tools to better support community engagement.

2020

1. Adapting Usability Heuristics to the Context of Mobile Augmented Reality Audrey Labrie, and Jinghui Cheng In Adjunct Publication of the 33rd Annual ACM Symposium on User Interface Software and Technology [Abstract] [arXiv]

   Augmented reality (AR) is an emerging technology in mobile app design during recent years. However, usability challenges in these apps are prominent. There are currently no established guidelines for designing and evaluating interactions in AR as there are in traditional user interfaces. In this work, we aimed to examine the usability of current mobile AR applications and interpreting classic usability heuristics in the context of mobile AR. Particularly, we focused on AR home design apps because of their popularity and ability to incorporate important mobile AR interaction schemas. Our findings indicated that it is important for the designers to consider the unfamiliarity of AR technology to the vast users and to take technological limitations into consideration when designing mobile AR apps. Our work serves as a first step for establishing more general heuristics and guidelines for mobile AR.

2. ArguLens: Anatomy of Community Opinions On Usability Issues Using Argumentation Models Wenting Wang, Deeksha Arya, Nicole Novielli, Jinghui Cheng, and Jin L.C. Guo In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems [Abstract] [arXiv] [Video] [Blog]

   In open-source software (OSS), the design of usability is often influenced by the discussions among community members on platforms such as issue tracking systems (ITSs). However, digesting the rich information embedded in issue discussions can be a major challenge due to the vast number and diversity of the comments. We propose and evaluate ArguLens, a conceptual framework and automated technique leveraging an argumentation model to support effective understanding and consolidation of community opinions in ITSs. Through content analysis, we anatomized highly discussed usability issues from a large, active OSS project, into their argumentation components and standpoints. We then experimented with supervised machine learning techniques for automated argument extraction. Finally, through a study with experienced ITS users, we show that the information provided by ArguLens supported the digestion of usability-related opinions and facilitated the review of lengthy issues. ArguLens provides the direction of designing valuable tools for high-level reasoning and effective discussion about usability.

3. Capturing the Practices, Challenges, and Needs of Transportation Decision-Makers Nasim Sharbatdar, Yassine Lamine, Brigitte Milord, Catherine Morency, and Jinghui Cheng In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems [Abstract] [arXiv]

   Transportation decision-makers from government agencies play an important role in addressing the traffic network conditions, which in turn, have a major impact on the well-being of citizens. The practices, challenges, and needs of this group of practitioners are less represented in the HCI literature. We address this gap through an interview study with 19 practitioners from Transports Québec, a government agency responsible for transportation infrastructures in Québec, Canada. We found that this group of decision-makers can most benefit from research about data analysis tools and platforms that (1) provide information to support data quality awareness, (2) are interoperable with other tools in the complex workflow of the practitioners, and (3) support intuitive and customizable visual analytics. These implications can also be informative to the design of tools supporting other decision-making tasks and domains.

4. How Do Open Source Software Contributors Perceive and Address Usability? Valued Factors, Practices, and Challenges W. Wang, J. Cheng, and J. L. C. Guo IEEE Software [Abstract] [arXiv]

   Usability is an increasing concern in open source software (OSS). Given the recent changes in the OSS landscape, it is imperative to examine the OSS contributors’ current valued factors, practices, and challenges concerning usability. We accumulated this knowledge through a survey with a wide range of contributors to OSS applications. Through analyzing 84 survey responses, we found that many participants recognized the importance of usability. While most relied on issue tracking systems to collect user feedback, a few participants also adopted typical user-centered design methods. However, most participants demonstrated a system-centric rather than a user-centric view. Understanding the diverse needs and consolidating various feedback of end-users posed unique challenges for the OSS contributors when addressing usability in the most recent development context. Our work provided important insights for OSS practitioners and tool designers in exploring ways for promoting a user-centric mindset and improving usability practice in the current OSS communities.

2019

1. Usability of Virtual Reality Application Through the Lens of the User Community: A Case Study Wenting Wang, Jinghui Cheng, and Jin L.C. Guo In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems [Abstract] [arXiv]

   The increasing availability and diversity of virtual reality (VR) applications highlighted the importance of their usability. Function-oriented VR applications posed new challenges that are not well studied in the literature. Moreover, user feedback becomes readily available thanks to modern software engineering tools, such as app stores and open source platforms. Using Firefox Reality as a case study, we explored the major types of VR usability issues raised in these platforms. We found that 77% of usability feedbacks can be mapped to Nielsen’s heuristics while few were mappable to VR-specific heuristics. This result indicates that Nielsen’s heuristics could potentially help developers address the usability of this VR application in its early development stage. This work paves the road for exploring tools leveraging the community effort to promote the usability of function-oriented VR applications.

2. Activity-based Analysis of Open Source Software Contributors: Roles and Dynamics Jinghui Cheng, and Jin L. C. Guo In Proceedings of the 12th International Workshop on Cooperative and Human Aspects of Software Engineering [Abstract] [arXiv]

   Contributors to open source software (OSS) communities assume diverse roles to take different responsibilities. One major limitation of the current OSS tools and platforms is that they provide a uniform user interface regardless of the activities performed by the various types of contributors. This paper serves as a non-trivial first step towards resolving this challenge by demonstrating a methodology and establishing knowledge to understand how the contributors’ roles and their dynamics, reflected in the activities contributors perform, are exhibited in OSS communities. Based on an analysis of user action data from 29 GitHub projects, we extracted six activities that distinguished four Active roles and five Supporting roles of OSS contributors, as well as patterns in role changes. Through the lens of the Activity Theory, these findings provided rich design guidelines for OSS tools to support diverse contributor roles.

3. Interlocking Safety Cases for Unmanned Autonomous Systems in Shared Airspaces M. Vierhauser, S. Bayley, J. Wyngaard, W. Xiong, J. Cheng, J. Huseman, R. R. Lutz, and J. Cleland-Huang IEEE Transactions on Software Engineering [Abstract]

   The growing adoption of unmanned aerial vehicles (UAV) for tasks such as eCommerce, aerial surveillance, and environmental monitoring introduces the need for new safety mechanisms in an increasingly cluttered airspace. In our work we thus emphasize safety issues that emerge at the intersection of infrastructures responsible for controlling the airspace, and the diverse UAVs operating in their space. We build on safety assurance cases (SAC)- a state-of-the-art solution for reasoning about safety - and propose a novel approach based on interlocking SACs. The infrastructure safety case (ISAC) specifies assumptions upon UAV behavior, while each UAV demonstrates compliance to the ISAC by presenting its own (pluggable) safety case (pSAC) which connects to the ISAC through a set of interlock points. To collect information on each UAV we enforce a "trust but monitor" policy, supported by runtime monitoring and an underlying reputation model. We evaluate our approach in three ways: first by developing ISACs for two UAV infrastructures, second by running simulations to evaluate end-to-end effectiveness, and finally via an outdoor field-study with physical UAVs. The results show that interlocking SACs can be effective for identifying, specifying, and monitoring safety-related constraints upon UAVs flying in a controlled airspace.

4. Analysis and Detection of Information Types of Open Source Software Issue Discussions Deeksha Arya, Wenting Wang, Jin L. C. Guo, and Jinghui Cheng In Proceedings of the 41st International Conference on Software Engineering [Abstract] [arXiv]

   Most modern Issue Tracking Systems (ITSs) for open source software (OSS) projects allow users to add comments to issues. Over time, these comments accumulate into discussion threads embedded with rich information about the software project, which can potentially satisfy the diverse needs of OSS stakeholders. However, discovering and retrieving relevant information from the discussion threads is a challenging task, especially when the discussions are lengthy and the number of issues in ITSs are vast. In this paper, we address this challenge by identifying the information types presented in OSS issue discussions. Through qualitative content analysis of 15 complex issue threads across three projects hosted on GitHub, we uncovered 16 information types and created a labeled corpus containing 4656 sentences. Our investigation of supervised, automated classification techniques indicated that, when prior knowledge about the issue is available, Random Forest can effectively detect most sentence types using conversational features such as the sentence length and its position. When classifying sentences from new issues, Logistic Regression can yield satisfactory performance using textual features for certain information types, while falling short on others. Our work represents a nontrivial first step towards tools and techniques for identifying and obtaining the rich information recorded in the ITSs to support various software engineering activities and to satisfy the diverse needs of OSS stakeholders.

2018

1. Interlocking Safety Cases for Unmanned Autonomous Systems in Urban Environments Michael Vierhauser, Sean Bayley, Jane Wyngaard, Jinghui Cheng, Wandi Xiong, Robyn Lutz, Joshua Huseman, and Jane Cleland-Huang In Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings
2. How Do Practitioners Perceive Assurance Cases in Safety-critical Software Systems? Jinghui Cheng, Micayla Goodrum, Ronald Metoyer, and Jane Cleland-Huang In Proceedings of the 11th International Workshop on Cooperative and Human Aspects of Software Engineering [Abstract] [arXiv]

   Safety-critical software systems are those whose failure or malfunction could result in casualty and/or serious financial loss. In such systems, safety assurance cases (SACs) are an emerging approach that adopts a proactive strategy to produce structuralized safety justifications and arguments. While SACs are recommended in many software-intensive safety-critical domains, the lack of knowledge regarding the practitioners’ perspectives on using SACs hinders effective adoption of this approach. To gain such knowledge, we interviewed nine practitioners and safety experts who focused on safety-critical software systems. In general, our participants found the SAC approach beneficial for communication of safety arguments and management of safety issues in a multidisciplinary setting. The challenges they faced when using SACs were primarily associated with (1) a lack of tool support, (2) insufficient process integration, and (3) scarcity of experienced personnel. To overcome those challenges, our participants suggested tactics that focused on creating direct safety arguments. Process and organizational adjustments are also needed to streamline SAC analysis and creation. Finally, our participants emphasized the importance of knowledge sharing about SACs across software-intensive safety-critical domains.

3. How Do the Open Source Communities Address Usability and UX Issues?: An Exploratory Study Jinghui Cheng, and Jin L.C. Guo In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems [Abstract]

   Usability and user experience (UX) issues are often not well emphasized and addressed in open source software (OSS) development. There is an imperative need for supporting OSS communities to collaboratively identify, understand, and fix UX design issues in a distributed environment. In this paper, we provide an initial step towards this effort and report on an exploratory study that investigated how the OSS communities currently reported, discussed, negotiated, and eventually addressed usability and UX issues. We conducted in-depth qualitative analysis of selected issue tracking threads from three OSS projects hosted on GitHub. Our findings indicated that discussions about usability and UX issues in OSS communities were largely influenced by the personal opinions and experiences of the participants. Moreover, the characteristics of the community may have greatly affected the focus of such discussion.

4. Software Engineering for Machine-Learning Applications: The Road Ahead Foutse Khomh, Bram Adams, Jinghui Cheng, Marios Fokaefs, and Giuliano Antoniol IEEE Software [Abstract]

   The First Symposium on Software Engineering for Machine Learning Applications (SEMLA) aimed to create a space in which machine learning (ML) and software engineering (SE) experts could come together to discuss challenges, new insights, and practical ideas regarding the engineering of ML and AI-based systems. Key challenges discussed included the accuracy of systems built using ML and AI models, the testing of those systems, industrial applications of AI, and the rift between the ML and SE communities. This article is part of a theme issue on software engineering’s 50th anniversary.

2017

1. Leveraging Design Patterns to Support Designer-Therapist Collaboration When Ideating Brain Injury Therapy Games Jinghui Cheng, Dorian Anderson, Cynthia Putnam, and Jin Guo In Proceedings of the Annual Symposium on Computer-Human Interaction in Play [Abstract]

   Many therapists who work with patients who have had a brain injury (BI) include games to ameliorate boredom associated with repetitive rehabilitation exercises. However, designing effective, appropriate, and engaging games for BI therapy is challenging. Following a user-centered approach we created GaPBIT (Game Design Patterns for BI Therapy), a prototype tool that leveraged design patterns to support designer-therapist collaboration when ideating games for BI therapy. We observed the use of GaPBIT in six game ideation workshops that involved game designers and therapists. The tool effectively facilitated collaboration in the interdisciplinary teams. Findings also suggested that information tools like GaPBIT support but do not replace informative collaboration among designers and subject-matter experts (i.e., therapists in our study). We argue that our findings and research methodology generalize to other domains where communication and collaboration among interdisciplinary design teams are imperative for designing "successful" games; i.e. games that meet the varied goals among stakeholders.

2. Effects of Commercial Exergames on Motivation in Brian Injury Therapy Cynthia Putnam, Amanda Lin, Vansanth Subramanian, Dorian C. Anderson, Erica Christian, Bharathi Swaminathan, Sai Yalla, William Cotter, Danielle Ciccone, and Jinghui Cheng In Extended Abstracts Publication of the 2017 Annual Symposium on Computer-Human Interaction in Play - CHI PLAY ’17 Extended Abstracts
3. Cognitive Rehabilitation Potential of a Driving Simulation Game for BrainInjury Cynthia Putnam, Dorian C. Anderson, Wayne Hosley, Jinghui Cheng, and Lisa Goldman In Extended Abstracts Publication of the 2017 Annual Symposium on Computer-Human Interaction in Play - CHI PLAY ’17 Extended Abstracts
4. What Requirements Knowledge Do Developers Need to Manage Change in Safety-Critical Systems? Micayla Goodrum, Jane Cleland-Huang, Robyn Lutz, Jinghui Cheng, and Ronald Metoyer In 2017 IEEE 25th International Requirements Engineering Conference (RE) [Abstract]

   Developers maintaining safety-critical systems need to assess the impact a proposed change would have upon existing safety controls. By leveraging the network of traceability links that are present in most safety-critical systems, we can push timely information about related hazards, environmental assumptions, and safety requirements to developers. In this work we take a design science approach to discover the informational needs of developers as they engage in software maintenance activities and then propose and evaluate techniques for presenting and visualizing this information. Through a human-centered study involving five safety-critical system practitioners and 14 experienced developers, we analyze the way in which developers use requirements knowledge while maintaining safety-critical code, identify their informational needs, and propose and evaluate a supporting visualization technique. The insights proposed as a result of this study can be used to design requirements-based knowledge tools for supporting developers’ maintenance tasks.

5. Supporting Therapy-Centered Game Design for Brain Injury Rehabilitation Jinghui Cheng In College of Computing and Digital Media Dissertation
6. Semantically Enhanced Software Traceability Using Deep Learning Techniques Jin Guo, Jinghui Cheng, and Jane Cleland-Huang In 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE) [Abstract]

   In most safety-critical domains the need for traceability is prescribed by certifying bodies. Trace links are generally created among requirements, design, source code, test cases and other artifacts, however, creating such links manually is time consuming and error prone. Automated solutions use information retrieval and machine learning techniques to generate trace links, however, current techniques fail to understand semantics of the software artifacts or to integrate domain knowledge into the tracing process and therefore tend to deliver imprecise and inaccurate results. In this paper, we present a solution that uses deep learning to incorporate requirements artifact semantics and domain knowledge into the tracing solution. We propose a tracing network architecture that utilizes Word Embedding and Recurrent Neural Network (RNN) models to generate trace links. Word embedding learns word vectors that represent knowledge of the domain corpus and RNN uses these word vectors to learn the sentence semantics of requirements artifacts. We trained 360 different configurations of the tracing network using existing trace links in the Positive Train Control domain and identified the Bidirectional Gated Recurrent Unit (BI-GRU) as the best model for the tracing task. BI-GRU significantly out-performed state-of-the-art tracing methods including the Vector Space Model and Latent Semantic Indexing.

7. Towards a Prototype Tool Leveraging Design Patterns to Support Design of Games for Brain Injury Therapy Jinghui Cheng, and Cynthia Putnam In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA ’17

2016

1. Best Practices for Teaching Accessibility in University Classrooms: Cultivating Awareness, Understanding, and Appreciation for Diverse Users Cynthia Putnam, Maria Dahman, Emma Rose, Jinghui Cheng, and Glenn Bradford ACM Transactions on Accessible Computing
2. You Are Not the Player: Teaching Games User Research to Undergraduate Students Cynthia Putnam, José Zagal, and Jinghui Cheng In Games User Research: A Case Study Approach
3. Using the Kano Model to Balance Delight and Frustration for an Enterprise Application Jinghui Cheng, James Mulholland, and Anil Shankar In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA ’16
4. ’Choose a Game’: Creation and Evaluation of a Prototype Tool to Support Therapists in Brain Injury Rehabilitation Cynthia Putnam, Jinghui Cheng, Feng Lin, Sai Yalla, and Stephanie Wu In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems - CHI ’16
5. "Always a Tall Order": Values and Practices of Professional Game Designers of Serious Games for Health Jinghui Cheng, Cynthia Putnam, and Jin Guo In Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play - CHI PLAY ’16
6. Human-Centered Design in Practice: Roles, Definitions, and Communication Cynthia Putnam, Aaron Reiner, Emily Ryou, Morgan Caputo, Jinghui Cheng, Mace Allen, and Ravali Singamaneni Journal of Technical Writing and Communication
7. ’Choose a Game’: A Prototype Tool to Support Therapists Use Games in Brain Injury Rehabilitation Jinghui Cheng, and Cynthia Putnam In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA ’16

2015

1. Teaching Accessibility, Learning Empathy Cynthia Putnam, Maria Dahman, Emma Rose, Jinghui Cheng, and Glenn Bradford In Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility
2. Towards Efficacy-Centered Game Design Patterns for Brain Injury Rehabilitation: A Data-Driven Approach Jinghui Cheng, Cynthia Putnam, and Doris C Rusch In Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility
3. Therapeutic Gaming in Context: Observing Game Use for Brain Injury Rehabilitation Jinghui Cheng, and Cynthia Putnam In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA ’15

2014

1. Therapist-centered requirements: A multi-method approach of requirement gathering to support rehabilitation gaming Cynthia Putnam, and Jinghui Cheng In Proceedings of the IEEE 22nd International Requirements Engineering Conference (RE 2014)
2. Therapist Perspectives: Wii Active Videogames Use in Inpatient Settings with People Who Have Had a Brain Injury Cynthia Putnam, Jinghui Cheng, and Gregory Seymour Games for Health Journal

2013

1. Motion-games in brain injury rehabilitation: an in-situ multi-method study of inpatient care Cynthia Putnam, and Jinghui Cheng In Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’13)
2. Supporting therapists in motion-based gaming for brain injury rehabilitation Cynthia Putnam, Jinghui Cheng, Doris Rusch, André Berthiaume, and Robin Burke In CHI ’13 Extended Abstracts on Human Factors in Computing Systems (CHI EA ’13)
3. Helping therapists make evidence-based decisions about commercial motion gaming Cynthia Putnam, and Jinghui Cheng ACM SIGACCESS Accessibility and Computing
4. Helping Therapists Make Evidence-based Decisions About Commercial Motion Gaming Cynthia Putnam, and Jinghui Cheng SIGACCESS Access. Comput.

2012

1. How do professionals who create computing technologies consider accessibility? Cynthia Putnam, Kathryn Wozniak, Mary Jo Zefeldt, Jinghui Cheng, Morgan Caputo, and Carl Duffield In Proceedings of the 14th international ACM SIGACCESS conference on Computers and accessibility (ASSETS ’12)

2010

1. Fast Identification of Inactive Security Constraints in SCUC Problems Qiaozhu Zhai, Xiaohong Guan, Jinghui Cheng, and Hongyu Wu IEEE Transactions on Power Systems [Abstract]

   Security constrained unit commitment (SCUC) is one of the most important daily tasks that independent system operators (ISOs) or regional transmission organizations (RTOs) must accomplish in daily electric power market. Security constraints have long been regarded as difficult constraints for unit commitment problems. If the inactive security constraints can be identified and eliminated, the SCUC problem can be greatly simplified. In this paper, a necessary and sufficient condition for a security constraint to be inactive is established. It is proved that all inactive constraints can be identified by solving a series of small-scale mixed integer linear programming (MILP) problems. More importantly, an analytical sufficient condition is established and most of the inactive constraints can be quickly identified without solving MILP or linear programming (LP) problems. A very important feature of the conditions obtained is that they are only related to the load demands and parameters of the transmission network. Numerical testing is performed for three power grids and the results are impressive. Over 85% of the security constraints are identified as inactive and the crucial transmission lines affecting the total operating cost are among those associated with the remaining security constraints, providing useful information for transmission planning.