ABSTRACT. We study whether and in which phase Test-Driven Development (TDD) influences affective states of novice developers in terms of pleasure, arousal, dominance, and liking. We performed a controlled experiment with 29 novice developers. Developers in the treatment group performed a development task using TDD, whereas those in the control group used a non-TDD development approach. We compared the affective reactions to the development approaches, as well as to the implementation and testing phases, exploiting a lightweight, powerful, and widely used tool, i.e., Self-Assessment Manikin. We observed that there is a difference between the two development approaches in terms of affective reactions. Therefore, it seems that affective reactions play an important role when applying TDD and their investigation could help researchers to better understand such a development approach.

ABSTRACT. Despite the multitude of available software testing tools, literature lists lack of right tools and costs as problems for adopting a tool. We conducted a case study to analyze how a group of practitioners, familiar with Robot Framework (an open source, generic test automation framework), evaluate the tool. We based the case and the unit of analysis on our academia-industry relations, i.e., availability. We used a survey (n=68) and interviews (n=6) with convenience sampling to develop a comprehensive view of the phenomena. The study reveals the importance of understanding the interconnection of different criteria and the potency of the context on those. Our results show that uncofirmed or unfocused opinions about criteria, e.g., about Costs or Programming Skills, can lead to misinterpretations or hamper strategic decisions if overlooking required technical competence. We conclude surveys can serve as a useful instrument for collecting empirical knowledge about tool evaluation, but experiential reasoning collected with a complementary method is required to develop into comprehensive understanding about it.

ABSTRACT. Background: Test-case quality has always been one of the major concerns in software testing. To improve test-case quality, it is important to better understand how practitioners perceive the quality of test-cases. Objective: Motivated by that need, we investigated how practitioners define test-case quality and which aspects of test-cases are important for quality assessment. Method: We conducted semi-structured interviews with professional testers, developers, and test architects from a multinational software company in Sweden. Before the interviews, we asked participants for actual test cases (written in natural language) that they perceive as good, normal, and bad respectively together with rationales for their assessment. We also compared their opinions on shared test cases and contrasted their views with the relevant literature. Results: We present a quality model which consists of 14 test-case quality attributes. We also identify a misalignment in defining test-case quality among practitioners and between academia and industry, along with suggestions for improving test-case quality in industry. Conclusion: The results show that practitioners' background, including roles and working experience, are critical dimensions of how test-case quality is defined and assessed.

ABSTRACT. Context. Software testing and test automation are important activities in software development where frequent requirements changes and the fast delivery of soft-ware increments are supported by traditional and agile development processes. Test reports are often used as “proof of evidence” for executed software tests. However, the practical impact of test reports, such as decision making and quality assessment, requires structured information which might not be available in suffi-cient quality. Goal. In this paper we (a) report on needs of test reports of different stakeholders at a large-scale Austrian logistics organization, (b) develop candidate improvement actions based on the state of the practice, and (c) conceptually eval-uate selected improvement actions. Method. We used surveys and interviews to elicit needs and expected capabilities for test reporting and developed candidate improvement. We used expert discussions prioritize improvement actions in the organization context for further implementation. Results. Based on 25 recom-mended improvement actions, 21 were initially selected for implementation. Most of these accepted improvement action focus on regular test status reports and vis-ualization aspects of test reports. Conclusion. Although test reporting is systemat-ically applied in development processes, there is still some potential to improve test reports to gain (additional) benefits for related stakeholder.

ABSTRACT. With the increasing popularity of A/B testing and other experimentation practices in web systems, companies from a range of different domains are starting to look at continuous experimentation as way to guide product development and feature prioritization. Research in continuous experimentation traditionally focused on companies that have easy access to user data and that have a high degree of control of the product roadmap. However, little research has been conducted to under- stand how companies that have a low control of roadmap and have a large distance to the users, such as consultancy companies, can benefit from continuous experimentation practices. To address this problem, we per- formed an exploratory case study with a software consultancy company combined with a validation procedure with four additional companies. The contribution of this work is three-fold. First, we devised a model to classify a company in the terms of the distance to users and the control of roadmap. Second, we investigate how control of roadmap and distance to user impacts continuous experimentation. Finally, we present several challenges and benefits of continuous experimentation for companies and directions for future work.

ABSTRACT. While user-centered design (UCD) processes have widely been established in domains like end-user electronics and business-toconsumer products, such processes still lack widespread adaptation for the development of industrial human-machine interfaces (HMIs). Over a period of more than two years, we have worked as part of a development team at a company from the manufacturing domain in a pilot project to introduce a UCD process. During this period, we have - via participant observation - collected a set of observed practices and behaviors that violate well-known UCD principles. Furthermore, we derived some root causes of these violations. Our insights are that introducing a UCD processes cannot be performed isolated for a single development team but impacts the entire organization including management and requires trust as well as changes with regard to mindset, methods, technologies, and team organization.

ABSTRACT. Aligning IT strategies to business goals is a top priority for CIOs. However, measuring results that IT brings to business is a challenging task. We carried out a study to help an IT director of a large mining company to define OKRs (Objective Key Results) and quantitatively monitor the achievement of goals. We performed a participative case study to define OKRs for goals and initiatives to achieve them, by using GQM+Strategies to support us in that matter. After three meetings with the IT director and IT managers, we defined OKRs for five IT goals and initiatives to achieve them. GQM+Strategies and OKR can be used together, working in a complimentary way: OKR gives simplicity and agility to the process, while GQM+Strategies provides useful knowledge to define OKRs and initiatives to achieve them properly.

ABSTRACT. Software development has been impacted by the arrival of agile frameworks, especially in the last two decades. The HELENA Project (Hybrid dEveLop-mENt Approaches in software systems development) was developed to iden-tify the use of these frameworks in relation to more traditional ones. As part of this project, a survey was carried out in 55 countries, including Spain and Costa Rica. This paper presents the comparison of the results of these two countries, particularly in relation to two topics: the degree of agility of the ac-tivities of the software development life cycle and what are the most used methods and frameworks in each country. The results show similarities in both topics for the two countries, such as the fact that the most agile-oriented activities are Implementation/Coding and Integration/Testing, or the wide-spread use of agile frameworks with Scrum in the first place, followed by It-erative Development and Kanban. There are, however, some differences, such as a greater presence in Spain of scaling agile frameworks.

ABSTRACT. In this paper, we present an approach to migrate cross-platform apps toward a native platform (i.e., Android). The approach is tailored to the Ionic cross-platform solution, i.e., an open-source framework providing a mobile UI (User Interface) toolkit for developing high-quality cross-platform apps. The validity of our approach has been validated on an open-source app developed by means of Ionic (i.e., Movies-app). In such a way, we had two versions of the same app: one developed in Ionic and the other in Android. To investigate if there is a difference in the user experience when using these two versions, we conducted a user study. This user study also aimed at studying the presence of possible differences in the affective reaction of users when using these two versions of Movies-app. The results suggest that the user experience is better when users deal with the Android app version. Similar results were achieved with respect to the affective reaction of users. To conclude, the obtained results seem to suggest that the migration from Ionic towards Android matters.

ABSTRACT. A software development process involves numerous persons, includ-ing customers, domain experts, software engineers, managers, evaluators and cer-tifiers. Together, they produce some software that satisfies its requirements and its quality criteria at a certain point in time. This software contains faults and flaws of different levels of severity and at different levels of its production (spec-ification, design, etc.) so maintenance is needed in order to correct it. Perfective and adaptive maintenance is also needed to cope with changes in the environment or with new requirements, e.g. new functionalities. In this work, we introduce the Persistent Knowledge Monitor (PKM), which is being developed within the DECODER H2020 project for handling (i.e. storing, retrieving, merging and checking for consistency) all kinds of knowledge and information related to a software project. The PKM will be part of a platform capable of taking advantage of all the artefacts available in a software ecosystem, not only the source code, but also its version control system, abstract specifications, informal documents or reports, etc. for representing the software knowledge and improving the work-flow of software developers.

ABSTRACT. [Context] Failures in management of non-functional requirements (NFRs) (e.g., incomplete or ambiguous NFRs) are frequently identified as one of the root causes of failures in the transition of software products. Recent studies confirm that using a catalog of NFR templates for requirements elicitation positively impacts quality of requirements. However, practitioners are afraid of templates as the return on investment in this technique is still unknown.

[Aim] Our aim was to investigate how the usefulness of catalog of NFR templates and its maintenance costs change over time.

[Method] Using 41 industrial projects with 2,231 NFRs, we simulated 10,000 different random evolutions of a catalog of NFR templates. It allowed us to examine the distribution of catalog value, maintenance effort, catalog utilization over a sequence of projects (a counterpart of elapsing time).

[Results] From the performed analysis it follows that after considering about 40 projects we can expect catalog value of 75% or more and maintenance effort of 10% or less. Then one could expect about 400 templates in the catalog but only about 10% of them would be used by a single project (on average).

[Conclusions] Usefulness and maintenance costs of catalog of NFR templates depend on the number of projects used to develop it. A catalog of high usefulness and low maintenance effort need to be developed from about 40 projects. Since high variability of studied projects, this number in practice might be lower. From the perspective of a large or medium software company it seems not a big problem.

ABSTRACT. In the development of electrical machines, users run a batch of software by providing text-based data exchange files. The required structure and content of these files is often only informally documented and implicitly enforced by programs. Therefore, users are forced to execute programs without prior syntactic and semantic verification. To improve the quality of data exchange files, users need editor support that allows syntactic and semantic verification using grammar-based analyzers. In order to reduce the effort for creating grammars, we use grammar recovery. Grammar recovery analyzes existing software artifacts and makes the retrieved knowledge visible as grammar of the source language. Technical aspects that are not expressed in the analyzed artifacts will be added later. The assessment and completion of the extracted grammar requires both knowledge in software-language engineering and in the application domain. This paper examines whether the integration of grammar recovery with domain-specific languages is suitable for creating semi-automated processing support for data exchange files. In particular, we are interested in whether we can recover (1) a grammar and validation rules from documentation and a corpus of example files. Furthermore, we are interested in whether (2) a domain-specific language (DSL) allows domain experts to provide missing details and and evolve grammars. To answer these questions, we conducted an industrial case study on three different types of data exchange files. The results show that about 45% of the grammar rules could be recovered automatically and that the completion of the extracted grammars by end-users is a promising means to provide correct and maintainable grammars for data exchange files.

ABSTRACT. Over the past decade, continuous software development has become a common place in the field of software engineering. Containers like Docker are a lightweight solution that developers can use to deploy and manage applications. Containers are used to build both component-based architectures and microservice architectures. Still, practitioners often view containers only as way to lower resource requirements compared to virtual machines. In this research, we conducted a systematic mapping study to find information on how containers are used in software development. 56 primary studies were selected into this paper and they were categorized and mapped to identify the gaps in the current research. Based on the results containers are most often discussed in the context of cloud computing, performance and DevOps. We find that what is currently missing is more deeply focused research.

ABSTRACT. Abstract. [Context/Background] Machine Learning (ML) software has special ability for increasing technical debt due to ML-specic issues besides having all the problems of regular code. The term \Hidden Technical Debt" (HTD) was coined by Sculley et al. to address maintainability issues in ML software as an analogy to technical debt in traditional software. [Goal] The aim of this paper is to empirically analyse how HTD patterns emerge during the early development phase of ML software, namely the prototyping phase [Method] Therefore, we conducted a case study with subject systems as ML models planned to be integrated into the software system owned by Vasttraffik, the public transportation agency in the west area of Sweden. [Results] During our case study, we could detect HTD patterns, which have the potential to emerge in ML prototypes,, except for \Legacy Features", \Correlated features", and \Plain Old Data Type Smell". [Conclusion] Preliminary results indicate that emergence of signicant amount of HTD patterns can occur during prototyping phase However, generalizability of our results require analyses of further ML systems from various domains.

ABSTRACT. Today, security is still considered to late in the process of software engineering. Architectural rules for security can support software architects and developers in consciously taking security into account during design and implementation phase. They allow to monitor a software system's security level.

As a step towards monitoring and controlling the erosion of an architectur's security specifications we present a set of rules derived from well-known security building blocks such as patterns along with our identification process. Through these rules we aim to support architects in monitoring the implementation's conformance with security measures and, hence, in building secure software systems.

The architectural security rules we identified are evaluated through expert interviews with industrial software engineers.

ABSTRACT. Background: It is important to pay attention to non-functional requirements (NFRs). To adequately attend to NFRs, they must be documented - otherwise, developers would not know about their existence. We assume that there exists a positive correlation between the level of importance of an NFR and its documentation. Aims: The goal is to explore the relationship between the level of importance and degree of documentation for NFRs. Method: Based on a subset of data acquired from the most recent NaPiRE (Naming the Pain in Requirements Engineering) survey, we calculate for a standard set of NFR types comprising Compatibility, Maintainability, Performance, Portability, Reliability, Safety, Security, and Usability how often respondents state they document a specific type of NFR when they also state that this type of NFR is important. In addition, we calculate the occurrence of potential Technical Debt and Waste. Results: Our analyses based on 398 survey responses indicate that for four NFR types (Maintainability, Reliability, Usability, and Performance), more than 22% of the 398 survey respondents who labelled those NFR types as important stated that they did not document them. We interpret this as an indication that these NFR types have a higher risk of Technical Debt than other NFR types. Regarding Waste, the problem is less frequent, with the exception of Security with 15% of respondents stating that they document requirements which they do not consider important. Conclusions: There is a clear indication that for a fixed set of NFRs lack of documentation of important NFRs occurs often, suggesting a risk of Technical Debt. The potential risk of incurring Waste is also present but to a lesser extent.

ABSTRACT. InsighTD is a globally distributed family of industrial surveys on causes and effects of Technical Debt (TD). We are currently analyzing the gathered data from the independent replication of the questionnaire in Costa Rica. In total, 156 pro-fessionals from the Costa Rican software industry answered the survey. Initial results indicate that there is a broad familiarity with the concept of TD. For the given examples of TD, it seems that the type of TD was preventable. TD was monitored for slightly more than half of cases, and TD was not paid in most cas-es. We hope to shed light on the causes and effect of TD in Costa Rica and with it contributes to prevent and improve TD management.

ABSTRACT. BACKGROUND: Effort estimation models are often built based on history data from past projects in an organization. Filtering techniques have been proposed for improving the estimation accuracy. Chronological filtering relies on the time proximity among project data and ignores much old data. Relevancy filtering utilizes the proximity of characteristics among project data and ignores dissimilar data. Their interaction is interesting because one would be able to make more accurate estimates if a positive synergistic effect exists.

AIMS: To examine whether the chronological filtering and the relevancy filtering can contribute to improving the estimation accuracy together.

METHOD: moving windows approaches as chronological filtering and a nearest neighbor approach as relevancy filtering are applied to a single-company ISBSG data.

RESULTS: we observed a negative synergistic effect. Each of the filtering approaches brought better effort estimates than using the whole history data. However, their combination may cause worse effort estimates than using the whole history data.

CONCLUSIONS: Practitioners should care about a negative synergistic effect when combining the chronological filtering and the relevancy filtering.

ABSTRACT. Automating functional size measurement (FSM) for software applications that use specific development frameworks is a challenge for the industry. FSM automation brings benefits such as savings in time, costs, and better reliability of the measures, but it is hard to do. In this paper, a multi-case study was conducted to evaluate the accuracy of an automated software size estimation procedure implemented in a software development company. The measurement prototype tool called FastWorks FPA automatically obtains the IFPUG FPA function point estimation of software applications modeled in the organization' FastWorks framework. The measurement process using the tool is described, as well as the results of the case studies in which the tool was used in six different projects. The automated measurements extracted by the FastWorks FPA tool obtained an accuracy (magnitude of relative error) between 3.9\% and 12.9\% based on the total unadjusted function points. These results show that the functional size estimated by the tool could reasonably approximate the manual counting extracted by the specialists, thus probing the feasibility of automating the counting process.

ABSTRACT. Many studies have shown that the accuracy of the predictions obtained by estimation models built considering data collected by other companies (cross-company models) can be significantly worse than those of estimation models built employing a dataset collected by the single company (within-company models). This is due to the different characteristics among cross-company and within-company datasets. In this paper, we propose a simple approach based on the opinion of the experts that could help in the context of small/medium company that do not have data available from past developed projects. In particular, experts are in charge of selecting data from public cross-company datasets looking at the information about employed software development process and software technologies. The proposed strategy is based on the use of a Delphi approach to reach consensus among experts involved in the selection and the use of an estimation technique to build the estimation model using the selected data. To assess the strategy, we performed an empirical study considering a dataset from the PROMISE repository that includes information on the functional size expressed in terms of COSMIC and other information on the development process to be exploited for building the cross-company estimation model. We selected this dataset since COSMIC is the method used to size the applications by the company that provided the within-company dataset employed as test set to assess the accuracy of the built cross-company model. As for the estimation technique we used simple linear regression. We compared the accuracy of the obtained predictions with those of the cross-company model built without selecting the observations. The results are promising since the proposed strategy allow to improve effort predictions.

ABSTRACT. Based on interviews with 20 interviewees from four case study companies, this paper presents a list of key factors that enable efficient and effective exploratory testing of large-scale software systems. The nine factors are grouped into four themes: “The testers’ knowledge, experience and personality”, “Purpose and scope”, “Ways of working” and “Recording and reporting”. According to the interviewees, exploratory testing is a more creative way to work for the testers, and was therefore considered to make better use of the testers. Exploratory testing was also described as a good way to test system-wide and to test large-scale systems, especially exploratory testing with an end-user perspective. The identified key factors were confirmed by a series of follow-up interviews with the 20 interviewees and a cross-company workshop with 14 participants. This strengthens the generalizability of the findings, supporting that the list of key factors can be applied to projects in a large segment of the software industry. This paper also presents the results from a systematic literature review including 129 publications related to exploratory testing. No publication were found that summarizes the key factors that enable efficient and effective exploratory testing, which supports the novelty of the findings.

ABSTRACT. For more than a decade, data driven development has attracted attention as one of the most powerful means to improve effectiveness and ensure value delivery to customers. In online companies, controlled experimentation is the primary technique to measure how customers respond to variants of deployed software. In B2B companies, an interest for data driven development is rapidly emerging and experiments are run on selected instances of the system or as comparisons of previously computed data to ensure quality, improve configurations and explore new value propositions. Although the adoption of data driven development is challenging in general, it is especially so for embedded systems companies and for companies developing on-premise solutions. Due to complex systems with hardware dependencies, safety-critical functionality and strict regulations, these companies have longer development cycles, less frequent deployments and limited access to data. In this paper, and based on multi-case study research, we explore the specific challenges that embedded systems companies and companies developing on-premise solutions experience when adopting data driven development practices. The contribution of the paper is two-fold. First, we provide empirical evidence in which we identify the key challenges that embedded systems and on-premise solutions companies experience as they evolve through the process of adopting data driven development practices. Second, we define the key focus areas that these companies need to address for evolving their data driven development adoption process.

ABSTRACT. DevOps is a cultural movement that aims the collaboration of all the stake-holders involved in the development, deployment and operation of software to deliver a quality product or service in the shortest possible time. DevOps is relatively recent, and companies have developed their DevOps practices largely from scratch. Our research aims to conduct an analysis on practicing DevOps in +20 software-intensive companies to provide patterns of DevOps practices and identify their benefits and barriers. This paper presents the pre-liminary analysis of an exploratory case study based on the interviews to rel-evant stakeholders of two (multinational) companies. The results show the benefits (software delivery performance) and barriers that these companies are dealing with, as well as DevOps team topology they approached during their DevOps transformation. This study aims to help practitioners and re-searchers to better understand DevOps transformations and the contexts where the practices worked. This, hopefully, will contribute to strengthening the evidence regarding DevOps and supporting practitioners in making better informed decisions about the return of investment when adopting DevOps.

ABSTRACT. Artificial intelligence (AI) is becoming increasingly widespread in system development endeavors. As AI systems affect various stakeholders due to their unique nature, the growing influence of these systems calls for ethical considerations. Academic discussion and practical examples of autonomous system failures have highlighted the need for implementing ethics in software development. However, research on methods and tools for implementing ethics into AI system design and development in practice is still lacking. This paper begins to address this focal problem by providing elements needed for producing a baseline for ethics in AI based software development. This is achieved by reporting initial results from an industrial multiple case study on AI systems development in the health care sector. In the context of this study, ethics were perceived as interplay of transparency, responsibility and accountability, upon which research model is outlined. Through these cases, we explore the current state of practice out on the field in the absence of formal methods and tools for ethically aligned design.