IEEE Transactions on Software Engineering

PrePrint: Class Schema Evolution for Persistent Object-Oriented Software: Model, Empirical Study, and Automated Support

With the wide support for object serialization in object-oriented programming languages, persistent objects have become common place and most large object-oriented software systems rely on extensive amounts of persistent data. Such systems also evolve over time. Retrieving previously persisted objects from classes whose schema has changed is however difficult, and may lead to invalidating the consistency of the application. The ESCHER framework addresses these issues through an IDE-integrated approach that handles class schema evolution by managing versions of the code and generating transformation functions automatically. The infrastructure also enforces class invariants to prevent the introduction of potentially corrupt objects. This article describes a model for class attribute changes, a measure for class evolution robustness, four empirical studies, and the design and implementation of the ESCHER system.

PrePrint: Evaluating Dynamic Software Update Safety Using Systematic Testing

Dynamic software updating (DSU) systems patch programs on the fly without incurring downtime. To avoid failures due to the updating process itself, many DSU systems employ timing restrictions. However, timing restrictions are theoretically imperfect, and their practical effectiveness is an open question. This paper presents the first significant empirical evaluation of three popular timing restrictions: activeness safety (AS), which prevents updates to active functions; con-freeness safety (CFS), which only allows modifications to active functions when doing so is provably type-safe; and manual identification of the event-handling loops during which an update may occur. We evaluated these timing restrictions using a series of DSU patches to three programs: OpenSSH, vsftpd, and ngIRCd. We systematically applied updates at each distinct update point reached during execution of a suite of system tests for these programs to determine which updates pass and which fail. We found that all three timing restrictions prevented most failures, but only manual identification allowed none. Further, although CFS and AS allowed many more update points, manual identification still supported updates with minimal delay. Finally, we found that manual identification required the least developer effort. Overall, we conclude that manual identification is most effective.

PrePrint: Does Software Process Improvement Reduce the Severity of Defects? A Longitudinal Field Study

As firms increasingly rely on information systems to perform critical functions the consequences of software defects can be catastrophic. Although the software engineering literature suggests that software process improvement can help to reduce software defects, the actual evidence is equivocal. For example, improved development processes may only remove the “easier” syntactical defects, while the more critical defects remain. Rigorous empirical analyses of these relationships have been very difficult to conduct due to the difficulties in collecting the appropriate data on real systems from industrial organizations. This field study analyzes a detailed data set consisting of 7,545 software defects that were collected on software projects completed at a major software firm. Our analyses reveal that higher levels of software process improvement significantly reduce the likelihood of high severity defects. In addition, we find that higher levels of process improvement are even more beneficial in reducing severe defects when the system developed is large or complex, but are less beneficial when requirements are ambiguous, unclear or incomplete. Our findings reveal the benefits and limitations of software process improvement for the removal of severe defects and suggest where investments in improving development processes may have their greatest effects.

PrePrint: SMT-Based Bounded Model Checking for Embedded ANSI-C Software

Propositional bounded model checking has been applied successfully to verify embedded software but remains limited by increasing propositional formula sizes and the loss of high-level information during the translation preventing potential optimizations to reduce the state space to be explored. These limitations can be overcome by encoding high-level information in theories richer than propositional logic and using SMT solvers for the generated verification conditions. Here, we propose the application of different background theories and SMT solvers to the verification of embedded software written in ANSI-C in order to improve scalability and precision in a completely automatic way. We have modified and extended the encodings from previous SMT-based bounded model checkers to provide more accurate support for variables of finite bit width, bit-vector operations, arrays, structures, unions and pointers. We have integrated the CVC3, Boolector, and Z3 solvers with the CBMC front-end and evaluated them using both standard software model checking benchmarks and typical embedded software applications from telecommunications, control systems, and medical devices. The experiments show that our ESBMC model checker can analyze larger problems than existing tools and substantially reduce the verification time.

PrePrint: Size-Constrained Regression Test Case Selection Using Multi-Criteria Optimization

This paper proposes a novel approach for selecting and ordering a predetermined number of test cases from an existing test suite. Our approach forms an Integer inear Programming problem using two different coverage-based criteria, and uses constraint relaxation to find many close-to-optimal solution points. These points are then combined to obtain a final solution using a voting mechanism. The selected subset of test cases are then prioritized using a greedy algorithm that maximizes minimum coverage in an iterative manner. The proposed approach has been empirically evaluated and the results show significant improvements over existing approaches for some cases and comparable results for the est. Moreover, our approach provides more consistency compared to existing approaches.

PrePrint: QoS Assurance for Dynamic Reconfiguration of Component Based Software Systems

A major challenge of dynamic reconfiguration is Quality of Service (QoS) assurance, which is meant to reduce appli-cation disruption to the minimum for the system’s transformation. However, this problem has not been well studied. This paper investigates the problem for component-based software systems from three points of view. First, the whole spectrum of QoS characteristics is defined. Second, the logical and physical requirements for QoS characteristics are analyzed and solutions to achieve them are proposed. Third, prior work is classified by QoS characteristics and then realized by abstract reconfiguration strategies. On this basis, quantitative evaluation of the QoS assurance abilities of existing work and our own approach is conducted through three steps. First, a proof-of-concept prototype called the reconfigurable component model is implemented to support the representation and testing of the reconfiguration strategies. Second, a reconfiguration benchmark is proposed to expose the whole spectrum of QoS problems. Third, each reconfiguration strategy is tested against the benchmark and the testing results are evaluated. The most important conclusion from our investigation is that the classified QoS characteristics can be fully achieved under some acceptable constraints.

PrePrint: Forecasting Risk Impact on ERP Maintenance with Augmented Fuzzy Cognitive Maps

Worldwide, firms have made great efforts to implement Enterprise Resource Planning (ERP) systems. Despite these efforts, ERP adoption success is not guaranteed. Successful adoption of an ERP system also depends on proper system maintenance. For this reason, companies should follow a maintenance strategy that drives the ERP system toward success. However, in general, ERP maintenance managers do not know what conditions they should target to successfully maintain their ERP systems. Furthermore, numerous risks threaten these projects, but they are normally dealt with intuitively. To date, there has been limited literature published regarding ERP maintenance risks or ERP maintenance success. To address this need, we have built a dynamic simulation tool that allows ERP managers to foresee the impact of risks on maintenance goals. This research would help professionals manage their ERP maintenance projects. Moreover, it covers a significant gap in the literature.

PrePrint: Performance Specification and Evaluation with Unified Stochastic Probes and Fluid Analysis

Rapid and accessible performance evaluation of complex software systems requires two critical features: being able to specify useful performance metrics easily and the capability to analyse massively distributed architectures, without recourse to large compute clusters. We present the unified stochastic probe, a performance specification mechanism for process algebra models that combines many existing ideas: state and action-based activation, location-based specification, many-probe specification and immediate signalling. These features, between them, allow the precise and compositional construction of complex performance measurements. The paper shows how a subset of the stochastic probe language can be used to specify common response time measures in massive process algebra models. The second contribution of the paper is to show how these response-time measures can be analysed using so-called fluid techniques to produce rapid results. In doing this, we extend the fluid approach to incorporate immediate activities and a new type of response-time measure. Finally, we calculate various response-time measurements on a complex distributed wireless network of O(10^129) states in size.

PrePrint: On Fault Representativeness of Software Fault Injection

The injection of software faults in software components to assess the impact of these faults on other components or on the system as a whole, allowing the evaluation of fault tolerance, is relatively new compared to decades of research on hardware fault injection. This paper presents an extensive experimental study (more than 3.8 millions of individual experiments in three real systems) to evaluate the representativeness of faults injected by a state-of-the-art approach (G-SWFIT). Results show that a significant share (up to 72%) of injected faults cannot be considered representative of residual software faults, as they are consistently detected by regression tests, and that representativeness of injected faults is affected by the fault location within the system, resulting in different distributions of representative/non-representative faults across files and functions. Therefore, we propose a new approach to refine the faultload by removing faults that are not representative of residual software faults. This filtering is essential to assure meaningful results and to reduce the cost (in terms of number of faults) of software fault injection campaigns in complex software. The proposed approach is based on classification algorithms, is fully automatic, and can be used for improving fault representativeness of existing software fault injection approaches.

PrePrint: Empirical Principles and an Industrial Case Study in Retrieving Equivalent Requirements via Natural Language Processing Techniques

Though very important in software engineering, linking artifacts of the same type (clone detection) or different types (traceability recovery) is extremely tedious, error-prone, and effort-intensive. Past research focused on supporting analysts with techniques based on Natural Language Processing (NLP) to identify candidate links. Because many NLP techniques exist and their performance varies according to context, it is crucial to define and use reliable evaluation procedures. The aim of this paper is to propose a set of seven principles for evaluating the performance of NLP techniques in identifying equivalent requirements. In this paper we conjecture, and verify, that NLP techniques perform on a given dataset according to both ability and the odds of identifying equivalent requirements correctly. For instance, when the odds of identifying equivalent requirements are very high, then it is reasonable to expect that NLP techniques will result in good performance. Our key idea is to measure this random factor of the specific dataset(s) in use and then adjust the observed performance accordingly. To support the application of the principles we report their practical application to a case study that evaluates the performance of a large number of NLP techniques for identifying equivalent requirements in the context of an Italian company in the defense and aerospace domain.

PrePrint: Random Testing: Theoretical Results and Practical Implications

A substantial amount of work has shed light on whether random testing is actually a useful testing technique. Despite its simplicity, several successful real-world applications have been reported in the literature. Although it is not going to solve all possible testing problems, random testing appears to be an essential tool in the hands of software testers. In this paper, we review and analyze the debate about random testing. Its benefits and drawbacks are discussed. Novel results addressing general questions about random testing are also presented, such as how long random does testing need on average to achieve testing targets (e.g., coverage), how does it scale and how likely is it to yield similar results if we re-run it on the same testing problem (predictability). Due to its simplicity that makes the mathematical analysis of random testing tractable, we provide precise and rigorous answers to these questions. Results show that there are practical situations in which random testing is a viable option. Our theorems are backed up by simulations and we show how they can be applied to most types of software and testing criteria. In light of these results, we then assess the validity of empirical analyses reported in the literature and derive guidelines for both practitioners and scientists.

PrePrint: Language-Independent and Automated Software Composition: The FEATUREHOUSE Experience

Superimposition is a composition technique that has been applied successfully in many areas of software development. Although superimposition is a general-purpose concept, it has been (re)invented and implemented individually for various kinds of software artifacts. We unify languages and tools that rely on superimposition by using the language-independent model of feature structure trees (FSTs). On the basis of the FST model, we propose a general approach to the composition of software artifacts written in different languages. Furthermore, we offer a supporting framework and tool chain, called FeatureHouse. We use attribute grammars to automate the integration of additional languages. In particular, we have integrated Java, C#, C, Haskell, Alloy, and JavaCC. A substantial number of case studies demonstrate the practicality and scalability of our approach and reveal insights into the properties that a language must have in order to be ready for superimposition. We discuss perspectives of our approach and demonstrate how we extended FeatureHouse with support for XML languages (in particular, XHTML, XMI/UML, and Ant) and alternative composition approaches (in particular, aspect weaving). Rounding off our previous work, we provide here a holistic view of the FeatureHouse approach based on rich experience with numerous languages and case studies and reflections on several years of research.

PrePrint: Matching and Merging of Variant Feature Specifications

Model Management addresses the problem of managing an evolving collection of models, by capturing the relationships between models and providing well-defined operators to manipulate them. In this article, we describe two such operators for manipulating feature specifications described using hierarchical state machine models: Match, for finding correspondences between models, and Merge, for combining models with respect to known or hypothesized correspondences between them. Our Match operator is heuristic, making use of both static and behavioural properties of the models to improve the accuracy of matching. Our Merge operator preserves the hierarchical structure of the input models, and handles differences in behaviour through parameterization. This enables us to automatically construct merges that preserve the semantics of hierarchical state machines. We report on tool support for our Match and Merge operators, and illustrate and evaluate our work by applying these operators to a set of telecommunication features built by AT&T.

PrePrint: On the Value of Ensemble Effort Estimation

Background: Despite decades of research, there is no consensus on which software effort estimation methods produce the most accurate models. Aim: Prior work has reported that, given M estimation methods, no single method consistently outperforms all others. Perhaps rather than recommending one estimation method as best, it is wiser to generate estimates from ensembles of multiple estimation methods. Method: 9 learners were combined with 10 pre-processing options to generate 9 × 10 = 90 solo-methods. These were applied to 20 data sets and evaluated using 7 error measures. This identified the best n (in our case n = 13) solo-methods that showed stable performance across multiple datasets and error measures. The top 2, 4, 8 and 13 solo-methods were then combined to generate 12 multi-methods, which were then compared to the solo-methods. Results: (i) The top 10 (out of 12) multi-methods significantly out-performed all 90 solo-methods. (ii) The error rates of the multi- methods were significantly less than the solo-methods. (iii) The ranking of the best multi-method was remarkably stable. Conclusion: While there is no best single effort estimation method, there exist best combinations of such effort estimation methods.

PrePrint: Programmer-Friendly Refactoring Errors

Refactoring tools, common to many integrated development environments, can help programmers to restructure their code. These tools sometimes refuse to restructure the programmer's code, instead giving the programmer a textual error message that she must decode if she wishes to understand the reason for the tool's refusal, and what corrective action to take. This article describes a graphical alternative to textual error messages called Refactoring Annotations. It reports on two experiments, one using an integrated development environment and the other using paper mockups, that show that programmers can use Refactoring Annotations to quickly and accurately understand the cause of refactoring errors.

PrePrint: Modeling Product Line Software Assets Using Domain-Specific Kits

Software Product Line Engineering (SPLE) is a prominent paradigm for the assembly of a family of products using product line core assets. The modeling of software assets that together form the actual products is critical for achieving the strategic benefits of software product lines. We propose a feature-based approach to software asset modeling based on abstractions provided by Domain-Specific Kits. This approach involves a software asset metamodel used to derive asset modeling languages that define reusable software assets in domain-specific terms. The approach also prescribes a roadmap for modeling these software assets in conjunction with the product line reference architecture. Asset capabilities can be modeled using feature diagrams as the external views of the software assets. Internal views can be expressed in terms of domain-specific artifacts with variability points, where the domain-specific artifacts are created using Domain-Specific Kits. This approach produces loosely coupled and highly cohesive software assets that are reusable for multiple product lines. The approach is validated by assessing software asset reuse in two different product lines in the finance domain. We also evaluated the productivity gains in large-scale complex projects, and found that the approach yielded a significant reduction in the total project effort.

PrePrint: Automatic Detection of Unsafe Dynamic Component Loadings

Dynamic loading of software components is a widely used mechanism for improved system modularity and flexibility. Correct component resolution is critical for reliable and secure software execution, however, programming mistakes may lead to unintended or even malicious components to be resolved and loaded. In particular, dynamic loading can be hijacked by placing an arbitrary file with the specified name in a directory searched before resolving the target component. In this paper, we present the first automated technique to detect unsafe dynamic component loadings. Our analysis has two phases: 1) apply dynamic binary instrumentation to collect runtime information on component loading; and 2) analyze the collected information to detect vulnerable component loadings. For evaluation, we implemented our technique to detect unsafe component loadings in popular software on Microsoft Windows and Linux. Our evaluation shows that unsafe loading is prevalent in software on both OS platforms, and it is more severe on Microsoft Windows. In particular, our tool detected more than 4,000 unsafe component loadings, and some lead to remote code execution on Microsoft Windows.

PrePrint: Towards A Tool-Based Development Methodology for Pervasive Computing Applications

Despite much progress, developing a pervasive computing application remains a challenge because of a lack of conceptual frameworks and supporting tools. This challenge involves coping with heterogeneous devices, overcoming the intricacies of distributed systems technologies, working out an architecture for the application, encoding it in a program, writing specific code to test the application, and finally deploying it. This paper presents a design language and a tool suite covering the development life-cycle of a pervasive computing system. The design language allows to define a taxonomy of area-specific building-blocks, abstracting over their heterogeneity. This language also includes a layer to define the architecture of an application, following an architectural pattern commonly used in the pervasive computing domain. Our underlying methodology assigns roles to the stakeholders, providing separation of concerns. Our tool suite includes a compiler that takes design artifacts written in our language as input, and generates a programming framework that supports the subsequent development stages, namely implementation, testing and deployment. Our methodology has been applied on a wide spectrum of areas. Based on these experiments, we assess our approach through three criteria: expressiveness, usability and productivity.

PrePrint: A Static Approach to Prioritizing JUnit Test Cases

Over the past few years, many test case prioritization techniques have been proposed in the literature. Most of them require data on dynamic execution in the form of code coverage information for test cases. However, the collection of dynamic code coverage information has several associated drawbacks including cost increases and reduction in prioritization precision. In this paper, we propose an approach to prioritizing test cases in the absence of coverage information that operates on Java programs tested under the JUnit framework. Our approach, JUPTA analyzes the static call graphs of JUnit test cases and the program under test to estimate the ability of each test case to achieve code coverage, and then schedules the order of these test cases based on those estimates. The experimental results show that the test suites constructed by JUPTA are more effective than those in random and untreated test orders in fault-detection effectiveness. Although the test suites constructed by dynamic coverage-based techniques retain fault-detection effectiveness advantages, the fault-detection effectiveness abilities of test suites constructed by JUPTA are close to those dynamic techniques, and the fault-detection effectiveness abilities of the test suites constructed by some of JUPTA's variants are better than several of those techniques.

PrePrint: Automatically Generating Test Cases for Specification Mining

Dynamic specification mining observes program executions to infer models of normal program behavior. What makes us believe that we have seen sufficiently many executions? The TAUTOKO typestate miner generates test cases that cover previously unobserved behavior, systematically extending the execution space and enriching the specification. To our knowledge, this is the first combination of systematic test case generation and typestate mining–a combination with clear benefits: On a sample of 800 defects seeded into six Java subjects, a static typestate verifier fed with enriched models would report significantly more true positives, and significantly fewer false positives than the initial models.

PrePrint: A Systematic Review of Fault Prediction Performance in Software Engineering

Background: The accurate prediction of where faults are likely to occur in code can help direct test effort, reduce costs and improve the quality of software. Objective: We investigate how the context of models, the independent variables used and the modelling techniques applied, influence the performance of fault prediction models. Method:We used a systematic literature review to identify 208 fault prediction studies published from January 2000 to December 2010. We synthesise the quantitative and qualitative results of 36 studies which report sufficient contextual and methodological information according to the criteria we develop and apply. Results: The models that perform well tend to be based on simple modelling techniques such as Na�ve Bayes or Logistic Regression. Combinations of independent variables have been used by models that perform well. Feature selection has been applied to these combinations when models are performing particularly well. Conclusion: The methodology used to build models seems to be influential to predictive performance. Although there are a set of fault prediction studies in which confidence is possible, more studies are needed that use a reliable methodology and which report their context, methodology and performance comprehensively.

PrePrint: Formal Specification-Based Inspection for Verification of Programs

Software inspection is a static analysis technique that is widely used for defect detection, but which suffers from a lack of rigor. In this paper, we address this problem by taking advantage of formal specification and analysis to support a systematic and rigorous inspection method. The aim of the method is to use inspection to determine whether every functional scenario defined in the specification is implemented correctly by a set of program paths and whether every program path of the program contributes to the implementation of some functional scenario in the specification. The method comprises five steps: deriving functional scenarios from the specification; deriving paths from the program; linking scenarios to paths; analyzing paths against the corresponding scenarios; and producing an inspection report, and allows for a systematic and automatic generation of a checklist for inspection. We present an example to show how the method can be used, and describe an experiment to evaluate its performance by comparing it to perspective-based reading (PBR). The result shows that our method may be more effective in detecting function-related defects than PBR but slightly less effective in detecting implementation-related defects. We also describe a prototype tool to demonstrate the supportability of the method, and draw some conclusions about our work.

PrePrint: Analyzing the Effect of Gain Time on Soft Task Scheduling Policies in Real-Time Systems

In hard real-time systems, gain time is defined as the difference between the worst-case execution time of a hard task and its actual processor consumption at run time. This paper presents the results of an empirical study about how the presence of a significant amount of gain time in a hard real-time system questions the advantages of using the most representative scheduling algorithms or policies for aperiodic or soft tasks in fixed-priority preemptive systems. The work presented here refines and complements many other studies in this research area, in which such policies have been introduced and compared. This work has been performed by using the authors' testing framework for soft scheduling policies, which produces actual, synthetic, randomly-generated applications, executes them in an instrumented real-time operating system, and finally processes such information to obtain several statistical outcomes. The results show that, in general, the presence of a significant amount of gain time reduces the performance benefit of the scheduling policies under study when compared to serving the soft tasks in background, which is considered the theoretical worst case. In some cases, this performance benefit is so small that the use of a specific scheduling policy for soft tasks is questionable.

PrePrint: Architecture-Based Reliability Prediction with the Palladio Component Model

With the increasing importance of reliability in business and industrial software systems, new techniques of architecture-based reliability engineering are becoming an integral part of the development process. These techniques can assist system architects in evaluating the reliability impact of their design decisions. Architecture-based reliability engineering is only effective if the involved reliability models reflect the interaction and usage of software components and their deployment to potentially unreliable hardware. However, existing approaches either neglect individual impact factors on reliability or hard-code them into formal models, which limits their applicability in component-based development processes. This paper introduces a reliability modelling and prediction technique that considers the relevant architectural factors of software systems and explicitly models the component usage profile and execution environment. The technique offers a UML-like modelling notation, whose models are automatically transformed into a formal analytical model. Our work builds upon the Palladio Component Model, employing novel techniques of information propagation and reliability assessment. We validate our technique with sensitivity analyses and simulation in two case studies. The case studies demonstrate effective support of usage profile analysis and architectural configuration ranking, together with the employment of reliability-improving architecture tactics.

PrePrint: Mutation-Driven Generation of Unit Tests and Oracles

To assess the quality of test suites, mutation analysis seeds artificial defects (mutations) into programs; a non-detected mutation indicates a weakness in the test suite. We present an automated approach to generate unit tests that detect these mutations for object-oriented classes. This has two advantages: First, the resulting test suite is optimized towards finding defects rather than covering code. Second, the state change caused by mutations induces oracles that precisely detect the mutants. Evaluated on 10 open source libraries, our MuTest prototype generates test suites that find significantly more seeded defects than the original manually written test suites.

Presented By:

Prepare for Tomorrow, Today, with Cisco

		The way the world consumes and shares data will dramatically change in the next five years. Is your network ready to handle the load? Prepare for the future of the network with Cisco.
		socialmedia.cisco.com

Ads by Pheedo

PrePrint: Solving the Large Scale Next Release Problem with a Backbone Based Multilevel Algorithm

The Next Release Problem (NRP) aims to optimize customer profits and requirements selection for the software releases. The research on the NRP is restricted by the growing scale of requirements. In this paper, we propose a Backbone based Multilevel Algorithm (BMA) to address the large scale NRP. In contrast to direct solving approaches, BMA employs multilevel reductions to downgrade the problem scale and multilevel refinements to construct the final optimal set of customers. In both reductions and refinements, the backbone is built to fix the common part of the optimal customers. Since it is intractable to extract the backbone in practice, the approximate backbone is employed for the instance reduction while the soft backbone is proposed to augment the backbone application. In the experiments, to cope with the lack of open large requirements databases, we propose a method to extract instances from open bug repositories. Experimental results on 15 classic instances and 24 realistic instances demonstrate that BMA can achieve better solutions on the large scale NRP instances than direct solving approaches. Our work provides a reduction approach for solving large scale problems in search based requirements engineering.

PrePrint: The Link between Dependency and Co-Change: Empirical Evidence

We investigate the relationship between class dependency and change propagation (co-change) in software written in Java. On the one hand we find a strong correlation between dependency and co-change. Furthermore, we provide empirical evidence for the propagation of change along paths of dependency. These findings support the often alleged role of dependencies as propagators of change. On the other hand we find that approximately half of all dependencies are never involved in co-changes and that the vast majority of co-changes pertain to only a small percentage of dependencies. This means that inferring the co-change characteristics of a software architecture solely from its dependency structure results in a severely distorted approximation of co-change characteristics. Any metric which uses dependencies alone to pass judgment on the evolvability of a piece of Java software is thus implausible. As a consequence we suggest to always take both the change characteristics and the dependency structure into account when evaluating software architecture.

PrePrint: Clone Management for Evolving Software

Recent research results suggest a need for code clone management. In this paper, we introduce JSync, a novel clone management tool for evolving software. JSync provides two main functions to support developers in being aware of the clone relation among code fragments as software systems evolve and in making consistent changes as they create or modify cloned code. JSync represents source code and clones as (sub)trees in Abstract Syntax Trees, measures code similarity based on structural characteristic vectors, and describes code changes as tree editing scripts. The key techniques of JSync include the algorithms to compute tree editing scripts, to detect and update code clones and their groups, to analyze the changes of cloned code to validate their consistency, and to recommend relevant clone synchronization and merging. Our empirical evaluation on several real-world systems shows that JSync is efficient and accurate in clone detection and updating, and provides the correct detection of the defects resulting from inconsistent changes to clones and the correct recommendations for change propagation across cloned code.

PrePrint: Coping with Existing Systems in Information Systems Development

Determining how to cope with existing systems is an important issue for information systems development (ISD). In this paper, we investigate how well different ISD patterns are suited for coping with existing systems. Empirical results, gathered from three software development projects undertaken by a financial institution, suggest propositions regarding how ISD patterns and existing systems affect the characteristics of objective ISD complexity, which in turn determine overall experienced complexity. Existing systems increase complexity due to conflicting interdependencies, but ISD patterns that reduce this complexity, such as those that employ bottom-up or concurrent consideration patterns, are best suited for coping with existing systems. In contrast, top-down and iterative focusing patterns, as classically used in new development, increase the complexity associated with conflicting interdependency, which makes them particularly unsuited for coping with existing systems in ISD.

PrePrint: Structural Complexity and Programmer Team Task Strategy: An Experimental Test

This study develops and empirically tests the idea that the impact of structural complexity on perfective maintenance of object-oriented software is significantly determined by the team strategy of programmers (independent or collaborative). We analyzed two key dimensions of software structure, coupling and cohesion, with respect to the maintenance effort and the perceived ease-of-maintenance by pairs of programmers. Hypotheses based on the distributed cognition and task interdependence theoretical frameworks were tested using data collected from a controlled lab experiment employing professional programmers. The results show a significant interaction effect between coupling, cohesion, and programmer team strategy on both maintenance effort and perceived ease-of-maintenance. Highly cohesive and low-coupled programs required lower maintenance effort and were perceived to be easier to maintain than the low cohesive programs and high-coupled programs. Further, our results would predict that managers who allocate maintenance tasks to independent or collaborative programming teams depending on the structural complexity of software could lower their team’s maintenance effort by as much as 70% over managers who use simple uniform resource allocation policies. These results highlight the importance of achieving congruence between team strategies employed by collaborating programmers and the structural complexity of software.

PrePrint: A Practical Approach to Size Estimation of Embedded Software Components

To estimate Software Code Size early in the development process is important for developing cost-efficient embedded systems. We have applied the COSMIC FSM (Functional Size Measurement) method for Size Estimation of Embedded Software Components in the automotive industry. Experiments were conducted using data from two automotive companies. The experiments show strong correlation between Functional Size and Software Code Size, which is important for obtaining accurate estimation results. This paper presents the characteristics and results of our work, and aims to provide a practical framework for how to use COSMIC FSM for Size Estimation purposes. We investigate the results from our earlier experiments, and conduct further experiments to identify such a framework. Based on these activities we conclude that a clear purpose of the estimation process, a well-defined domain allowing categorization of software, consistent content and quality of requirements, and historical data from implemented software are key factors for Size Estimation of Embedded Software Components.

PrePrint: Formal Analysis of the Probability of Interaction Fault Detection Using Random Testing

Modern systems are becoming highly configurable to satisfy the varying needs of customers and users. However, high levels of configurability entail new challenges. Some faults might be revealed only if a particular combination of features is selected in the delivered products. But testing all combinations is usually not feasible in practice, due to their extremely large numbers. Combinatorial testing is a technique to generate smaller test suites for which all combinations of t features are guaranteed to be tested. In this paper, we formally prove that, when there are no constraints among the features that can be part of a product, then random testing is much more effective than one would intuitively expect at detecting feature interaction faults when compared with combinatorial testing. More importantly, random testing becomes even more effective as the number of features increases and converges towards equal effectiveness with combinatorial testing. Given that combinatorial testing entails significant computational overhead in the presence of large numbers of features, the results suggest that there are realistic scenarios in which random testing may outperform combinatorial testing in large systems. However, when constraints are present among features, then random testing can fare arbitrarily worse than combinatorial testing.

PrePrint: Exemplar: A Source Code Search Engine For Finding Highly Relevant Applications

A fundamental problem of finding software applications that are highly relevant to development tasks is the mismatch between the high-level intent reflected in the descriptions of these tasks and low-level implementation details of applications. To reduce this mismatch we created an approach called Exemplar (EXEcutable exaMPLes ARchive) for finding highly relevant software projects from large archives of applications. After a programmer enters a natural-language query that contains high-level concepts (e.g., MIME, data sets), Exemplar retrieves applications that implement these concepts. Exemplar ranks applications in three ways. First, we consider the descriptions of applications. Second, we examine the Application Programming Interface (API) calls used by applcations. Third, we analyze the dataflow among those API calls.We performed two case studies (with professional and student developers) to evaluate how these three rankings contribute to the quality of the search results from Exemplar. The results of our studies show that the combined ranking of application descriptions and API documents yields the most-relevant search results. We released Exemplar and our case study data to the public.

PrePrint: Mining Crosscutting Concerns through Random Walks

Inspired by our past manual aspect mining experience, this paper describes a probabilistic random walk model to approximate the process of discovering crosscutting concerns in the absence of domain knowledge about the application investigated. The random walks are performed on the concept graphs extracted from the program sources to calculate metrics of “utilization” and “aggregation” for each program element. We rank all the program elements based on these metrics and use a threshold to produce a set of candidates that represent crosscutting concerns (CCs). We implemented the algorithm as the Prism CC Miner (PCM) and evaluated PCM on Java applications ranging from a small-scale drawing application to a medium-sized middleware application and a large-scale enterprise application server. Our quantification shows that PCM is able to produce comparable results to manual mining efforts. With a 95% accuracy the top 125 crosscutting candidates are identified automatically. PCM is also significantly more effective as compared to conventional mining approaches.

PrePrint: Fluid Rewards for a Stochastic Process Algebra

Reasoning about the performance of models of software systems typically entails the derivation of metrics such as throughput, utilisation, and response time. If the model is a Markov chain, these are expressed as real functions of the chain, called reward models. The computational complexity of reward-based metrics is of the same order as the solution of the Markov chain, making the analysis infeasible when evaluating large-scale systems. In the context of the stochastic process algebra PEPA, the underlying continuous-time Markov chain has been shown to admit a deterministic (fluid) approximation as a solution of an ordinary differential equation, which effectively circumvents state-space explosion. This paper is concerned with approximating Markovian reward models for PEPA with fluid rewards, i.e., functions of the solution of the differential equation problem. It shows that (a) the Markovian reward models for typical metrics of performance enjoy asymptotic convergence to their fluid analogues, and that (b), via numerical tests, the approximation yields satisfactory accuracy in practice.

PrePrint: Reasoning about the Reliability of Diverse Two-Channel Systems in which One Channel is "Possibly Perfect"

This paper considers the problem of reasoning about the reliability of fault-tolerant systems with two "channels" (i.e., components) of which one, A, supports only a claim of reliability, while the other, B, by virtue of extreme simplicity and extensive analysis, supports a plausible claim of "perfection." We begin with the case where either channel can bring the system to a safe state. We show that, conditional upon knowing pA (the probability that A fails on a randomly selected demand) and pB (the probability that channel B is imperfect), a conservative bound on the probability that the system fails on a randomly selected demand is simply pA.pB. That is, there is conditional independence between the events "A fails" and "B is imperfect." The second step of the reasoning involves epistemic uncertainty about (pA, pB) and we show that under quite plausible assumptions, a conservative bound on system pfd can be constructed from point estimates for just three parameters. We discuss the feasibility of establishing credible estimates for these parameters. We extend our analysis from faults of omission to those of commission, and then combine these to yield an analysis for monitored architectures of a kind proposed for aircraft.

PrePrint: What Do We Know About the Effectiveness of Software Design Patterns

Context: Many software development practices are based upon codification of expert knowledge, often with little or no underpinning from objective empirical evidence. Objectives: To investigate how extensively the use of software design patterns has been subjected to empirical study, and what evidence is available about how and when their use can provide an effective mechanism for knowledge transfer about design. Method: We conducted a systematic literature review in the form of a mapping study, searching the literature up to the end of 2009 to identify relevant primary studies about the use of the 23 patterns catalogued in the book by the `Gang of Four'. Results: Our searches identified 611 candidate papers. Applying our inclusion/exclusion criteria resulted in a final set of ten papers that described 11 instances of `formal' experimental studies of object-oriented design patterns. We augmented our analysis by including seven `experience' reports that described application of patterns using less rigorous observational forms. Conclusions: There was some support for the usefulness of patterns in providing a framework for maintenance, and some qualitative indication that they do not help novices learn about design. We recommend that researchers use case studies that focus upon some key patterns, and seek to identify the impact that their use can have upon maintenance.

Presented By:

Welcome the Zettabyte Era with Cisco

		Today we use terms like gigabyte and terabyte when it comes to data. Five years from now, we will enter the era of the zettabyte. Connect with Cisco across the web through various social channels as we guide you through the future of the Internet.
		socialmedia.cisco.com

Ads by Pheedo

PrePrint: Two Studies of Framework-Usage Templates Extracted from Dynamic Traces

Object-oriented frameworks are widely used to develop new applications. They provide reusable concepts that are instantiated in application code through potentially complex implementation steps such as subclassing, implementing interfaces, and calling framework operations. Unfortunately, many modern frameworks are difficult to use because of their large and complex APIs and frequently-incomplete user documentation. To cope with these problems, developers often use existing framework applications as a guide. However, locating concept implementations in those sample applications is typically challenging due to code tangling and scattering. To address this challenge, we introduce the notion of concept-implementation templates, which summarize the necessary concept-implementation steps and identify them in the sample application code, and a technique, named FUDA, to automatically extract such templates from dynamic traces of sample applications. This article further presents the results of two experiments conducted to evaluate the quality and usefulness of FUDA templates. The experimental evaluation of FUDA with 14 concepts in five widely-used frameworks suggests that the technique is effective in producing templates with relatively few false positives and false negatives for realistic concepts by using two sample applications. Moreover, we observed in a user study with 28 programmers that the use of templates reduced the concept-implementation time compared to when documentation was used.

PrePrint: Fault Localization for Dynamic Web Applications

In recent years, there has been significant interest in fault-localization techniques that are based on statistical analysis of program constructs executed by passing and failing executions. This paper shows how the Tarantula, Ochiai, and Jaccard fault-localization algorithms can be enhanced to localize faults effectively in Web applications written in PHP, by using an extended domain for conditional and function-call statements, and by using a source mapping. We also propose several novel test-generation strategies that are geared towards producing test suites that have maximal fault-localization effectiveness. We implemented various fault-localization techniques and test-generation strategies in Apollo, and evaluated them on several open-source PHP applications. Our results indicate that a variant of the Ochiai algorithm that includes all our enhancements localizes 87.8% of all faults to within 1% of all executed statements, compared to only 37.4% for the unenhanced Ochiai algorithm. We also found that all the test-generation strategies that we considered are capable of generating test suites with maximal fault-localization effectiveness, when given an infinite time budget for test generation. However, on average, a directed strategy based on path-constraint similarity achieves this maximal effectiveness after generating only 6.5 tests, compared to 46.8 tests for an undirected test-generation strategy.

PrePrint: Precise Calling Context Encoding

Calling contexts are very important for a wide range of applications such as profiling, debugging, and event logging. Most applications perform expensive stack walking to recover contexts. The resulting contexts are often explicitly represented as a sequence of call sites and hence bulky. We propose a technique to encode the current calling context of any point during an execution. In particular, an acyclic call path is encoded into one number through only integer additions. Recursive call paths are divided into acyclic subsequences and encoded independently. We leverage stack depth in a safe way to optimize encoding: if a calling context can be safely and uniquely identified by its stack depth, we do not perform encoding. We propose an algorithm to seamlessly fuse encoding and stack depth based identification. The algorithm is safe because different contexts are guaranteed to have different IDs. It also ensures contexts can be faithfully decoded. Our experiments show that our technique incurs negligible overhead (0%-6.4% on average). For most medium-sized programs, it can encode all contexts with just one number. For large programs, we are able to encode most calling contexts to a few numbers. We also present our experience of applying context encoding to debugging crash based failures.

PrePrint: DESSERT: A DividE-and-Conquer Methodology for Identifying CategorieS, ChoiceS, and ChoicE Relations for Test Case Generation

This paper extends the CHOiCe reLATion framEwork, abbreviated as CHOC'LATE, which assists software testers in the application of category/choice methods to testing. CHOC'LATE assumes that the tester is able to construct a single choice relation table from the entire specification; this table then forms the basis for test case generation using the associated algorithms. This assumption, however, may not hold true when the specification is complex and contains many specification components. For such a specification, the tester may construct a preliminary choice relation table from each specification component, and then consolidate all the preliminary tables into a final table to be processed by CHOC'LATE for test case generation. However, it is often difficult to merge these preliminary tables because such merging may give rise to inconsistencies among choice relations or overlaps among choices. To alleviate this problem, we introduce a DividE-and-conquer methodology for identifying categorieS, choiceS, and choicE Relations for Test case generation, abbreviated as DESSERT. The theoretical framework and the associated algorithms are discussed. To demonstrate the viability and effectiveness of our methodology, we describe case studies using the specifications of three real-life commercial software systems.

PrePrint: MOSES: A Framework for QoS Driven Runtime Adaptation of Service-Oriented Systems

Architecting software systems according to the service-oriented paradigm, and designing runtime self-adaptable systems are two relevant research areas in today's software engineering. In this paper we address issues that lie at the intersection of these two important fields. First, we present a characterization of the problem space of self-adaptation for service-oriented systems, thus providing a frame of reference where our and other approaches can be classified. Then, we present MOSES, a methodology and a software tool implementing it to support QoS-driven adaptation of a service-oriented system. It works in a specific region of the identified problem space, corresponding to the scenario where a service-oriented system architected as a composite service needs to sustain a traffic of requests generated by several users. MOSES integrates within a unified framework different adaptation mechanisms. In this way it achieves a greater flexibility in facing various operating environments and the possibly conflicting QoS requirements of several concurrent users. Experimental results obtained with a prototype implementation of MOSES show the effectiveness of the proposed approach.

PrePrint: Tools for the Rapid Prototyping of Provably Correct Ambient Intelligence Applications

Ambient Intelligence technologies have not yet been widely adopted in safety critical scenarios. This principally has been due to fact that acceptable degrees of dependability have not been reached for the applications that rely on such technologies. However, the new critical application domains, like Ambient Assisted Living and Smart Hospitals, which are currently emerging, are increasing the need for methodologies and tools that can improve the reliability of the final systems. This paper presents a middleware architecture for safety critical Ambient Intelligence applications, which provides the developer with services for runtime verification. It is now possible to continuously monitor and check the running system against correctness properties defined at the design time. Moreover, a visual tool, which allows the formal design of several of the characteristics of an Ambient Intelligence application and the automatic generation of setting up parameters and code for the middleware infrastructure, is also presented.

PrePrint: Pert: The Application-Aware Tailoring for Java Object Persistence

Persistence is a widely used technique which allows the objects that represent the results of some lengthy computation to outlive the process that creates it, in order to considerably speed-up subsequent program executions. We observe that conventional persistence techniques usually do not consider the application contexts of the persistence operations, where not all of the object states are necessary to be persisted. Leveraging this observation, we have designed and implemented a framework, PERT, which first performs static program analysis to estimate the actual usage of the persisted object, given the context of its usage in the program. The Pert runtime uses the statically computed information to make efficiently tailoring decisions to prune the redundant object states during the persistence operations. Our evaluation result shows that the Pert-based optimization can speedup the conventional persistence operations by 1 to 45 times. The amount of persisted data is also dramatically reduced, as the result of the application-aware tailoring.

PrePrint: Palantír: Early Detection of Development Conflicts Arising from Parallel Code Changes

The earlier a conflict is detected, the easier it is to resolve – this is the main precept of workspace awareness. Workspace awareness seeks to provide users with information of relevant ongoing parallel changes occurring in private workspaces, thereby enabling the early detection and resolution of potential conflicts. The key approach is to unobtrusively inform developers of potential conflicts arising because of concurrent changes to the same file and dependency violations in ongoing parallel work. This paper describes our research goals, approach, and implementation of workspace awareness through Palantír and includes a comprehensive evaluation involving two laboratory experiments. We present both quantitative and qualitative results from the experiments, which demonstrate that the use of Palantír, as compared to not using Palantír: (1) leads to both earlier detection and earlier resolution of a larger number of conflicts, (2) leaves fewer conflicts unresolved in the code base that was ultimately checked in, and (3) involves reasonable overhead. Furthermore, we report on interesting changes in users’ behavior, especially how conflict resolution strategies changed among Palantír users.

PrePrint: Adaptation of Service Protocols Using Process Algebra and On-the-Fly Reduction Techniques

Reuse and composition are increasingly advocated and put into practice in modern software engineering. However, the software entities that are to be reused to build an application, e.g., services, have seldom been developed to integrate and to cope with the application requirements. As a consequence, they present mismatch, which directly hampers on their reusability and the possibility to compose them. Software Adaptation has become a hot topic as a non-intrusive solution to work mismatch out using corrective pieces named adaptors. However, adaptation is a complex issue, especially when behavioral interfaces, or conversations, are taken into account. In this article, we present state-of-the-art techniques to generate adaptors given the description of reused entities' conversations and an abstract specification of the way mismatch can be solved. We use a process algebra to encode the adaptation problem, and propose on-the-fly exploration and reduction techniques to compute adaptor protocols. Our approach follows the model-driven engineering paradigm, applied to service-oriented computing as a representative field of composition-based software engineering. We take service description languages as inputs of the adaptation process and we implement adaptors as centralized service compositions, i.e., orchestrations. Our approach is completely tool-supported.

PrePrint: Mutable Protection Domains: Adapting System Fault Isolation for Reliability and Efficiency

As software systems are becoming increasingly complex, the likelihood of faults and unexpected behaviors will naturally increase. Many researchers have developed methods to contain faults at run-time, by using software and hardware-based techniques to define protection domains. These approaches tend to impose isolation boundaries on software components that are static, and thus remain intact while the system is running. An unfortunate consequence of statically structured protection domains is that they may impose undue overhead on the communication between separate components. This paper proposes a new run-time technique called Mutable Protection Domains (MPD) that trades communication cost for fault isolation in our COMPOSITE operating system. MPD dynamically adapts hardware isolation between interacting software components, depending on observed communication "hot-paths", with the purpose of maximizing fault isolation where possible. In this sense, MPD naturally tends towards a system of maximal component isolation, while collapsing protection domains where costs are prohibitive. By increasing isolation for low-cost interacting components, MPD limits the scope of impact of future unexpected faults. We demonstrate the utility of MPD using a web server, and identify different hot-paths for different workloads, that dictate adaptations to system structure. Experiments show up to 40% improvement in throughput, compared to a statically-organized system, while maintaining high fault isolation.

PrePrint: Comparing Semi-Automated Clustering Methods for Persona Development

Current and future information systems require a better understanding of the interactions between users and systems in order to improve system use, and ultimately, success. The use of personas as design tools is becoming more widespread as researchers and practitioners discover its benefits. This paper presents an empirical study comparing the performance of existing qualitative and quantitative clustering techniques for the task of identifying personas and grouping system users into those personas. A method based on Factor (Principal Components) Analysis performs better than two other methods which use Latent Semantic Analysis and Cluster Analysis as measured by similarity to expert manually defined clusters.

PrePrint: BURN: Enabling Workload Burstiness in Customized Service Benchmarks

We introduce BURN, a methodology to create customized benchmarks for testing multi-tier applications under time-varying resource usage conditions. Starting from a set of pre-existing test workloads, BURN finds a policy that interleaves their execution to stress the multi-tier application and generate controlled burstiness in resource consumption. This is useful to study, in a controlled way, the robustness of software services to sudden changes in the workload characteristics and in the usage levels of the resources. The problem is tackled by a model-based technique which first generates Markov models to describe resource consumption patterns of each test workload. Then, a policy is generated using an optimization program which sets as constraints a target request mix and user-specified levels of burstiness at the different resources in the system. Burstiness is quantified using a novel metric called overdemand, which describes in a natural way the tendency of a workload to keep a resource congested for long periods of time and across multiple requests.

PrePrint: Data Mining Techniques for Software Effort Estimation: A Comparative Study

A predictive model is required to be accurate and comprehensible in order to inspire confidence in a business setting. Both aspects have been assessed in a software effort estimation setting by previous studies. However, no univocal conclusion to which technique is the most suited has been reached. This study addresses this issue by reporting on the results of a large scale benchmarking study. Different types of techniques are under consideration including techniques inducing tree/rule based models like M5 and CART, linear models such as various types of linear regression, non-linear models (MARS, multi layered perceptron neural networks, radial basis function networks and least squares support vector machines), and estimation techniques that not explicitly induce a model (e.g. a case based reasoning approach). Furthermore, the aspect of feature subset selection by using a generic backward input selection wrapper is investigated. The results are subjected to rigorous statistical testing and indicate that ordinary least squares regression in combination with a logarithmic transformation performs best. Another key finding is that by selecting a subset of highly predictive attributes such as project size, development, and environment related attributes, typically a significant increase in estimation accuracy can be obtained.