Evaluation

Hypothesis

Besides the constructive part of the project, comprising the development of the transformation level, the mapping level as well as the compiler in between, the analytical part of the project will represent a substantial effort. Thereby, an in-depth comparison of our approach to existing model transformation languages plays a central role. In this context the following hypothesis will be verified: “The provision of a framework for developing model transformations comprising (1) abstraction mechanisms in terms of a high-level mapping view and a low-level transformation view (2) reuse mechanisms in terms of concepts for genericity and composition and (3) a dedicated runtime model in terms of Coloured Petri Nets, enhances the quality of model transformations and their development in the sense of ISO 9126, focusing on usability (i.e., understandability and learnability) and maintainability (i.e., analyzability and debuggability).”

Verification

The methodology for verifying the above mentioned hypothesis builds on three major pillars.

(1) Case studies. First, appropriate case studies for transforming heterogeneous structural as well as behavioural models will be set up and implemented with distinct existing model transformation languages, including TROPIC, the results being evaluated on basis of a suitable subset of the ISO 126 software quality model. Since there are no benchmarks for model transformation approaches available, we intend to use a representative selection of metamodels defining tructural and behavioural languages. For this, we aim to use, on the one hand, the well-known Class2Relational example and on the other hand the CSP2ActivityDiagrams example. Furthermore, based on a detailed investigation of structural heterogeneities between existing business process modelling languages such as BPEL (Business Process Execution Language) and UML Activity Diagrams, we will conduct additional case
studies. For measuring the results of this implementation effort with respect to our hypothesis, a software quality model based on ISO 9126 will be established.

(2) Empirical studies. Second, the findings of these case studies will be further reflected by conducting empirical studies with students from our model engineering courses (around 200 master students every year). The goal of these studies is twofold. First, usability in the sense of understandability will be verified. For this, the students will be divided into three subgroups, whereas each subgroup gets a transformation definition in a certain transformation language hereby, evaluating its understandability based on questionnaires. Second, usability in the sense of learnability will be verified. According to this, three distinct transformation approaches will be presented in three timeslots. Afterwards the students will have to solve a certain problem with each of these approaches.

(3) Collaborative studies. Third, dedicated workshops will be held together with internationally renowned inventors of other model transformation languages to additionally review the value of our proposed framework with respect to other approaches. We will arrange a workshop with the founder of Coloured Petri Nets in order to discuss the intended extensions to the formalism and a workshop with the founders of the model transformation languages ATL and TGG in order to discuss similarities and differences of our approaches. These workshops should result in joint publications.