Review Criteria Catalog
The criteria used to evaluate and review the different modeling languages originate partly from the various review papers identified during the first literature search [BWKG14, BBF+18, BCR+19, KS19, NMJ19, SDA12, QK18]. However, this review considers only criteria aspects that may be relevant for the subsequent ADL selection. Therefore, some criteria have been slightly modified or extended. The selected criteria sets from the review papers are language characteristics, cloud modeling capabilities and tooling support. In addition to this more general perspective, two other criteria sets are examined that focus on aspects directly related to the quality model and the aim to represent CNA architectures formally: representation of proposed entities and architectural evaluations. The following presents a detailed description of the different aspects considered and analyzed within the mentioned criteria sets.
Architectural Evaluations
Since the CNA quality model intends to evaluate architectures, it could be advantageous if the modeling language under consideration has already been used for other architectural evaluations. Moreover, the quality model plans to include additional metrics such that the model can also be applied for quantitative evaluations. Consequently, choosing a modeling language that has been previously considered in combination with metrics can be desirable.
Other architectural evaluations
This aspect considers whether the modeling language has already been used to evaluate system architectures before, preferably even in the cloud context. Such an evaluation may have been done qualitatively or quantitatively. Furthermore, if it has not been used for a general architectural evaluation, it is considered if the language has instead been utilized for a more specific evaluation, for example focusing on certain architectural related quality aspects. Analyzing an application’s availability or robustness with the help of the modeling language can be seen as an example of such a focus. Preferably, the evaluation even considered quality aspects that are part of the quality model.
Metrics
The metrics aspect analyzes whether the modeling language has been applied in a context combined with metrics or even used to obtain metrics. This could have been realized in different ways. Metrics might have been derived based on the model created with the respective modeling language, for example, by counting the occurrence of certain types of model elements. Or, the created model is transformed into a form capable of determining certain defined metrics. Alternatively, metrics may have been directly linked to specific model elements, often described as constraints or requirements. The options already considered in the literature for the respective modeling language are analyzed within this criterion. In their review, Kritikos and Skrzypek [KS19] also examined their chosen modeling languages regarding some metric-related aspects.
References
// TODO