Skip to content

Evaluation Criteria Catalog

In order to evaluate different technological approaches concerning the Saga pattern, some criteria have to be defined against which the evaluation can be performed. The following sections consider several areas of interest related to characteristics of microservices and the Saga pattern as well as some quality attributes of the ISO/IEC 25010 Quality Models1. For each area, the aspects that an evaluator should examine are explained and described. Some of the described criteria have been taken from the previous paper published by Dürr et al. [3]. The aim is to create a criteria catalog that can be used to analyze other technologies in this context.


Security

Considering Security aspects is also important when implementing the Saga pattern. It might not be desirable that every service has the ability to start a Saga or that services communicate in a non–encrypted way. Therefore, ensuring and configuring communication security is important, especially since external services might also be included in a Saga.

Encrypted communication

This criterion analyzes if the communication between the orchestrator and participants is encrypted and, if not, whether the developer can easily activate it. For example, a manual change of the communication mechanism from HTTP to HTTPS.

Authentication support

Investigating whether the technology offers the possibility that services have to authenticate themselves to the orchestrator, for example, in order to be allowed to participate in a Saga.

Authorization support

This criterion extends the Authentication support criterion by checking if there also exists the option to grant or restrict access to certain functions or resources. An example could be that only one specific service is allowed to start Sagas, whereas the others can only participate in it.


References

[1] N. Alshuqayran, N. Ali, and R. Evans, "A Systematic Mapping Study in Microservice Architecture." IEEE Computer Society, 2016, pp. 44–51. [Online]. Available: https://dx.doi.org/10.1109/SOCA.2016.15

[2] S. Newman, Building Microservices – Designing Fine–Grained Systems, 1st ed. O’Reilly Media, Inc., 2015, ISBN: 9781491950357.

[3] K. Dürr, R. Lichtenthäler, and G. Wirtz, "An Evaluation of Saga Pattern Implementation Technologies," in Proceedings of the 13th European Workshop on Services and their Composition (ZEUS 2021), Bamberg, Germany, February 25–26, 2021, ser. CEUR Workshop Proceedings, vol. 2839. CEUR-WS.org, 2021, pp. 74–82. [Online]. Available: http://ceur-ws.org/Vol-2839/paper12.pdf

[4] D. Cruz, T. Wieland, and A. Ziegler, "Evaluation Criteria for Free/Open Source Software Products Based on Project Analysis," Software Process: Improvement and Practice, vol. 11, no. 2, pp. 107–122, 2006. [Online]. Available: https://doi.org/10.1002/spip.257

[5] J. P. Confino and P. A. Laplante, "An Open Source Software Evaluation Model," Int. J. Strateg. Inf. Technol. Appl., vol. 1, no. 1, pp. 60–77, 2010. [Online]. Available: https://doi.org/10.4018/jsita.2010101505

[6] T. Cerny, M. J. Donahoo, and M. Trnka, "Contextual Understanding of Microservice Architecture: Current and Future Directions," ACM SIGAPP Applied Computing Review, vol. 17, no. 4, pp. 29–45, 2018. [Online]. Available: https://dx.doi.org/10.1145/3183628.3183631

[7] O. Zimmermann, "Microservices Tenets," Computer Science – Research and Development, vol. 32, no. 3–4, pp. 301–310, 2016. [Online]. Available: https://dx.doi.org/10.1007/s00450-016-0337-0



Last update: 2022-02-15
Back to top