Skip to content

Evaluation Criteria Catalog

In order to evaluate different technological approaches concerning the Saga pattern, some criteria have to be defined against which the evaluation can be performed. The following sections consider several areas of interest related to characteristics of microservices and the Saga pattern as well as some quality attributes of the ISO/IEC 25010 Quality Models1. For each area, the aspects that an evaluator should examine are explained and described. Some of the described criteria have been taken from the previous paper published by Dürr et al. [3]. The aim is to create a criteria catalog that can be used to analyze other technologies in this context.

Portability

Another important aspect is whether and to what extent the implementation technology supports deploying the implementation to different platforms and environments. Popular options here are nowadays containers or deploying the application directly to the cloud. Therefore, the following criteria consider the support of these.

Containerization

This criterion evaluates if support is given for containerizing the application, for example using Docker4, DockerCompose5 or Kubernetes6. This can include providing Docker Images on DockerHub7 for required infrastructure services or sample Docker or Kubernetes files.

Cloud deployment

Investigating whether extra support is provided to deploy the application to one or more specific cloud providers. For example, such support could be documentation or a guide explaining how to deploy the application to a specific cloud.

References

[1] N. Alshuqayran, N. Ali, and R. Evans, "A Systematic Mapping Study in Microservice Architecture." IEEE Computer Society, 2016, pp. 44–51. [Online]. Available: https://dx.doi.org/10.1109/SOCA.2016.15

[2] S. Newman, Building Microservices – Designing Fine–Grained Systems, 1st ed. O’Reilly Media, Inc., 2015, ISBN: 9781491950357.

[3] K. Dürr, R. Lichtenthäler, and G. Wirtz, "An Evaluation of Saga Pattern Implementation Technologies," in Proceedings of the 13th European Workshop on Services and their Composition (ZEUS 2021), Bamberg, Germany, February 25–26, 2021, ser. CEUR Workshop Proceedings, vol. 2839. CEUR-WS.org, 2021, pp. 74–82. [Online]. Available: http://ceur-ws.org/Vol-2839/paper12.pdf

[4] D. Cruz, T. Wieland, and A. Ziegler, "Evaluation Criteria for Free/Open Source Software Products Based on Project Analysis," Software Process: Improvement and Practice, vol. 11, no. 2, pp. 107–122, 2006. [Online]. Available: https://doi.org/10.1002/spip.257

[5] J. P. Confino and P. A. Laplante, "An Open Source Software Evaluation Model," Int. J. Strateg. Inf. Technol. Appl., vol. 1, no. 1, pp. 60–77, 2010. [Online]. Available: https://doi.org/10.4018/jsita.2010101505

[6] T. Cerny, M. J. Donahoo, and M. Trnka, "Contextual Understanding of Microservice Architecture: Current and Future Directions," ACM SIGAPP Applied Computing Review, vol. 17, no. 4, pp. 29–45, 2018. [Online]. Available: https://dx.doi.org/10.1145/3183628.3183631

[7] O. Zimmermann, "Microservices Tenets," Computer Science – Research and Development, vol. 32, no. 3–4, pp. 301–310, 2016. [Online]. Available: https://dx.doi.org/10.1007/s00450-016-0337-0

  1. https://iso25000.com/index.php/en/iso-25000-standards/iso-25010?start=0, last accessed 2021-07-06 ↩

  2. https://zipkin.io/, last accessed 2021-07-09 ↩

  3. https://opentracing.io/, last accessed 2021-06-06 ↩

  4. https://www.docker.com/, last accessed: 2021-06-28 ↩

  5. https://docs.docker.com/compose/, last accessed: 2021-06-28 ↩

  6. https://kubernetes.io/, last accessed: 2021-06-28 ↩

  7. https://hub.docker.com/, last accessed: 2021-06-28 ↩

  8. https://docs.github.com/en/get-started/quickstart/fork-a-repo, last accessed: 2021-06-27 ↩

Last update: 2022-02-15
Back to top