European Union / European Regional Development Fund / Investing in your future

# X-Road: Operational Monitoring Testing Strategy

Version: 0.8

Date Version Description Author
0.5 Initial version
23.01.2017 0.6 Added license text, table of contents and version history Sami Kallio
05.03.2018 0.7 Added terms and abbreviations reference and moved terms to term doc Tatu Repo
05.02.2020 0.8 Update information about the test suites. Ilkka Seppälä

# Table of Contents

# License

This document is licensed under the Creative Commons Attribution-ShareAlike 3.0 Unported License. To view a copy of this license, visit

# 1 Introduction

# 1.1 Purpose

In this document, we define the testing strategy for the development project of the operational monitoring components of the X-Road system.

A detailed testing plan will be delivered as a separate document in [TEST-OPMON].

# 1.2 Terms and Abbreviations

See X-Road terms and abbreviations documentation [TA-TERMS].

# 1.3 References

HD_1 -- Hanke Lisa 1: X-tee monitooringu tehniline kirjeldus,
HD_2 -- Hanke Lisa 2: Mittefunktsionaalsed nõuded,
HD_4 -- Hanke Lisa 4: Testimise korraldus,
ARC-OPMOND -- Cybernetica AS. X-Road: Operational Monitoring Daemon Architecture. Document ID: ARC-OPMOND.
PP -- Cybernetica AS. Tööde kirjeldus koos ajakavaga,
REC-OPMON -- Cybernetica AS. X-Road Operational Monitoring: Requirements. Document ID: REC-OPMON. Location: project XTEE6 repository docs directory Service monitoring/Requirements.
UC-OPMON -- Cybernetica AS. X-Road: Operational Monitoring Daemon Use Case Model. Document ID: UC-OPMON.
TEST-OPMON -- Cybernetica AS. X-Road: Operational Monitoring Testing Plan. Document ID: TEST-OPMON.
IG-SS -- Cybernetica AS. X-Road: Security Server Installation Guide. Document ID: IG-SS.
TA-TERMS -- X-Road Terms and Abbreviations. Document ID: TA-TERMS.

# 2 Requirements Relevant to Testing

The functional requirements of the monitoring system are described in [REC-OPMON].
The architecture of the system is described in [ARC-OPMOND].
The items of data that are collected and forwarded by the monitoring system, are defined in [HD_1].
The required paths of data exchange are defined in [HD_1].
The requirements of access control are defined in [HD_1].
The proper default configuration of the system is defined in [HD_1].
The non-functional requirements of the monitoring system are described in [REC-OPMON] and [HD_2].

# 3 Testability of the System

The operational monitoring system is a machine-to-machine data exchange system, and the events that are relevant to monitoring can be triggered programmatically, once the necessary configuration has been carried out. Thus, the system lends itself very well to automated testing.

Regular X-Road message exchange as well as monitoring data exchange via all the required paths can be simulated for both integration and load testing. Because all the relevant configuration will be implemented as text files, modifying the configuration of the monitoring system can be scripted easily, if necessary.

# 4. Types of Testing Used

The required types of testing are defined in [HD_4]. In the following sections, all the required types are covered.

# 4.1 Integration Testing

The integration tests will be used for testing the required functionality of the security servers and the operational monitoring daemon working together, while regular X-Road requests as well as operational monitoring and health data requests are handled. The goal is to cover the main functionality with automated tests. Manual tests will be carried out by the development team as needed, as well as during acceptance testing if necessary. The descriptions of manual tests will be provided in [TEST-OPMON].

# 4.2 Unit Testing

Generally, unit tests are used for testing the behaviour of single units of code (functions, methods) with clearly defined inputs and outputs.

The required coverage and scope of unit tests have not been defined for this project. Thus, unit tests will be written for units of code that implement protocols, conversion between data formats etc, including tests against invalid or unexpected input. Such units of code will be isolated for testability.

For larger components that are used for control logic, system-wide data exchange as well as configuration, unit tests are not justified and will not be written. The behaviour and error handling of such components will be covered by automated or manual integration tests.

Unit tests will not be written for third-party protocol implementations (such as JMXMP).

# 5 Tools, Languages and Libraries

The integration tests will be implemented in Python 3. During the Elaboration Phase, the approach and tools for describing the integration test steps will be selected. Also, the libraries required for networking, parsing of messages etc, will be selected during the Elaboration Phase.

The unit tests will be programmed in Java 8, using the JUnit library.

# 6 Management and Source Control of Automated Tests

All the source code and the necessary data and configuration files (or samples where necessary) of the integration, load and unit tests will be committed to the main Git development repository of the project. The tests and test data, once these are implemented, will be included in the pull request made at the end of each relevant iteration of the project.

# 7 Setup and Management of the Testing Environment. Continuous Integration

The testing environment and continuous integration setup during development will closely mimic the required target testing environment (as described in [HD_4]) with some additions. The relevant configuration files and documentation will be committed to the main Git development repository for simple migration to the target testing system.

# 7.1 The Servers Used for Automated Testing

The automated testing environment details can be found in document src/systemtest/op-monitoring/xrd-opmon-tests/

A SoapUI-based mock server will be used for providing the X-Road services necessary for testing the system. The mock service along with usage instructions can be found in the source repository of the project at src/systemtest/op-monitoring/xrd-mock-soapui.