Partial Test Oracle in Model Transformation Testing

Writing test oracles for model transformations is a difficult task. First, oracles must deal with models which are complex data. Second, the tester cannot always predict the expected value of all the properties of the output model produced by a transformation. In this paper, we propose an approach to create efficient oracles for validating part of the produced output model.

In this approach we presume that output models can be divided into two parts, a predictable part and a non-predictable one. After identifying the latter, we use it to create a filter. Before providing a (partial) verdict, the oracle compares actual output model with the expected output model, returning a difference model, and uses the filter to discard the differences related to the unpredictable part. The approach infers the unpredictable part from the model transformation specification, or from older output models, in the case of regression testing.

The approach is supported by a tool to build such partial oracles. We run several experiments writing partial oracles to validate output models returned by two model transformations. We validate our proposal comparing the effectiveness and complexity of partial oracles with oracles based on full model comparisons and contracts.