Variation test based on a example for a response

[thim81]

use-case 1

To be more concrete, we have an endpoint that is used to create a placeholder for html content. We have different types (e.g. html, image, css, script, ...) that all have some shared properties and some specific properties. Whenever we do a creation call, the request contains one of the specific types and the expected response should also contain the result for that specific type.
So when thinking about a Portman solution it would be good to be able to link the request to a specific response (e.g. by using the example names)

use-case 2

Often use OpenAPI defined examples for error responses - there might be 5 different 400 response examples)
image

As these are a set of values vs a schema, is it possible to check the response values that they are compliant.

The variation overrides can set the scenario up and we are then checking that the schema and values are compliant vs the API definition?

[thim81]

@erwincuppens & @savage-alex, you would like to use the response example to validate?

• If the schema matches the example? For this you would use the response, since the schema is always based on the response properties and never the example

• if the response matches the example? Like a content-check? Where we compare the full response with the defined example?

[savage-alex]

@erwincuppens & @savage-alex, you would like to use the response example to validate?

• If the schema matches the example? For this you would use the response, since the schema is always based on the response properties and never the example
• if the response matches the example? Like a content-check? Where we compare the full response with the defined example?

Agree that the schema is used for the response validation (such as someone sending a string in place of an int but again thats the schema validation in play)

A content check would be good to find differences between examples and real data returned.