excludeForOperations not behaving as expected

[deformedonion]

Hello,

This may be a misunderstanding of how this all works on my part, can you clarify what exactly the behavior of excludeForOperations is with regards to contract tests? I have the following route in my spec

"paths": {
    "/products": {
      "get": {
        "tags": ["product-controller"],
        "summary": "getProduct",
        "operationId": "getProductUsingGET",
        "responses": {
          "200": {
            "description": "OK",
            "content": {
              "application/json": {
                "schema": {
                  "type": "array",
                  "items": { "$ref": "#/components/schemas/Product" }
                }
              }
            }
          },
          "400": {
            "description": "Bad Request"
          },
          "401": { "description": "Unauthorized" },
          "403": { "description": "Forbidden" },
          "404": { "description": "Not Found" }
        }
      },
      "post": {
        "tags": ["product-controller"],
        "summary": "addProduct",
        "operationId": "addProductUsingPOST",
        "requestBody": {
          "content": {
            "application/json": {
              "schema": { "$ref": "#/components/schemas/ProductDto" }
            }
          }
        },
        "responses": {
          "201": { "description": "Created" },
          "400": {
            "description": "Bad Request"
          },
          "401": { "description": "Unauthorized" },
          "403": { "description": "Forbidden" },
          "404": { "description": "Not Found" }
        }
      }
    }
  }

with the following config

 {
  "version": 1.0,
   "tests": {
       "contractTests": [
           {
           "openApiOperation": "*::/products",
           "excludeForOperations": ["addProductUsingPOST", "POST::/products"]
           }   
       ]
   }
}

but when generating contract tests, the post route appears in the collection. is that expected and i'm just misunderstanding what this does, or am i doing something wrong? ideally, i'd like to have that post request excluded from the collection

[challakamal]

I am having same issue, its not excluding in contractTests. It's working as expected in variationTests. I am on Portman version 1.23.1

[tillig]

I did some debugging/research and I think the bug is in Portman.convertToPostmanCollection

The spec is taken and initially converted to a Postman test suite, after which the tests themselves are generated and added to that initial generated suite's "folder structure." However, the excluded operations are not counted when this initial suite is generated. While all of the other "substitute request variables" types of operations happen to update all the generated tests (which is why the tests actually look like they could run), there's a missing step where the initially generated set of tests should be looking at excludeForOperations and pulling those out.

[thim81]

hi @deformedonion

If you want to exclude the operation from Postman all together, we provide built-in filtering options.

Have a look at the filtering example

@tillig is correct and a expert in Portman config.

The flow is:

• OpenAPI -> Filtered OpenAPI -> Convert to Postman -> Inject tests ... -> export Postman collection