Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge allOf #828

Closed
wants to merge 96 commits into from
Closed

Merge allOf #828

wants to merge 96 commits into from

Conversation

reuvenharrison
Copy link
Contributor

@reuvenharrison reuvenharrison commented Aug 18, 2023

Objective

This PR introduces a function that simplifies OpenAPI specifications by removing subschemas under allOf and replacing them with a single merged (or flattened) top-level schema.
The resulting schema remains equivalent to the original one in terms of validation.
The use case we had in mind for this PR is for detecting breaking-changes which is implemented in https://github.com/Tufin/oasdiff using kin-openapi3. There may be other use cases which we did not think about.

The Merge function

  • The Merge function accepts a schema as input and returns a new equivalent schema without allOf
  • If the specification can't be merged, Merge returns an error
  • Source: openapi3/schema_merge.go

Unit testing

  • Test how Merge works for different kinds of subschemas under allOf
  • Source: openapi3/schema_merge_test.go

Validation testing

  • In order to verify equivalence of the original schema and the merged schema, we apply ValidateRequest to various inputs and verify that either the input matches both schemas or the input doesn't match both schemas, if one matches and the other doesn't, then the schemas are not equivalent.
  • Source: openapi3filter/validation_merge_allof_test.go

Development Status

  • This PR is still under active development, and not all schema fields are currently supported.
  • At present, we support the following schema fields:
    • Properties
    • AdditionalProperties
    • MinProps
    • MaxProps
    • Type
    • Format
    • Required
    • Min
    • Max
    • ExclusiveMin
    • ExclusiveMax
    • Items
    • MinItems
    • MaxItems
    • UniqueItems
    • Enum
    • MinLength
    • MaxLength
    • MultipleOf
    • Pattern: not supported because combining regex expressions requires positive lookahead which isn't supported by kin-openapi's regex engine
    • Title
    • Description
    • Not
    • OneOf and AnyOf: supported, but there might be some tricky situations where the result doesn't quite match the original specification, we are working to resolve these cases.

},
})
require.NoError(t, err)
require.Equal(t, []string{"string", "boolean"}, merged.OneOf[0].Value.Required)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should be boolean only.

},
})

require.EqualError(t, err, TypeErrorMessage)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This error is used for two different scenarios.
I recommend to use a dedicated error here.

orderMap := make(map[string]int)
orderMap[formatInt32] = 1
orderMap[formatInt64] = 2
orderMap[formatDouble] = 3
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't the order be reversed?
According to the OpenAPI Spec, Double is wider than Float:

Type Format Description
number float Floating-point numbers.
number double Floating-point numbers with double precision.

require.Equal(t, "base schema", merged.Title)
}

func TestMerge_NumericFormat(t *testing.T) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recommend to split this into 3 tests

schema := Schema{}
merged, err := Merge(schema)
require.NoError(t, err)
require.Equal(t, &schema, merged) //todo &schema
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest to clarify the TODO message or remove it

}
merged, err := Merge(schema)
require.NoError(t, err)
require.Equal(t, &schema, merged) //todo &schema
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest to clarify the TODO message or remove it

@reuvenharrison
Copy link
Contributor Author

Hi @fenollp,
Can we do anything to help with this PR?

Please note that we expect frequent tweaks on this functionality during the coming months until it is stable, so
perhaps it would be better to implement it in https://github.com/Tufin/oasdiff to speed up the process?

Thank you,
Reuven

@fenollp
Copy link
Collaborator

fenollp commented Sep 5, 2023

Hi @reuvenharrison !
I believe this should be implemented as a schema Walker. So first let's add a Walk method so a callback func can be called by the visitor. Then impl a MergeAllOf func that can be given to that visitor.
This should provide a simple and extensible API for you and other consumers of this lib (and I have other use cases for that schema walker). Also it helps cut this PR in two parts.

Does this seem reasonable to you?

I don't kind this func to change in the future. I agree its impl could leave in your repo but I'd rather the lib offers this kind of thing from upstream somewhat. But if you prefer let's have the visitor in this lib and you impl your merge in your repo and once it's stable enough for you then open a second PR with your merge impl. That's probably the easiest for everyone.

@reuvenharrison
Copy link
Contributor Author

Hi @fenollp,
Thanks for your quick reply.
We will proceed according to your suggestion above.
Regards,
Reuven

@reuvenharrison
Copy link
Contributor Author

We implemented this in oasdiff in order to answer the primary requirement ASAP:
https://github.com/Tufin/oasdiff/blob/main/ALLOF.md

Happy to assist integration/migration to this repo if anyone wants to lead this effort.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants