Add test case to check if all ops are being verified #169
Conversation
Mgluhovskoi
added
documentation
|
I was able to implement a test case that checks if most of the operations are being verified or not, but I am not sure how to get a list of all ____s. I tried to use inspect.getmembers, but they don't show up here. Any recommendations? |
| @@ -181,3 +181,21 @@ def test_neg_dtype_constraints(test_data): | |||
| api_call_locals, namespace = _run_dtype_constraints_subtest(test_data) | |||
| if isinstance(api_call_locals[RETURN_VALUE], tp.Tensor): | |||
| assert api_call_locals[RETURN_VALUE].dtype == namespace[return_dtype] | |||
|
|
|||
|
|
|||
| operations = [obj.__qualname__ for _, obj in inspect.getmembers(tp) if inspect.isfunction(obj)] | |||
There was a problem hiding this comment.
Could you use tests.helper.get_all_tripy_interfaces here?
There was a problem hiding this comment.
It seems like tests.helper.get_all_tripy_interfaces doesn't work anymore
There was a problem hiding this comment.
Could you elaborate? We use it for our docstring testing, so it should still be working.
There was a problem hiding this comment.
I keep getting the following error when calling it:
test_dtype_constraints.py:29: in <module>
print(get_all_tripy_interfaces())
../../tripy/common/datatype.py:48: in __repr__
return cls.name
E AttributeError: type object 'BaseDtype' has no attribute 'name
There was a problem hiding this comment.
Oh this is an issue do to me trying to print the result.
There was a problem hiding this comment.
I'll fix that in #170
Co-authored-by: pranavm-nvidia <[email protected]> Signed-off-by: Mgluhovskoi <[email protected]>
| @@ -181,3 +181,63 @@ def test_neg_dtype_constraints(test_data): | |||
| api_call_locals, namespace = _run_dtype_constraints_subtest(test_data) | |||
| if isinstance(api_call_locals[RETURN_VALUE], tp.Tensor): | |||
| assert api_call_locals[RETURN_VALUE].dtype == namespace[return_dtype] | |||
|
|
|||
|
|
|||
| def get_all_possible_verif_ops(): | |||
There was a problem hiding this comment.
Another approach could be to check all public APIs, and then for classes, check their methods. We can then filter those on whether any of the type annotations include tp.Tensor.
… this commit has lots of issue and it may be easier to just to add tensor checking from scratch
| for block in tests.helper.consolidate_code_blocks(obj.__doc__) | ||
| if isinstance(block, tests.helper.DocstringCodeBlock) | ||
| ] | ||
| if blocks is None: |
There was a problem hiding this comment.
Will this ever happen? Even if nothing is found, blocks will be an empty list, which is not None.
|
Adding this in #274 |
Added one test case that checks if all ops have their data types verified. I also added a list where functions can be excluded if needed.