Testing changes and caching behaviour #144
Comments
|
Right now this is not supported, but PR for this would be really welcomed! |
|
mmkay, I thought so. Thanks. I will add this to the list of things I want to look at (no guarantees on a timeline :p) |
|
So on this, once you're happy with #152 and that's merged, I have one last prefactor lined up (delfick/pytest-mypy-plugins@schema/split-from-execution...delfick:pytest-mypy-plugins:separation/use-fixtures, will have an explanation of what it's doing when I make it a PR) and then finally the changes to make it possible to specify followups - delfick/pytest-mypy-plugins@separation/use-fixtures...delfick:pytest-mypy-plugins:scenarios/followups Though I have a problem I haven't quite being able to figure out where I change a file and then mypy doesn't see the change to that file. But if I do the same thing outside of pytest it does see the change. It's very weird. edit: using the power of sleeping on it, I fixed the problem delfick@89f2d52 |
|
yay, after that current PR and the next two, I think I have all the changes in a state that is ready to use, was able to make this test on my extension on django-stubs https://github.com/delfick/extended-mypy-django-plugin/actions/runs/8981236513/job/24666367526 to show one of the things I need to fix that was slowing me down so much (an edge case where an app is removed from INSTALLED_APPS but is still present on the file system) |
|
@sobolevn hello, I was wondering, I do want to eventually upstream these changes to this project, but my changes are quite extensive and your time for reviewing appears to be limited. Do you mind if I make my fork into it's own package in the meantime? |
In the beginning there was pytest-mypy-plugins as used by django-stubs and I needed the ability to test how mypy incremental and fine-grained caches interact with my mypy plugin. I created this issue on pytest-mypy-plugins asking if it were possible typeddjango/pytest-mypy-plugins#144 The answer was that this functionality wasn't in the plugin and PRs were welcome I made a few PRs including typeddjango/pytest-mypy-plugins#152 which was then the base for a couple more branches. Ultimately it was a lot of change and getting that merged wasn't a thing. After many months I decided to take what I had done in those branches and what I had here and create pytest-typing-runner as a more sophisticated, tested, documented, typed version of what I had written across those branches and this repo. The project should be able to handle more than just mypy, but that has yet to be done. Also I want to make it so that pytest-typing-runner can be used in django-stubs, which involves a few more additional features. The docs for the project are also in need of some more effort. But the pytest-typing-runner plugin is now fully tested and I'm happy with it's API, so I've released a version and made this project use it
In the beginning there was pytest-mypy-plugins as used by django-stubs and I needed the ability to test how mypy incremental and fine-grained caches interact with my mypy plugin. I created this issue on pytest-mypy-plugins asking if it were possible typeddjango/pytest-mypy-plugins#144 The answer was that this functionality wasn't in the plugin and PRs were welcome I made a few PRs including typeddjango/pytest-mypy-plugins#152 which was then the base for a couple more branches. Ultimately it was a lot of change and getting that merged wasn't a thing. After many months I decided to take what I had done in those branches and what I had here and create pytest-typing-runner as a more sophisticated, tested, documented, typed version of what I had written across those branches and this repo. The project should be able to handle more than just mypy, but that has yet to be done. Also I want to make it so that pytest-typing-runner can be used in django-stubs, which involves a few more additional features. The docs for the project are also in need of some more effort. But the pytest-typing-runner plugin is now fully tested and I'm happy with it's API, so I've released a version and made this project use it Also it appears that I need more control over when pytest-typing-runner loads vs my extension to it in this project and I've had to turn off pytest autoload to make this work, I will look at this next
In the beginning there was pytest-mypy-plugins as used by django-stubs and I needed the ability to test how mypy incremental and fine-grained caches interact with my mypy plugin. I created this issue on pytest-mypy-plugins asking if it were possible typeddjango/pytest-mypy-plugins#144 The answer was that this functionality wasn't in the plugin and PRs were welcome I made a few PRs including typeddjango/pytest-mypy-plugins#152 which was then the base for a couple more branches. Ultimately it was a lot of change and getting that merged wasn't a thing. After many months I decided to take what I had done in those branches and what I had here and create pytest-typing-runner as a more sophisticated, tested, documented, typed version of what I had written across those branches and this repo. The project should be able to handle more than just mypy, but that has yet to be done. Also I want to make it so that pytest-typing-runner can be used in django-stubs, which involves a few more additional features. The docs for the project are also in need of some more effort. But the pytest-typing-runner plugin is now fully tested and I'm happy with it's API, so I've released a version and made this project use it Also it appears that I need more control over when pytest-typing-runner loads vs my extension to it in this project and I've had to turn off pytest autoload to make this work, I will look at this next
In the beginning there was pytest-mypy-plugins as used by django-stubs and I needed the ability to test how mypy incremental and fine-grained caches interact with my mypy plugin. I created this issue on pytest-mypy-plugins asking if it were possible typeddjango/pytest-mypy-plugins#144 The answer was that this functionality wasn't in the plugin and PRs were welcome I made a few PRs including typeddjango/pytest-mypy-plugins#152 which was then the base for a couple more branches. Ultimately it was a lot of change and getting that merged wasn't a thing. After many months I decided to take what I had done in those branches and what I had here and create pytest-typing-runner as a more sophisticated, tested, documented, typed version of what I had written across those branches and this repo. The project should be able to handle more than just mypy, but that has yet to be done. Also I want to make it so that pytest-typing-runner can be used in django-stubs, which involves a few more additional features. The docs for the project are also in need of some more effort. But the pytest-typing-runner plugin is now fully tested and I'm happy with it's API, so I've released a version and made this project use it
In the beginning there was pytest-mypy-plugins as used by django-stubs and I needed the ability to test how mypy incremental and fine-grained caches interact with my mypy plugin. I created this issue on pytest-mypy-plugins asking if it were possible typeddjango/pytest-mypy-plugins#144 The answer was that this functionality wasn't in the plugin and PRs were welcome I made a few PRs including typeddjango/pytest-mypy-plugins#152 which was then the base for a couple more branches. Ultimately it was a lot of change and getting that merged wasn't a thing. After many months I decided to take what I had done in those branches and what I had here and create pytest-typing-runner as a more sophisticated, tested, documented, typed version of what I had written across those branches and this repo. The project should be able to handle more than just mypy, but that has yet to be done. Also I want to make it so that pytest-typing-runner can be used in django-stubs, which involves a few more additional features. The docs for the project are also in need of some more effort. But the pytest-typing-runner plugin is now fully tested and I'm happy with it's API, so I've released a version and made this project use it
|
@sobolevn Hello, just thought I'd let you know that I ended up creating a fork of this plugin with all of my changes https://pytest-typing-runner.readthedocs.io/en/latest/ The documentation still needs quite a bit of improvement tbh, but I've ran out of time to devote to that for now. It currently only supports mypy but it shouldn't be difficult to make it possible to make it support running pyright or any other type checker. And makes it possible and easy to create scenarios where we test what happens when the code changes against a warm cache. It also is built around making it possible to write these tests like any other pytest test rather than constraining to a yaml definition (ultimately it provides a I've also created an example of how it could be used in django-stubs without major surgery to those tests if you're interested in that - https://github.com/typeddjango/django-stubs/compare/master...delfick:django-stubs:using-pytest-typing-runner?expand=1 An example of using it to write tests with followup scenarios can be found in https://github.com/delfick/extended-mypy-django-plugin/tree/main/tests Either way, I want to say thanks for pytest-mypy-plugins! It gave me a massive head start with that work, I really appreciate everything you do around mypy and django stubs! |
Hello,
I'm not sure if I'm missing something obvious, but is there patterns for testing how my plugin interacts with mypy caching for both mypy and dmypy when code is changed?
So say I make a test case with code in a particular way and it passes mypy. I want a test that show it still passes mypy if I change that code in a particular way. Same with dmypy.
The text was updated successfully, but these errors were encountered: