Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

top-level venv doesn't work #465

Open
vanschelven opened this issue Jan 28, 2025 · 2 comments · May be fixed by #468
Open

top-level venv doesn't work #465

vanschelven opened this issue Jan 28, 2025 · 2 comments · May be fixed by #468
Assignees
Labels
P2 major: an upcoming release research-needed type: bug

Comments

@vanschelven
Copy link

vanschelven commented Jan 28, 2025

Describe the bug

When the virtualenv is the same as the project-dir, fawltydeps doesn't work

To Reproduce

When the venv is external, everything is fine:

klaas@pop-os:/tmp$ git clone [email protected]:simplejson/simplejson.git
Cloning into 'simplejson'...
[..]
klaas@pop-os:/tmp$ python -m venv externalvenv
klaas@pop-os:/tmp$ . externalvenv/bin/activate
(externalvenv) klaas@pop-os:/tmp$ pip install fawltydeps
Collecting fawltydeps
  Using cached fawltydeps-0.18.0-py3-none-any.whl (64 kB)
[..]
(externalvenv) klaas@pop-os:/tmp$ cd simplejson/
(externalvenv) klaas@pop-os:/tmp/simplejson$ fawltydeps 
[..]
WARNING:fawltydeps.limited_eval:Failed to parse assignment of 'cmdclass': Call(func=Name(id='dict', ctx=Load()), args=[], keywords=[keyword(arg='test', value=Name(id='TestCommand', ctx=Load()))]) from dict(test=TestCommand) @ setup.py:100

These imports appear to be undeclared dependencies:
- 'setuptools'

For a more verbose report re-run with the `--detailed` option.

Approach with "top level venv", however:

klaas@pop-os:/tmp/simplejson$ python -m venv .
klaas@pop-os:/tmp/simplejson$ . bin/activate
(simplejson) klaas@pop-os:/tmp/simplejson$ pip install fawltydeps
Collecting fawltydeps
  Using cached fawltydeps-0.18.0-py3-none-any.whl (64 kB)
[..]
(simplejson) klaas@pop-os:/tmp/simplejson$ fawltydeps 

No undeclared or unused dependencies detected.

Being explicit doesn't help:

(simplejson) klaas@pop-os:/tmp/simplejson$ fawltydeps --pyenv .

No undeclared or unused dependencies detected.

Expected behavior

I expect this to work equally well as when using a venv outside of the project-dir, or a deeply nested venv

Environment

  • Linux (pop-os 22.04)
  • Version of the code: FawltyDeps v0.18.0
@jherland
Copy link
Member

Thanks for reporting! To be honest this is the first time I've seen top-level venv, and I was not aware that keeping your source tree "inside" a virtualenv, like this, was even an option.

I seem to remember that we have some code in FawltyDeps for automatically excluding the contents of virtualenvs when we search for code and dependency declarations. This is to avoid FawltyDeps analyzing and reporting problems with code that happened to be installed in a virtualenv, but that is not part of your project. I suspect that this check now regards your entire source tree as being "inside" a virtualenv and therefore excludes it...

Will look into this as time permits.

@jherland jherland added P2 major: an upcoming release type: bug research-needed labels Jan 28, 2025
@vanschelven
Copy link
Author

I have no idea how (un)canonical my approach is, it's a habit I've picked up a loooong time ago, and it has never bitten me before. Also, because venv does this (name-of-venv) thing with your prompt, it's actually quite a good match since it shows the name of the relevant project rather than (venv) in this approach.

Happy to hear more, though this is certainly not a blocker for me.

@jherland jherland self-assigned this Jan 28, 2025
jherland added a commit that referenced this issue Jan 28, 2025
This sample project has a virtualenv in the same directory as the
project itself. Currently this causes FawltyDeps to find zero code and
zero dependencies in the project.
jherland added a commit that referenced this issue Jan 28, 2025
Issue #465 highlights a peculiar case where the top-level project
directory is also itself the root of a virtualenv. Our traversal code
will skip any (given or detected) Python environment, under the
assumption that we never want to analyze code/deps from a package
installed within. However, we here have a special case where -- from the
traversal code's POV -- we have found a pyenv rooted at a directory that
also appears in the given settings.code and/or settings.deps (by default
settings.code, settings.deps and settings.pyenvs are all ".").

Work around this special case as follows: When a Python environment is
found rooted at a path that is _also_ directly given in either
settings.code or settings.deps, then we _don't_ want to skip traversal
of the entire directory, rather we only skip traversal of the specific
subdirectories that form the actual contents of the environment:

 - Any package dirs found by validate_pyenv_source() (i.e. any
   "site-packages" dirs).
 - The "bin/" (or "Scripts\" on Windows) subdir where we expect to find
   any tools installed into the environment.

We could probably extend this handling to _any_ Python environment
(including those found after we start the traversal proper), but I'm not
100% sure that this won't trigger false positives where some weird
Python environment (e.g. a Conda or poetry2nix environment) happens to
contain extra "stuff" missing from the above list that would adversely
affect our analysis.
@jherland jherland linked a pull request Jan 28, 2025 that will close this issue
@jherland jherland linked a pull request Jan 28, 2025 that will close this issue
jherland added a commit that referenced this issue Jan 28, 2025
This sample project has a virtualenv in the same directory as the
project itself. Currently this causes FawltyDeps to find zero code and
zero dependencies in the project.
jherland added a commit that referenced this issue Jan 28, 2025
Issue #465 highlights a peculiar case where the top-level project
directory is also itself the root of a virtualenv. Our traversal code
will skip any (given or detected) Python environment, under the
assumption that we never want to analyze code/deps from a package
installed within. However, we here have a special case where -- from the
traversal code's POV -- we have found a pyenv rooted at a directory that
also appears in the given settings.code and/or settings.deps (by default
settings.code, settings.deps and settings.pyenvs are all ".").

Work around this special case as follows: When a Python environment is
found rooted at a path that is _also_ directly given in either
settings.code or settings.deps, then we _don't_ want to skip traversal
of the entire directory, rather we only skip traversal of the specific
subdirectories that form the actual contents of the environment:

 - Any package dirs found by validate_pyenv_source() (i.e. any
   "site-packages" dirs).
 - The "bin/" (or "Scripts\" on Windows) subdir where we expect to find
   any tools installed into the environment.

We could probably extend this handling to _any_ Python environment
(including those found after we start the traversal proper), but I'm not
100% sure that this won't trigger false positives where some weird
Python environment (e.g. a Conda or poetry2nix environment) happens to
contain extra "stuff" missing from the above list that would adversely
affect our analysis.
jherland added a commit that referenced this issue Feb 21, 2025
This sample project has a virtualenv in the same directory as the
project itself. Currently this causes FawltyDeps to find zero code and
zero dependencies in the project.
jherland added a commit that referenced this issue Feb 21, 2025
Issue #465 highlights a peculiar case where the top-level project
directory is also itself the root of a virtualenv. Our traversal code
will skip any (given or detected) Python environment, under the
assumption that we never want to analyze code/deps from a package
installed within. However, we here have a special case where -- from the
traversal code's POV -- we have found a pyenv rooted at a directory that
also appears in the given settings.code and/or settings.deps (by default
settings.code, settings.deps and settings.pyenvs are all ".").

Work around this special case as follows: When a Python environment is
found rooted at a path that is _also_ directly given in either
settings.code or settings.deps, then we _don't_ want to skip traversal
of the entire directory, rather we only skip traversal of the specific
subdirectories that form the actual contents of the environment:

 - Any package dirs found by validate_pyenv_source() (i.e. any
   "site-packages" dirs).
 - The "bin/" (or "Scripts\" on Windows) subdir where we expect to find
   any tools installed into the environment.

We could probably extend this handling to _any_ Python environment
(including those found after we start the traversal proper), but I'm not
100% sure that this won't trigger false positives where some weird
Python environment (e.g. a Conda or poetry2nix environment) happens to
contain extra "stuff" missing from the above list that would adversely
affect our analysis.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P2 major: an upcoming release research-needed type: bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants