Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Some clj-kondo support #9

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

WIP: Some clj-kondo support #9

wants to merge 1 commit into from

Conversation

sogaiu
Copy link

@sogaiu sogaiu commented Oct 14, 2020

This code has some draft clj-kondo support for the following constructs for libpython-clj:

  • import-python
  • def_pylib_fn

It also includes an attempt to handle export-symbols from tech.parallel.


To test it out:

  1. Generate a project from the template

If you have a very recent tools.deps clj installation (having -X support) the following invocation can be used to generate from the template:

clj -Sdeps '{:deps {seancorfield/clj-new {:mvn/version "1.1.228"}}}' -X:new create :template '"https://github.com/sogaiu/clj-template@1f9679b5386be5b2114972d2d161812b9886ae45"' :name fun/libpython-clj-kondo

For a not-as-recent clj installation:

clj -Sdeps '{:deps {seancorfield/clj-new {:mvn/version "0.8.6"}}}' -m clj-new.create https://github.com/sogaiu/clj-template@1f9679b5386be5b2114972d2d161812b9886ae45 fun/libpython-clj-kondo

Upon success a libpython-clj-kondo project directory should be created.

  1. Observe the effects of the clj-kondo hooks

One way to see the effects is to first replace the generated src/libpython-clj-kondo/libpython-clj-kondo.clj file with the following content:

(ns libpython-clj-kondo.libpython-clj-kondo
    (:require
              libpython-clj-kondo.python
              [tech.parallel.utils :refer [export-symbols]]
              [libpython-clj.jna.base :refer [def-pylib-fn]]
              [libpython-clj.require :refer [require-python import-python]]))

(import-python)

(builtins.list/hello)

(python.list/hello)

(comment
  (require-python 'os)
  (os/getcwd))

(export-symbols libpython-clj-kondo.python
                initialize-python!)

(def-pylib-fn PyBuffer_IsContiguousNot
  "Some doc"
  Integer
  [order byte])

Then run a recent clj-kondo on it:

$ clj-kondo --lint src/libpython-clj-kondo/libpython-clj-kondo.clj 
src/libpython-clj-kondo/libpython-clj-kondo.clj:16:4: warning: Unresolved namespace os. Are you missing a require?
linting took 16ms, errors: 0, warnings: 1

There is only one warning. Without the hooks, one would likely see something like:

$ clj-kondo --lint src/libpython-clj-kondo/libpython-clj-kondo.clj 
src/libpython-clj-kondo/libpython-clj-kondo.clj:2:16: warning: namespace libpython-clj.python is required but never used
src/libpython-clj-kondo/libpython-clj-kondo.clj:2:52: warning: #'libpython-clj.python/py. is referred but never used
src/libpython-clj-kondo/libpython-clj-kondo.clj:2:56: warning: #'libpython-clj.python/py.- is referred but never used
src/libpython-clj-kondo/libpython-clj-kondo.clj:2:61: warning: #'libpython-clj.python/py.. is referred but never used
src/libpython-clj-kondo/libpython-clj-kondo.clj:2:66: warning: #'libpython-clj.python/py* is referred but never used
src/libpython-clj-kondo/libpython-clj-kondo.clj:2:70: warning: #'libpython-clj.python/py** is referred but never used
src/libpython-clj-kondo/libpython-clj-kondo.clj:10:4: warning: Unresolved namespace os. Are you missing a require?
linting took 8ms, errors: 0, warnings: 7

Alternatively, in an editor with clj-kondo, view the file.


Note that if libpython-clj or other portion is updated in one's project, the corresponding clj-kondo hooks information may need to be updated depending on the nature of the changes in the updated dependencies.

@jjtolton
Copy link
Collaborator

@sogaiu is this still WIP status?

@sogaiu
Copy link
Author

sogaiu commented Oct 19, 2020

@jjtolton As it might affect newly created projects, I was hoping that at least one other person might try it and share how it went for them 😅

FWIW, I'm not in a hurry on this and happy to come back to it later :)

@jjtolton
Copy link
Collaborator

jjtolton commented Dec 9, 2020

@sogaui, Does the PR in the main repo clj-python/libpython-clj#135 supercede this or is it complementary?

@sogaiu
Copy link
Author

sogaiu commented Dec 9, 2020

@jjtolton The short of it is that I think at this point it's better to not have the hooks in the template.

To give some background, if as further releases to libpython-clj are made, constructs that the hooks are meant to work for change in certain ways, the particular version of hooks a user has installed need to change appropriately. At the moment (and possibly forever?) that requires user-intervention.

If the hooks were to be in the template, then all template-using folks would end up with those hooks enabled, IIUC. Although that may be convneient, it seems likely to lead to folks having hooks enabled without being aware of how to handle the hooks updating process.

With the hooks being shipped with libpython-clj itself, one doesn't automatically get hooks enabled, so the out-of-date situation should be unlikely to occur. If hooks are installed, I think it's then likely it was the user who did the installing (famous last words?), so I think they have a better chance of being able to handle an update (which is likely to be almost the same process -- possibly less steps).

On a side note, a benefit of shipping the hooks with libpython-clj itself is that it's possible to arrange for a situation where it's much easier to find the right version of hooks for libpython-clj [1].

The alternative before was to obtain hooks from: https://github.com/clj-kondo/config -- but this can lead to version mismatch problems (between libpython-clj and hooks for it) if there are ever changes to libpython-clj hooks.

AFAIK, there is no straight-forward way to dig up an appropriate version of hooks for a particular library at that repository -- the easiest thing is just to fetch the latest version, but if your project happens to be using a library version that's not matched to the latest version of the corresponding hooks, you'd need to go spelunking at the repository to try to find an appropriate version.

Sorry, this was a bit long. I hope it made sense, but if it didn't, please feel free to say so!


[1] That means ensuring that the hooks libpython-clj ships with are kept in sync with libpython-clj IIUC.

@jjtolton
Copy link
Collaborator

jjtolton commented Dec 10, 2020 via email

@sogaiu
Copy link
Author

sogaiu commented Dec 10, 2020

I think checking with @borkdude may be good.

My current understanding is that it's usually a matter of overwriting what was already there. So copying things into place like the original installation.

@borkdude
Copy link

To avoid version mismatches it's probably best to use the importing functionality as described here:

https://github.com/borkdude/clj-kondo/blob/master/doc/config.md#exporting-and-importing-configuration

This may need some documentation notes on the library side to explain to users how to use this.

@borkdude
Copy link

Oh, sorry. I wasn't aware that this project is a template. Generating the hook files as part of a template may work as well, but clj-kondo already has a nice mechanism for this which I linked in the previous comment.

@sogaiu
Copy link
Author

sogaiu commented Dec 10, 2020

FWIW, I think the PR that was a part of the libpython-clj is set up to do the importing / exporting thing. I think what I referred to as "copying" / "overwriting" above is the same. In my opinion, the "copying" step seems likely to be a conscious decision a user would make, not an automatic upgrade (see below for more on this).

That libpython-clj PR is explained here: clj-python/libpython-clj#135 (comment)

It points to a comment here: clj-kondo/config#1 (comment) which shows the command used to do the import / export thing:

clj-kondo --no-warnings --lint "$(clojure -Spath)"

As I understand it, one doesn't usually invoke clj-kondo in this way (at least I don't), which I think is a good thing because you don't "accidentally" change your configuration. So as the docs state:

Note: configuration is only copied when all of these requirements are met:

There is a .clj-kondo directory that can be used as a target.
The --no-warnings flag is used to indicate that clj-kondo is used to populate the cache.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants