WARNING: both StatsBase and Distances export "pairwise!"; uses of it in module Clustering must be qualified

[alifarhat40]

Hello, I am trying to run the code here using the same example:
I was told to post an issue here as it might be an issue with Clustering.jl.

https://github.com/niclaspopp/MultivariateDiscretization.jl/blob/master/MultivariateDiscretization/Doc%20v0.1.0.ipynb

However, I get the following error:

`ipd = greedy_IPD(Testdata,dims,35)

`WARNING: both StatsBase and Distances export "pairwise"; uses of it in module Clustering must be qualified
WARNING: both StatsBase and Distances export "pairwise!"; uses of it in module Clustering must be qualified
UndefVarError: pairwise not defined
Stacktrace:
[1] _kmeans!(X::Adjoint{Float64, Vector{Float64}}, weights::Nothing, centers::Matrix{Float64}, maxiter::Int64, tol::Float64, displevel::Int64, distance::Distances.SqEuclidean, rng::Random._GLOBAL_RNG)
@ Clustering C:\Users\alifa.julia\packages\Clustering\lI5MF\src\kmeans.jl:141
[2] kmeans!(X::Adjoint{Float64, Vector{Float64}}, centers::Matrix{Float64}; weights::Nothing, maxiter::Int64, tol::Float64, display::Symbol, distance::Distances.SqEuclidean, rng::Random._GLOBAL_RNG)
@ Clustering C:\Users\alifa.julia\packages\Clustering\lI5MF\src\kmeans.jl:71
[3] kmeans(X::Adjoint{Float64, Vector{Float64}}, k::Int64; weights::Nothing, init::Symbol, maxiter::Int64, tol::Float64, display::Symbol, distance::Distances.SqEuclidean, rng::Random._GLOBAL_RNG)
@ Clustering C:\Users\alifa.julia\packages\Clustering\lI5MF\src\kmeans.jl:114
[4] kmeans(X::Adjoint{Float64, Vector{Float64}}, k::Int64)
@ Clustering C:\Users\alifa.julia\packages\Clustering\lI5MF\src\kmeans.jl:95
[5] greedy_IPD(M::Matrix{Float64}, ndim::Int64, T::Int64, disc::Symbol, skip::Vector{Any}, limit::Int64)
@ MultivariateDiscretization C:\Users\alifa.julia\packages\MultivariateDiscretization\D0sKt\src\IPD.jl:225
[6] greedy_IPD(M::Matrix{Float64}, ndim::Int64, T::Int64)
@ MultivariateDiscretization C:\Users\alifa.julia\packages\MultivariateDiscretization\D0sKt\src\IPD.jl:188
[7] top-level scope
@ In[4]:1

[alifarhat40]

@jiahao @pluskid @johnmyleswhite @devmotion
Clustering.jl has

using StatsBase
using Distances

which is bad practice for a package, it should specifically import individual functions they want, not implicitly import all names in those modules. That's what is causing this problem. The issue won't be fixed without editing Clustering.jl.

[alyst]

@alifarhat40 This issue was resolved quite some time ago (see e.g. #235) -- both StatsBase.jl and Distances.jl packages import StatsAPI.jl, and pairwise() is defined there.
Clustering.jl tests (including kmeans) pass on CI, and on my local machine.
But maybe you are using some old versions of the packages, which should not be supported by the Clustering.jl.
If you tell me your project config, I can see if "compat" entries for Clustering.jl have to be updated.

[alifarhat40]

Status C:\Users\alifa\.julia\environments\v1.8\Project.toml
⌅ [336ed68f] CSV v0.7.10
⌅ [aaaa29a8] Clustering v0.14.3
[5ae59095] Colors v0.12.10
⌅ [a93c6f00] DataFrames v0.21.8
⌅ [0c46a032] DifferentialEquations v6.18.0
⌅ [31c24e10] Distributions v0.23.8
⌃ [497a8b3b] DoubleFloats v1.2.6
[7073ff75] IJulia v1.24.2
[b964fa9f] LaTeXStrings v1.3.1
[dd9280a6] MultivariateDiscretization v0.1.0
⌅ [6f286f6a] MultivariateStats v0.7.0
[d96e819e] Parameters v0.12.3
⌅ [91a5bcdd] Plots v1.39.0
⌃ [be4d8f0f] Quadmath v0.5.6
⌅ [2913bbd2] StatsBase v0.33.21
[37e2e46d] LinearAlgebra
Info Packages marked with ⌃ and ⌅ have new versions available, but those with ⌅ are restricted by compatibility constraints from upgrading. To see why use status --outdated

[alifarhat40]

I tired ]up but still get the same error.

Status C:\Users\alifa\.julia\environments\v1.8\Project.toml
⌅ [336ed68f] CSV v0.7.10
⌅ [aaaa29a8] Clustering v0.14.4
[5ae59095] Colors v0.12.10
⌅ [a93c6f00] DataFrames v0.21.8
⌅ [0c46a032] DifferentialEquations v6.18.0
⌅ [31c24e10] Distributions v0.23.8
⌃ [497a8b3b] DoubleFloats v1.2.6
[7073ff75] IJulia v1.24.2
[b964fa9f] LaTeXStrings v1.3.1
[dd9280a6] MultivariateDiscretization v0.1.0
⌅ [6f286f6a] MultivariateStats v0.7.0
[d96e819e] Parameters v0.12.3
[91a5bcdd] Plots v1.40.3
⌃ [be4d8f0f] Quadmath v0.5.6
⌅ [2913bbd2] StatsBase v0.33.21
[37e2e46d] LinearAlgebra
Info Packages marked with ⌃ and ⌅ have new versions available, but those with ⌅ are restricted by compatibility constraints from upgrading. To see why use status --outdated

[alyst]

Thanks. That's the packages that you depend on directly. Can you check which version of the Distances.jl is in your Manifest.toml?
Note that you use Clustering.jl v0.14.4, which is rather old (the current one is v0.15.7), you may consider resolving compatibility constraints to upgrade to the newer version.
If your compatibility constraints are essential, I can try releasing v0.14.x fix that would limit compatibility to older Distances.jl or StatsBase.jl versions that do not have pairwise() name clash.

[alifarhat40]

[[deps.Distances]]
deps = ["LinearAlgebra", "Statistics"]
git-tree-sha1 = "a5b88815e6984e9f3256b6ca0dc63109b16a506f"
uuid = "b4f34e82-e78d-54a5-968a-f98e89d6e8f7"
version = "0.9.2"

[alifarhat40]

How can I resolve compatibility constraints? Any tutorial I can follow? I am new to Julia. Thanks.

[alyst]

How can I resolve compatibility constraints? Any tutorial I can follow? I am new to Julia. Thanks.

Unfortunately, I am not aware of a nice tutorial. You can start with what Julia suggests, i.e. running ]status --outdated.
It will show you the dependency tree of the constrained packages alongside with the specific version constraints that are imposed.
So potentially the package that causes the constraints would be somewhere in the root of the tree.
But honestly, I find this output a little bit cryptic and often not straightforward to interpret.

Just as a sanity check -- check that your Project.toml does not contain explicit version constraints, and that you are not using some specific branches or development versions of the packages that were not updated for a while (in Manifest.toml you would see that the entry for such package contains path ="...".

Another thing, it looks like you are using Julia 1.8. The current one is 1.10, and I suspect many packages have switched to 1.9+, in particular because v1.9 had introduced package extensions.
So if you can upgrade to the recent Julia, maybe your dependency constraints would be resolved automatically.

[alifarhat40]

Upgrading Julia on Windows is also a mess. No clear instructions on how to do that either. I tried several instructions. No luck.

[alyst]

juliaup is working quite nicely to me also on windows. the only inconvenience is that you are using julia version-specific project environment, you would have to replicate it for the new version. you may want to consider switching to an environment in a specific folder (see activate command).