Dynamic clustering for points on a torus

[Aspro89]

Hi,

Let us suppose I have a long sequence of snapshots of a system of 1000 points that move randomly on a torus (e.g., the interval [0, 1] with periodic boundaries) and form clusters. At the beginning a few clusters are formed, and eventually they all merge to one big cluster.
What is the best algorithm to find clusters that evolve in time? I certainly need the number of clusters, but possibly also their positions, width, and weight as quantities that evolve in time.

Background: the system is an interacting particle system of the McKean-Vlasov type. The point particles move because of a mean-field interaction among them and Brownian noise.

Thanks for your help!
Alberto

[MaxHalford]

Hey there @Aspro89! I'm sorry but we're not a consulting company, we can't provide such answers. We can however with specific issues/bugs you encounter when using River. Have you tried River's cluster module?

[Aspro89]

Hi MaxHalford!
Thanks, and I'm sorry about the question: I'm too inexperienced.

I've tried to use the cluster module, but at the moment I fail in understanding the simple examples in the documentation. For instance, in the DenStream page, I don't understand the data set:
X = [ [-1, -0.5], [-1, -0.625], [-1, -0.75], [-1, -1], [-1, -1.125], [-1, -1.25], [-1.5, -0.5], [-1.5, -0.625], [-1.5, -0.75], [-1.5, -1], [-1.5, -1.125], [-1.5, -1.25], [1, 1.5], [1, 1.75], [1, 2], [4, 1.25], [4, 1.5], [4, 2.25], [4, 2.5], [4, 3], [4, 3.25], [4, 3.5], [4, 3.75], [4, 4], ]
``
Is the first index of the array X time? Then: are the data two-dimensional? It seems that the points in 2D don't move, but new points are added at each time step; right? Why does the algorithm calculate two clusters (when I see at least three/four of them)?
It seems that the algorithm keeps the values of the variable X at all times, and then loops over it through stream.iter_array(X). It is possible to avoid saving all values of X, and to do the clustering online where the values of X are generated?
Moreover, I couldn't find in the documentation what the feature dictionary exactly mean.

I apologise... I think this happens because I'm not used to such kind of algorithms. So the real question could be: where should one start to be able to understand the cluster module?

[MaxHalford]

Hey there. Just another question: have you tried batch clustering methods from scikit-learn? I'm trying to get a full picture of your situation :)

[Aspro89]

Yes! From sklearn.cluster, I've tried (H)DBSCAN and OPTICS. Then I also tried (Bayesian)GaussianMixture from sklearn.mixture. I can deal with all of them.

[MaxHalford]

I've tried to use the cluster module, but at the moment I fail in understanding the simple examples in the documentation. For instance, in the DenStream page, I don't understand the data set:
X = [ [-1, -0.5], [-1, -0.625], [-1, -0.75], [-1, -1], [-1, -1.125], [-1, -1.25], [-1.5, -0.5], [-1.5, -0.625], [-1.5, -0.75], [-1.5, -1], [-1.5, -1.125], [-1.5, -1.25], [1, 1.5], [1, 1.75], [1, 2], [4, 1.25], [4, 1.5], [4, 2.25], [4, 2.5], [4, 3], [4, 3.25], [4, 3.5], [4, 3.75], [4, 4], ]

Each record represents a 2D point. The time is implicit: it depends in which order you send in the points.

Is the first index of the array X time? Then: are the data two-dimensional? It seems that the points in 2D don't move, but new points are added at each time step; right? Why does the algorithm calculate two clusters (when I see at least three/four of them)?

The clustering algorithm has given two clusters, but it might give 3/4 with different parameters. No clustering algorithm is perfect by default.

It seems that the algorithm keeps the values of the variable X at all times, and then loops over it through stream.iter_array(X). It is possible to avoid saving all values of X, and to do the clustering online where the values of X are generated?

It's up to you to decide how you want to process the data. In the example, we've kept all the samples in a list. But that's just for demo. You could also loop over your own data.

I apologise... I think this happens because I'm not used to such kind of algorithms. So the real question could be: where should one start to be able to understand the cluster module?

https://hoanganhngo610.github.io/river-clustering.kdd.2022/