Stumpy & Clustering

[Jalanjii]

Hi,

I am interested in clustering with MP measure. I can't find anything on that with Stumpy, is that true or I have missed something?
If you know any other resources please if you can share them.

[seanlaw]

@Jalanjii Welcome to the STUMPY community and thank you for your question. So, STUMPY does not provide time series clustering out-of-the-box but based on the Matrix Profile XII paper on MPdist, you should be able to create a pairwise distance matrix between all of your time series using stumpy.mpdist:

import numpy as np
import stumpy
import hdbscan


n = 1000
Ts = [np.random.rand(n) for i in range(80)]  # Generate 80 random time series
for i in range(20):
    # Generate 20 time series that look similar to `Ts[0]` but with added noise and a roll
    noise = np.random.uniform(0.0, 0.1, size=n)
    Ts.append(np.roll(Ts[0], i * 100) + noise)

# Compute the pairwise distances
distance_matrix = np.zeros((len(Ts), len(Ts)))
m = 50
for i in range(len(Ts)):
    for j in range(i+1, len(Ts)):
        distance = stumpy.mpdist(Ts[i], Ts[j], m)
        distance_matrix[i, j] = distance
        distance_matrix[j, i] = distance

And then you can feed this into your preferred clustering algorithm that can accept a distance matrix as input like hdbscan:

clusterer = hdbscan.HDBSCAN(metric='precomputed', min_cluster_size=3)
clusterer.fit(distance_matrix)
clusterer.labels_

[seanlaw]

Have you had a chance to read the Matrix Profile XII paper? This paper should clarify what an "mpdist" is (i.e., it is not simply the "matrix profile" between Ts[i] and Ts[j]).

[Jalanjii]

Indeed. I was actually going through it. Thank you for your replies.

[Jalanjii]

Hi Sean, I have a single time series, compared to what I had before. I want to build the Distance Matrix as you did in this tutorial, https://stumpy.readthedocs.io/en/latest/Tutorial_The_Matrix_Profile.html, there is a manual way to do that as it is explained, but the Stumpy way I don't find it as far as my reading has gone. For mpdist, it is useless I think in this case, right?

[seanlaw]

Hi Sean, I have a single time series, compared to what I had before. I want to build the Distance Matrix as you did in this tutorial, https://stumpy.readthedocs.io/en/latest/Tutorial_The_Matrix_Profile.html, there is a manual way to do that as it is explained, but the Stumpy way I don't find it as far as my reading has gone. For mpdist, it is useless I think in this case, right?

@Jalanjii If this is unrelated to clustering multiple time series, can you please start a new discussion?

[Jalanjii]

Ok, I just did. #651

[JaKasb]

The following paper contains a clustering method.
https://link.springer.com/article/10.1007/s10618-019-00668-6
However, I could not reproduce the clustering method.

[Jalanjii]

Thanks!