Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to get k means cluster for 1D data?

I have a csv file which looks like below

date                       mse                                                  
2018-02-11                 14.34
2018-02-12                 7.24
2018-02-13                 4.5
2018-02-14                 3.5
2018-02-16                 12.67
2018-02-21                 45.66
2018-02-22                 15.33
2018-02-24                 98.44
2018-02-26                 23.55
2018-02-27                 45.12
2018-02-28                 78.44
2018-03-01                 34.11
2018-03-05                 23.33
2018-03-06                 7.45
...                        ...

Now I want to get two clusters for the mse values so that I know what values lies to which cluster and their mean.

Now since I do not have any other set of values apart from mse (I have to provide X and Y), I would like to use just mse values to get a k means cluster.For now for the other set of values, I pass it as range which is of same size as no of mse values.This is what I did

from sklearn.cluster import KMeans
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D

df = pd.read_csv("generate_csv/all_data_device.csv", parse_dates=["date"])
f1 = df['mse'].values
# generate another list
f2 = list(range(0, len(f1)))
X = np.array(list(zip(f1, f2)))
kmeans = KMeans(n_clusters=2).fit(X)
labels = kmeans.predict(X)
# Centroid values
centroids = kmeans.cluster_centers_
#print(centroids)

fig = plt.figure()
ax = Axes3D(fig)
ax.scatter(X[:, 0], X[:, 1], c=labels)
ax.scatter(centroids[:, 0], centroids[:, 1], marker='*', c='#050505', s=1000)
plt.title('K Mean Classification')
plt.show()

How can I just use the mse values to get the k means cluster?I am aware of the function 'reshape()' but not quite sure how to use it?

like image 592
Souvik Ray Avatar asked May 01 '18 08:05

Souvik Ray


1 Answers

Demo:

In [29]: kmeans = KMeans(n_clusters=2)

In [30]: df['label'] = kmeans.fit_predict(df[['mse']])
# NOTE:                     ---->            ^     ^

In [31]: df
Out[31]:
          date    mse  label
0   2018-02-11  14.34      0
1   2018-02-12   7.24      0
2   2018-02-13   4.50      0
3   2018-02-14   3.50      0
4   2018-02-16  12.67      0
5   2018-02-21  45.66      0
6   2018-02-22  15.33      0
7   2018-02-24  98.44      1
8   2018-02-26  23.55      0
9   2018-02-27  45.12      0
10  2018-02-28  78.44      1
11  2018-03-01  34.11      0
12  2018-03-05  23.33      0
13  2018-03-06   7.45      0

plotting:

In [64]: ax = df[df['label']==0].plot.scatter(x='mse', y='label', s=50, color='white', edgecolor='black')

In [65]: df[df['label']==1].plot.scatter(x='mse', y='label', s=50, color='white', ax=ax, edgecolor='red')
Out[65]: <matplotlib.axes._subplots.AxesSubplot at 0xfa42be0>

In [66]: plt.scatter(kmeans.cluster_centers_.ravel(), [0.5]*len(kmeans.cluster_centers_), s=100, color='green', marker='*')
Out[66]: <matplotlib.collections.PathCollection at 0xfabf208>

enter image description here

like image 95
MaxU - stop WAR against UA Avatar answered Sep 28 '22 07:09

MaxU - stop WAR against UA