Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How can I tell if a dataframe is of mixed type?

I want to assign values to the diagonal of a dataframe. The fastest way I can think of is to use numpy's np.diag_indices and do a slice assignment on the values array. However, the values array is only a view and ready to accept assignment when a dataframe is of a single dtype

Consider the dataframes d1 and d2

d1 = pd.DataFrame(np.ones((3, 3), dtype=int), columns=['A', 'B', 'C'])
d2 = pd.DataFrame(dict(A=[1, 1, 1], B=[1., 1., 1.], C=[1, 1, 1]))

d1

   A  B  C
0  0  1  1
1  1  0  1
2  1  1  0

d2

   A    B  C
0  1  1.0  1
1  1  1.0  1
2  1  1.0  1

Then let's get our indices

i, j = np.diag_indices(3)

d1 is of a single dtype and therefore, this works

d1.values[i, j] = 0
d1

   A  B  C
0  0  1  1
1  1  0  1
2  1  1  0

But not on d2

d2.values[i, j] = 0
d2

   A    B  C
0  1  1.0  1
1  1  1.0  1
2  1  1.0  1

I need to write a function and make it fail when df is of mixed dtype. How do I test that it is? Should I trust that if it is, this assignment via the view will always work?

like image 586
piRSquared Avatar asked Oct 17 '17 15:10

piRSquared


Video Answer


2 Answers

You could use internal _is_mixed_type method

In [3600]: d2._is_mixed_type
Out[3600]: True

In [3601]: d1._is_mixed_type
Out[3601]: False

Or, check unique dtypes

In [3602]: d1.dtypes.nunique()>1
Out[3602]: False

In [3603]: d2.dtypes.nunique()>1
Out[3603]: True

A bit of de-tour, is_mixed_type checks how blocks are consolidated.

In [3618]: len(d1.blocks)>1
Out[3618]: False

In [3619]: len(d2.blocks)>1
Out[3619]: True

In [3620]: d1.blocks    # same as d1.as_blocks()
Out[3620]:
{'int32':    A  B  C
 0  0  1  1
 1  1  0  1
 2  1  1  0}

In [3621]: d2.blocks
Out[3621]:
{'float64':      B
 0  1.0
 1  1.0
 2  1.0, 'int64':    A  C
 0  1  1
 1  1  1
 2  1  1}
like image 161
Zero Avatar answered Sep 27 '22 22:09

Zero


def check_type(df):
  return len(set(df.dtypes)) == 1

or

 def check_type(df):
   return df.dtypes.nunique() == 1
like image 27
galaxyan Avatar answered Sep 27 '22 21:09

galaxyan