Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

pandas, convert DataFrame to MultiIndex'ed DataFrame

I have a pandas.DataFrame that I want to convert to a MultiIndexed pandas.DataFrame.

import numpy
import pandas
import itertools

xs = numpy.linspace(0, 10, 100)
ys = numpy.linspace(0, 0.1, 20)
zs = numpy.linspace(0, 5, 200)

def func(x, y, z):
    return x * y / z

vals = list(itertools.product(xs, ys, zs))
result = [func(x, y, z) for x, y, z in vals]

# Original DataFrame.
df = pandas.DataFrame(vals, columns=['x', 'y', 'z'])
df = pandas.concat((pandas.DataFrame(result, columns=['result']), df), axis=1)

# I want to turn `df` into this `df2`.
index = pandas.MultiIndex.from_tuples(vals, names=['x', 'y', 'z'])
df2 = pandas.DataFrame(result, columns=['result'], index=index)

Note that in this example I create what I want and what I have.

So, IRL I would start with df and want to turn it into df2 (and don't have access to vals and result), how do I do this?

like image 384
johnbaltis Avatar asked Nov 09 '16 12:11

johnbaltis


1 Answers

You need set_index:

print (df2.head())
                  result
x   y   z               
0.0 0.0 0.000000     NaN
        0.025126     0.0
        0.050251     0.0
        0.075377     0.0
        0.100503     0.0

print (df.set_index(['x','y','z']).head())

                  result
x   y   z               
0.0 0.0 0.000000     NaN
        0.025126     0.0
        0.050251     0.0
        0.075377     0.0
        0.100503     0.0

If need compare both DataFrames, need replace NaN to same values, else get False:

print (df.set_index(['x','y','z']).eq(df2).all())
result    False
dtype: bool

print (np.nan == np.nan)
False

print (df.fillna(1).set_index(['x','y','z']).eq(df2.fillna(1)).all())
result    True
dtype: bool
like image 120
jezrael Avatar answered Oct 24 '22 19:10

jezrael