I'm doing this to make categorical variables numbers
>>> df = pd.DataFrame({'x':['good', 'bad', 'good', 'great']}, dtype='category') x 0 good 1 bad 2 good 3 great
How can I get the mapping between the original values and the new values?
Proportions are often used to summarize categorical data and can be calculated by dividing individual frequencies by the total number of responses. In Python/pandas, df['column_name']. value_counts(normalize=True) will ignore missing data and divide the frequency of each category by the total in any category.
For categorical data you can use Pandas string functions to filter the data. The startswith() function returns rows where a given column contains values that start with a certain value, and endswith() which returns rows with values that end with a certain value.
You can create a dictionary mapping by enumerating (similar to creating a dictionary from a list by creating dictionary keys from the list indices):
dict( enumerate(df['x'].cat.categories ) ) # {0: 'bad', 1: 'good', 2: 'great'}
Alternatively, you could map the values and codes in every row:
dict( zip( df['x'].cat.codes, df['x'] ) ) # {0: 'bad', 1: 'good', 2: 'great'}
It's a little more transparent what is happening here, and arguably safer for that reason. It is also much less efficient as the length of the arguments to zip()
is len(df)
whereas the length of df['x'].cat.categories
is only the count of unique values and generally much shorter than len(df)
.
The reason Method 1 works is that the categories have type Index:
type( df['x'].cat.categories ) # pandas.core.indexes.base.Index
and in this case you look up values in an index just as you would a list.
There are a couple of ways to verify that Method 1 works. First, you can just check that a round trip retains the correct values:
(df['x'] == df['x'].cat.codes.map( dict( enumerate(df['x'].cat.categories) ) ).astype('category')).all() # True
or you can check that Method 1 and Method 2 give the same answer:
(dict( enumerate(df['x'].cat.categories ) ) == dict( zip( df['x'].cat.codes, df['x'] ) )) # True
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With