Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Tensorflow One Hot Encoder?

Does tensorflow have something similar to scikit learn's one hot encoder for processing categorical data? Would using a placeholder of tf.string behave as categorical data?

I realize I can manually pre-process the data before sending it to tensorflow, but having it built in is very convenient.

like image 979
Robert Graves Avatar asked Nov 12 '15 21:11

Robert Graves


People also ask

What is a one-hot encoded tensor?

One hot tensor is a Tensor in which all the values at indices where i =j and i!= j is same. Method Used: one_hot: This method accepts a Tensor of indices, a scalar defining depth of the one hot dimension and returns a one hot Tensor with default on value 1 and off value 0. These on and off values can be modified.

What function do we use to create one-hot encoded arrays of the labels in Tensorflow?

The function you will be calling is tf. ones() .

Is hot encoding required for deep learning?

One Hot Encoding is a common way of preprocessing categorical features for machine learning models. Preprocessing data is an essential step before building a Deep Learning model. When creating a deep learning project, it is not always that we come across clean and well-formatted data.

Does neural network require one hot encoding?

one_hot is simply an operation, so we'll need to create a Neural Network layer that uses this operation in order to include the One Hot Encoding logic with the actual model prediction logic.


2 Answers

As of TensorFlow 0.8, there is now a native one-hot op, tf.one_hot that can convert a set of sparse labels to a dense one-hot representation. This is in addition to tf.nn.sparse_softmax_cross_entropy_with_logits, which can in some cases let you compute the cross entropy directly on the sparse labels instead of converting them to one-hot.

Previous answer, in case you want to do it the old way: @Salvador's answer is correct - there (used to be) no native op to do it. Instead of doing it in numpy, though, you can do it natively in tensorflow using the sparse-to-dense operators:

num_labels = 10  # label_batch is a tensor of numeric labels to process # 0 <= label < num_labels  sparse_labels = tf.reshape(label_batch, [-1, 1]) derived_size = tf.shape(label_batch)[0] indices = tf.reshape(tf.range(0, derived_size, 1), [-1, 1]) concated = tf.concat(1, [indices, sparse_labels]) outshape = tf.pack([derived_size, num_labels]) labels = tf.sparse_to_dense(concated, outshape, 1.0, 0.0) 

The output, labels, is a one-hot matrix of batch_size x num_labels.

Note also that as of 2016-02-12 (which I assume will eventually be part of a 0.7 release), TensorFlow also has the tf.nn.sparse_softmax_cross_entropy_with_logits op, which in some cases can let you do training without needing to convert to a one-hot encoding.

Edited to add: At the end, you may need to explicitly set the shape of labels. The shape inference doesn't recognize the size of the num_labels component. If you don't need a dynamic batch size with derived_size, this can be simplified.

Edited 2016-02-12 to change the assignment of outshape per comment below.

like image 161
dga Avatar answered Sep 19 '22 14:09

dga


tf.one_hot() is available in TF and easy to use.

Lets assume you have 4 possible categories (cat, dog, bird, human) and 2 instances (cat, human). So your depth=4 and your indices=[0, 3]

import tensorflow as tf res = tf.one_hot(indices=[0, 3], depth=4) with tf.Session() as sess:     print sess.run(res) 

Keep in mind that if you provide index=-1 you will get all zeros in your one-hot vector.

Old answer, when this function was not available.

After looking though the python documentation, I have not found anything similar. One thing that strengthen my belief that it does not exist is that in their own example they write one_hot manually.

def dense_to_one_hot(labels_dense, num_classes=10):   """Convert class labels from scalars to one-hot vectors."""   num_labels = labels_dense.shape[0]   index_offset = numpy.arange(num_labels) * num_classes   labels_one_hot = numpy.zeros((num_labels, num_classes))   labels_one_hot.flat[index_offset + labels_dense.ravel()] = 1   return labels_one_hot 

You can also do this in scikitlearn.

like image 27
Salvador Dali Avatar answered Sep 16 '22 14:09

Salvador Dali