what is a DT_VARIANT tensor?

Question

The constructor of tf.data.Dataset takes an argument variant_tensor, which is only documented as:

A DT_VARIANT tensor that represents the dataset.

and

in the DatasetV2, we expect subclasses to create a variant_tensor and pass it in to the super() call.

Where can I learn what a "DT_VARIANT tensor" or a "variant_tensor" is?

Answer 1

A Variant Tensor can be a Tensor of any data type.

Some examples of Variant Tensor are shown below:

# Integer element
a = 1
# Float element
b = 2.0
# Tuple element with 2 components
c = (1, 2)
# Dict element with 3 components
d = {"a": (2, 2), "b": 3}
# Element containing a dataset
e = tf.data.Dataset.from_element(10)

Explanation about Variant Tensor or DT_Variant is shown below.

// This is an implementation of a type-erased container that can store an
// object of any type. The implementation is very similar to std::any, but has
// restrictions on the types of objects that can be stored, and eschews some of
// the fancier constructors available for std::any. An object of
// tensorflow::Variant is intended to be used as the value that will be stored
// in a tensorflow::Tensor object when its type is DT_VARIANT.
//
// tensorflow::Variant can store an object of a class that satisfies the
// following constraints:
//
// * The class is CopyConstructible.
// * The class has a default constructor.
// * It's either a protocol buffer, a tensorflow::Tensor, or defines the
// following functions:
//
//   string TypeName() const;
//   void Encode(VariantTensorData* data) const;
//   bool Decode(VariantTensorData data);

For more details, please refer TF Org Page and Github source code.