What is a Tensor kernel in machine learning?

Question

What is a Tensor kernel in machine learning? What is the difference between Tensor kernels with common kernels like RBF kernel? When tensor kernels are used and what their advantages and disadvantages?

Answer 1

This is more a semantic issue. A Kernel is an encapsulated function. In order to apply it to a data-tensor we need to convert the kernel to a tensor shape, but in the end it is still the same function.

• Understand tensors. They "are just high dimensional matrices".

• Understand ML Kernels. Basically a kernel is an encapsulated function. The kernel trick is used to pass a linear function over a (none-linear) kernel or filter, in order to achieve a complex none-linear function in a less complex way. This allows a linear function learning none-linear features, saving a lot of computational power, since using a polynomial function can become too expensive for large (or complicated) datasets training. I.e., we can compute linear functions using none-linear kernels to learn none-linear patterns.

So, RBF is represented as a tensor kernel (exponential gaussian kernel), but this is just the most popular kernel in machine learning to learn none-linear representations (specially used in SVM). SVR using linear and non-linear kernels.

There are other tensor kernels that you could use instead, like the Sigmoid Kernel or the polynomial kernel.