F# Pick function based on input?

Question

I have the following functions

let private sigmoid (z:float) =
    1.0 / (1.0 + exp(-z))
let private sigmoidM (z : Matrix<float>) : Matrix<float> =
    z.Map (fun x -> sigmoid(x))
let private sigmoidV (z:Vector<float>) =
    z.Map(fun x -> sigmoid(x))

I would like to just have sigmoid and it execute against a scalar, vector or matrix depending on the input.

That said, this function needs to be extremely performant as it is in the most critical section of loops. Any insight on how to do this? Feel free to post how with caution it will be slow if it will be slow.

Answer 1

You can use standard .NET overloading:

open MathNet.Numerics.LinearAlgebra

type Sigmoid() = class end with
    static member sigmoid (z:float) = 1.0 / (1.0 + exp(-z))
    static member sigmoid (z:Matrix<float>) = z.Map (fun x -> Sigmoid.sigmoid(x))
    static member sigmoid (z:Vector<float>) = z.Map (fun x -> Sigmoid.sigmoid(x))

// Usage
let x = Sigmoid.sigmoid 4.3
let y = Sigmoid.sigmoid (matrix [[1.0; 2.0]; [3.0; 4.0]])
let z = Sigmoid.sigmoid (vector  [1.0; 2.0])

// Results
val x : float = 0.9866130822
val y : Matrix<float> = 
          DenseMatrix 2x2-Double
            0.731059  0.880797
            0.952574  0.982014
val z : Vector<float> = seq [0.7310585786; 0.880797078]

This will not affect performance since the overload resolution is done at compile-time.

Not happy with standard .NET overloading? Don't want to code the function as a member? Do you want to make it more generic (accepting also float32) and extensible to other types?

Use static type constraints:

type Sigmoid() = class end with
    static member Sigmoid (_:Sigmoid, z:float  ) = 1.0  / (1.0  + exp(-z))
    static member Sigmoid (_:Sigmoid, z:float32) = 1.0f / (1.0f + exp(-z))

let inline _sigmoid (s:'Sigmoid) (x:'T) :'T =
    ((^T or ^Sigmoid) : (static member Sigmoid : 'Sigmoid * 'T -> 'T) (s, x))

let inline sigmoid x = _sigmoid (Sigmoid()) x 

type Sigmoid  with
    static member inline Sigmoid (_:Sigmoid, z:Matrix<'T>) = z.Map (fun x -> sigmoid x)
    static member inline Sigmoid (_:Sigmoid, z:Vector<'T>) = z.Map (fun x -> sigmoid x)

// Usage
let x = sigmoid 4.3
let y = sigmoid (matrix [[ 1.0; 2.0 ];[ 3.0; 4.0 ]])
let z = sigmoid (vector [ 1.0; 2.0 ])

let x' = sigmoid 4.3f
let y' = sigmoid (matrix [[1.0f; 2.0f];[ 3.0f; 4.0f]])
let z' = sigmoid (vector [ 1.0f; 2.0f])

UPDATE

Note that @TheInnerLight points out in the comments that for your specific sigmoid function you can also write:

let inline sigmoid z = 
    LanguagePrimitives.GenericOne / (LanguagePrimitives.GenericOne + exp(-z))

and that would work for float and float32

This would eventually work for vector and matrix as well, depending on their implementation.

That would be a better solution for your specific case if all operations negate, divide and exp are already generic over those types and they all support the GenericOne.

Unfortunately as of today MathNet doesn't implement GenericOne and exp for Matrix and Vector in such a way.