How to build a Convolution Neural Net in Azure Machine Learning?

Question

Someone should add "net#" as a tag. I'm trying to improve my neural network in Azure Machine Learning Studio by turning it into a convolution neural net using this tutorial:

https://gallery.cortanaintelligence.com/Experiment/Neural-Network-Convolution-and-pooling-deep-net-2

The differences between mine and the tutorial is I'm doing regression with 35 features and 1 label and they're doing classification with 28x28 features and 10 labels.

I start with the basic and 2nd example and get them working with:

input Data [35];

hidden H1 [100]
    from Data all;

hidden H2 [100]
    from H1 all;

output Result [1] linear
    from H2 all;

Now the transformation to convolution I misunderstand. In the tutorial and documentation here: https://docs.microsoft.com/en-us/azure/machine-learning/machine-learning-azure-ml-netsharp-reference-guide it doesn't mention how the node tuple values are calculated for the hidden layers. The tutorial says:

hidden C1 [5, 12, 12]
  from Picture convolve {
    InputShape  = [28, 28];
    KernelShape = [ 5,  5];
    Stride      = [ 2,  2];
    MapCount = 5;
  }

hidden C2 [50, 4, 4]
   from C1 convolve {
     InputShape  = [ 5, 12, 12];
     KernelShape = [ 1,  5,  5];
     Stride      = [ 1,  2,  2];
     Sharing     = [ F,  T,  T];
     MapCount = 10;
  }

Seems like the [5, 12, 12] and [50,4,4] pop out of no where along with the KernalShape, Stride, and MapCount. How do I know what values are valid for my example? I tried using the same values, but it didn't work and I have a feeling since he has a [28,28] input and I have a [35], I need tuples with 2 integers not 3.

I just tried with random values that seem to correlate with the tutorial:

const { T = true; F = false; }

input Data [35];

hidden C1 [7, 23]
  from Data convolve {
    InputShape  = [35];
    KernelShape = [7];
    Stride      = [2];
    MapCount = 7;
  }

hidden C2 [200, 6]
   from C1 convolve {
     InputShape  = [ 7, 23];
     KernelShape = [ 1,  7];
     Stride      = [ 1,  2];
     Sharing     = [ F,  T];
     MapCount = 14;
  }

hidden H3 [100]
  from C2 all;

output Result [1] linear
  from H3 all;

Right now it seems impossible to debug because the only error code Azure Machine Learning Studio ever gives is:

Exception":{"ErrorId":"LibraryException","ErrorCode":"1000","ExceptionType":"ModuleException","Message":"Error 1000: TLC library exception: Exception of type 'Microsoft.Numerics.AFxLibraryException' was thrown.","Exception":{"Library":"TLC","ExceptionType":"LibraryException","Message":"Exception of type 'Microsoft.Numerics.AFxLibraryException' was thrown."}}}Error: Error 1000: TLC library exception: Exception of type 'Microsoft.Numerics.AFxLibraryException' was thrown. Process exited with error code -2

Lastly my setup is

Thanks for the help!

Answer 1

The correct network definition for 35-column length input with given kernels and strides would be following:

const { T = true; F = false; }

input Data [35];

hidden C1 [7, 15]
  from Data convolve {
    InputShape  = [35];
    KernelShape = [7];
    Stride      = [2];
    MapCount = 7;
  }

hidden C2 [14, 7, 5]
   from C1 convolve {
     InputShape  = [ 7, 15];
     KernelShape = [ 1,  7];
     Stride      = [ 1,  2];
     Sharing     = [ F,  T];
     MapCount = 14;
  }

hidden H3 [100]
  from C2 all;

output Result [1] linear
  from H3 all;

First, the C1 = [7,15]. The first dimension is simply the MapCount. For the second dimension, the kernel shape defines the length of the "window" that's used to scan the input columns, and the stride defines how much it moves at each step. So the kernel windows would cover columns 1-7, 3-9, 5-11,...,29-35, yielding the second dimension of 15 when you tally the windows.

Next, the C2 = [14,7,5]. The first dimension is again the MapCount. For the second and third dimension, the 1-by-7 kernel "window" has to cover the input size of 7-by-15, using steps of 1 and 2 along corresponding dimensions.

Note that you could specify C2 hidden layer shape of [98,5] or even [490], if you wanted to flatten the outputs.