new shape and old shape must have the same number of elements

Question

For learning purpose, I am using Tensorflow.js, and I experience an error while trying to use the fit method with a batched dataset (10 by 10) to learn the process of batch training.

I have got a few images 600x600x3 that I want to classify (2 outputs, either 1 or 0)

Here is my training loop:

  const batches = await loadDataset()

  for (let i = 0; i < batches.length; i++) {
    const batch = batches[i]
    const xs = batch.xs.reshape([batch.size, 600, 600, 3])
    const ys = tf.oneHot(batch.ys, 2)

    console.log({
      xs: xs.shape,
      ys: ys.shape,
    })
    // { xs: [ 10, 600, 600, 3 ], ys: [ 10, 2 ] }

    const history = await model.fit(
      xs, ys,
      {
        batchSize: batch.size,
        epochs: 1
      }) // <----- The code throws here

    const loss = history.history.loss[0]
    const accuracy = history.history.acc[0]

    console.log({ loss, accuracy })
  }

Here is how I define the dataset

const chunks = chunk(examples, BATCH_SIZE)

const batches = chunks.map(
  batch => {
    const ys = tf.tensor1d(batch.map(e => e.y), 'int32')
    const xs = batch
      .map(e => imageToInput(e.x, 3))
      .reduce((p, c) => p ? p.concat(c) : c)
    return { size: batch.length, xs , ys }
  }
)

Here is the model:

const model = tf.sequential()
model.add(tf.layers.conv2d({
  inputShape: [600, 600, 3],
  kernelSize: 60,
  filters: 50,
  strides: 20,
  activation: 'relu',
  kernelInitializer: 'VarianceScaling'
}))
model.add(tf.layers.maxPooling2d({
  poolSize: [20, 20],
  strides: [20, 20]
}))
model.add(tf.layers.conv2d({
  kernelSize: 5,
  filters: 100,
  strides: 20,
  activation: 'relu',
  kernelInitializer: 'VarianceScaling'
}))

model.add(tf.layers.maxPooling2d({
  poolSize: [20, 20],
  strides: [20, 20]
}))
model.add(tf.layers.flatten())
model.add(tf.layers.dense({
  units: 2,
  kernelInitializer: 'VarianceScaling',
  activation: 'softmax'
}))

I get an error during the first iteration in the for-loop, from the .fit which is the following:

Error: new shape and old shape must have the same number of elements.
    at Object.assert (/Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/util.js:36:15)
    at reshape_ (/Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/ops/array_ops.js:271:10)
    at Object.reshape (/Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/ops/operation.js:23:29)
    at Tensor.reshape (/Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/tensor.js:273:26)
    at Object.derB [as $b] (/Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/ops/binary_ops.js:32:24)
    at _loop_1 (/Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/tape.js:90:47)
    at Object.backpropagateGradients (/Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/tape.js:108:9)
    at /Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/engine.js:334:20
    at /Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/engine.js:91:22
    at Engine.scopedRun (/Users/person/nn/node_modules/@tensorflow/tfjs-core/dist/engine.js:101:23)

I don't know what to understand from that and found no documentation or help on that specific error, any idea?

Answer 1

The issue of the model lies in the way the convolution is applied along with the maxPooling

The first layer is doing a convolution of kernelSize 60 with a strides of [20, 20] and 50 filters. The output of this layer will have the approximate shape [600 / 20, 600 / 20, 50] = [30, 30, 50]

The max pooling is applied with a stride of [20, 20]. The output of this layer will also have the approximate shape [30 / 20, 30 / 20, 50] =[1, 1, 50 ]

From this step, the model can no longer perform a convolution with a kernelSize 5. For the kernel shape [5, 5] is bigger than the input shape [1, 1] resulting in the error that is thrown. The only convolution the model can perform is that of a kernel whose size is 1. Obviously, that convolution will output the input without any transformation.

The same rule applies to the last maxPooling whose poolingSize cannot be different from 1, otherwise an error will be thrown.

Here is a snippet:

const model = tf.sequential()
model.add(tf.layers.conv2d({
  inputShape: [600, 600, 3],
  kernelSize: 60,
  filters: 50,
  strides: 20,
  activation: 'relu',
  kernelInitializer: 'VarianceScaling'
}))
model.add(tf.layers.maxPooling2d({
  poolSize: [20, 20],
  strides: [20, 20]
}))
model.add(tf.layers.conv2d({
  kernelSize: 1,
  filters: 100,
  strides: 20,
  activation: 'relu',
  kernelInitializer: 'VarianceScaling'
}))

model.add(tf.layers.maxPooling2d({
  poolSize: 1,
  strides: [20, 20]
}))
model.add(tf.layers.flatten())
model.add(tf.layers.dense({
  units: 2,
  kernelInitializer: 'VarianceScaling',
  activation: 'softmax'
}))

model.compile({optimizer: 'sgd', loss: 'meanSquaredError'});
model.fit(tf.ones([10, 600, 600, 3]), tf.ones([10, 2]), {batchSize: 4});

model.predict(tf.ones([1, 600, 600, 3])).print()

<html>
  <head>
    <!-- Load TensorFlow.js -->
    <script src="https://cdn.jsdelivr.net/npm/@tensorflow/[email protected]"> </script>
  </head>

  <body>
  </body>
</html>