Menu
I am currently trying to design a Node.js code that would be able to tell if an image is blurry or not. In order to achieve that, I took my inspiration from this question. So what I need to do is compute a matrix laplacian (is that how we say ?) and then compute variance.
I have no problem doing it with OpenCV (using opencv4nodejs) :
# load image
_cvImg = cv.imread _file
# get grayscale image
_cvImgGray =_cvImg.bgrToGray()
# Compute laplacian
_laplacian = _cvImgGray.laplacian(cv.CV_8U)
# Get the standard deviation
_meanStdDev = _laplacian.meanStdDev()
_stddevarray = _meanStdDev.stddev.getDataAsArray()
_stdDeviation = _stddevarray[0]
# Get the variation
_variation = Math.pow(_stdDeviation, 2)
But now, I am using Tensorflow.js and it is really less easy... Here what I tried to do:
# load image
_cvImg = cv.imread _file
#convert frame to a tensor
try
_data = new Uint8Array(_frame.cvtColor(cv.COLOR_BGR2RGB).getData().buffer)
_tensorFrame = tfjs.tensor3d(_data, [_frame.rows, _frame.cols, 3])
catch _err
@log.error "Error instantiating tensor !!!"
@log.error _err.message
# get grayscale image
_grayscaleFrame = _tensorFrame.mean(2).expandDims(2)
# prepare convolution to get laplacian
laplaceFilter = tfjs.tensor2d([[0,1,0], [1,-4,1], [0,1,0]])
laplaceFilter3D = laplaceFilter.expandDims(2)
# get laplacian
_laplacian = tfjs.conv1d _tensorFrame, laplaceFilter3D, 1, 'same'
# get standard deviation
_standardDeviation = tfjs.moments(_laplacian2).variance.buffer().values[0]
# get variance
_variance = _standardDeviation * _standardDeviation
# dispose tensor to avoid memeory leaks
_tensorFrame.dispose()
Not so surprisingly, the above code doesn't work. I know that my convolution should be two dimensional (tf.conv2d) instead of 1-D (tf.conv1d) since I am working on an image. If I look at tf.conv2d in API, I can see this signature:
tf.conv2d (x, filter, strides, pad, dataFormat?, dilations?, dimRoundingMode?)
And filter should be a tf.Tensor4D ! But I have no idea how the following filter matrix could be converted to tf.Tensor4D
1
1 4 1
1
Am I doing something wrong ? How can I get laplacian of a matrix in Tensorflow.js ? i.e How can I perform a simple convolution between my tensor2d representing my image and the above filter ?
641
The convolution can be performed over the 2d tensor by first expanding its dimension. Likewise for the filter by using tf.reshape
After the computation, tf.squeeze will remove dimensions of shape 1, thus returning a 2d tensor
const x = tf.tensor2d(Array.from({length: 16}, (_, k) => k+1), [4, 4]); // input image 2d
const filter = tf.tensor2d([0, 1, 0, 1, 0, 1, 0, 1, 0], [3, 3]) // input filter 2d
x.reshape([1, ...x.shape, 1]).conv2d(filter.reshape([...filter.shape, 1, 1]), 1, 'valid').squeeze().print() // conv<html>
<head>
<!-- Load TensorFlow.js -->
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/[email protected]"> </script>
</head>
<body>
</body>
</html>If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
