How to improve accuracy of a FeedForward Neural Network?

Question

I want to draw StackOverflow's logo with this Neural Network:

The NN should ideally become [r, g, b] = f([x, y]). In other words, it should return RGB colors for a given pair of coordinates. The FFNN works pretty well for simple shapes like a circle or a box. For example after several thousands epochs a circle looks like this:

Try it yourself: https://codepen.io/adelriosantiago/pen/PoNGeLw

---

However since StackOverflow's logo is far more complex even after several thousands of iterations the FFNN's results are somewhat poor:

From left to right:

1. StackOverflow's logo at 256 colors.
2. With 15 hidden neurons: The left handle never appears.
3. 50 hidden neurons: Pretty poor result in general.
4. 0.03 as learning rate: Shows blue in the results (blue is not in the orignal image)
5. A time-decreasing learning rate: The left handle appears but other details are now lost.

Try it yourself: https://codepen.io/adelriosantiago/pen/xxVEjeJ

Some parameters of interest are synaptic.Architect.Perceptron definition and learningRate value.

---

How can I improve the accuracy of this NN?

Could you improve the snippet? If so, please explain what you did. If there is a better NN architecture to tackle this type of job could you please provide an example?

Additional info:

• Artificial Neural Network library used: Synaptic.js
• To run this example in your localhost: See repository

Answer 1

By adding another layer, you get better results :

let perceptron = new synaptic.Architect.Perceptron(2, 15, 10, 3)

There are small improvements that you can do to improve efficiency (marginally): Here is my optimized code:

const width = 125
const height = 125
const outputCtx = document.getElementById("output").getContext("2d")
const iterationLabel = document.getElementById("iteration")
const stopAtIteration = 3000
let perceptron = new synaptic.Architect.Perceptron(2, 15, 10, 3)
let iteration = 0

let inputData = (() => {
  const tempCtx = document.createElement("canvas").getContext("2d")
  tempCtx.drawImage(document.getElementById("input"), 0, 0)
  return tempCtx.getImageData(0, 0, width, height)
})()

const getRGB = (img, x, y) => {
  var k = (height * y + x) * 4;
  return [
    img.data[k] / 255, // R
    img.data[k + 1] / 255, // G
    img.data[k + 2] / 255, // B
    //img.data[(height * y + x) * 4 + 3], // Alpha not used
  ]
}
const paint = () => {
  var imageData = outputCtx.getImageData(0, 0, width, height)
  for (let x = 0; x < width; x++) {
    for (let y = 0; y < height; y++) {
      var rgb = perceptron.activate([x / width, y / height])
      var k = (height * y + x) * 4;
      imageData.data[k] = rgb[0] * 255
      imageData.data[k + 1] = rgb[1] * 255
      imageData.data[k + 2] = rgb[2] * 255
      imageData.data[k + 3] = 255 // Alpha not used
    }
  }
  outputCtx.putImageData(imageData, 0, 0)

  setTimeout(train, 0)
}

const train = () => {
  iterationLabel.innerHTML = ++iteration

  if (iteration > stopAtIteration) return

  let learningRate = 0.01 / (1 + 0.0005 * iteration) // Attempt with dynamic learning rate
  //let learningRate = 0.01 // Attempt with non-dynamic learning rate
      
  for (let x = 0; x < width; x += 1) {
    for (let y = 0; y < height; y += 1) {
      perceptron.activate([x / width, y / height])
      perceptron.propagate(learningRate, getRGB(inputData, x, y))
    }
  }
  paint()
}

const startTraining = (btn) => {
  btn.disabled = true
  train()
}

EDIT : I made another CodePen with even better results:

https://codepen.io/xurei/pen/KKzWLxg

It is likely to be over-fitted BTW. The perceptron definition:

let perceptron = new synaptic.Architect.Perceptron(2, 8, 15, 7, 3)