Google Inceptionism: obtain images by class

Question

In the famous Google Inceptionism article, http://googleresearch.blogspot.jp/2015/06/inceptionism-going-deeper-into-neural.html they show images obtained for each class, such as banana or ant. I want to do the same for other datasets.

The article does describe how it was obtained, but I feel that the explanation is insufficient.

There's a related code https://github.com/google/deepdream/blob/master/dream.ipynb

but what it does is to produce a random dreamy image, rather than specifying a class and learn what it looks like in the network, as shown in the article above.

Could anyone give a more concrete overview, or code/tutorial on how to generate images for specific class? (preferably assuming caffe framework)

Answer 1

I think this code is a good starting point to reproduce the images Google team published. The procedure looks clear:

1. Start with a pure noise image and a class (say "cat")
2. Perform a forward pass and backpropagate the error wrt the imposed class label
3. Update the initial image with the gradient computed at the data layer

There are some tricks involved, that can be found in the original paper.

It seems that the main difference is that Google folks tried to get a more "realistic" image:

By itself, that doesn’t work very well, but it does if we impose a prior constraint that the image should have similar statistics to natural images, such as neighboring pixels needing to be correlated.