How to create a file like the MNIST dataset

Question

So i have a bunch of images (PNG) and their names can be used as labels. Is there a tool or an example on how i can take those images and create a dataset like the MNIST IDX one.

There are a lot of examples on how to read them but no examples on how to create them.

Any help would be greatly appreciated!

Answer 1

This can help.

The -images- files were written using the function save-idx3-ubyte and the -labels- files with the function save-idx1-ubyte. The functions in questions are defined in lush/lsh/libidx/idx-io.lsh The file format is described in the section "IDX file format for compilable I/O" which is under "standard libraries" -> "Tensor/Matrix/Scalar/Vector libraries" -> "IDX reading and writing".

The 32-bit ints in the header of the files may be byte-swapped.

-- Yann

http://sourceforge.net/p/lush/mailman/message/28879896/