Prototyping neural networks

Question

from your experience, which is the most effective approach to implement artificial neural networks prototypes? It is a lot of hype about R (free, but I didn't work with it) or Matlab (not free), another possible choice is to use a language like C++/Java/C#. The question is mainly targeting the people that tried to test some neural networks architectures or learning algorithms.

If your choice is to use a programming language different from the three mentioned above, can you tell me their names and some explanations concerning your choice (excepting: this is the only/most used language known by me).

Thanks.

Answer 1

Since neural networks are used a lot by scientists, and not so much by programmers, choose a programming language that targets scientists and provides good libraries for neural networks.

Python is a reasonable choice since it is widely used by scientists. Two distributions to start with are

http://www.pythonxy.com/

http://code.enthought.com/

These Python distributions include a lot of add-on modules that are not in the standard Python library but which are very useful for the type of programming that scientists do. Searching the Python package index comes up with a few neural network packages that may be OK if you are just learning about neural networks.

http://pypi.python.org/pypi?:action=search&term=neural&submit=search

However, if you are doing serious work with Neural Networks you will want something like the Fast Neural Network library. This comes with Python bindings so that you can do your programming in Python, use the wide variety of Python modules for graphing, visualisation, data manipulation and so on. But, your neural networks will run using optmised compiled code from the FANN library. The best of both worlds.

In other words, for running the actual neural network code, you need C, not Java. Since C libraries don't integrate that well with Java, choose a language that does integrate smoothly with C libraries. Python does that and is also rather more productive than Java since there is a lot less lines of code required to explain your algorithms. Some people have found a 10 times increase in productivity over Java.

You mentioned R, perhaps because it has statistical functions that you would need to use, or perhaps you have people available who can write R code. Again, choosing Python versus R is not an either-or decision. You can use both.

The RPY library allows Python programs to access R libraries and code. Using this you would write your main programs in Python and treat R as a tool to provide libraries, in the same way that you make use of the FANN library which was written in C.

https://rpy2.github.io/

There is another module called RSPython that works both ways, so that R programs can access libraries written in Python. This would be useful if you have an expert R programmer helping you.

http://www.omegahat.org/RSPython/

And that's not all. You can leverage Python to simplify Java programming. If you have a Java neural network engine, you can still write most of your program in Python using the Jython version which runs on the Java VM and allows you to use any Java libraries and classes in your code. And you can still use the fast FANN library as well since they provide bindings for Java programs.

The main reason that I recommend Python for your work is that it is used by an awful lot of scientists which is why there are two science oriented distributions available. A second reason is that Python is very easy for novice programmers to get started with, and in exploring neural networks you will probably start with simpler simulations and work up to more complex ones with more data manipulation and analysis of results. Python will allow you to build up your own library of code, and become an expert Python programmer so that you can focus more of your attention on the neural network problems. If you bounce around with a bit of Java, and a bit of C++ and a bit of R, then you will be able to spend less time on neural networks. That strategy might be good for someone who wants a career in programming, but not for someone who wants to achieve some significant results with neural networks.

Even if your neural network work extends into specialised hardware, so-called neuromorphic chips, you can still leverage Python as this paper from the NIH demonstrates:

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2701676/

Answer 2

I tried using both higher-abstraction level languages (matlab, java) and lower ones (C). Both using toolboxes and libraries and coding them myself. The overall response? Neither is the perfect tool. Please keep in mind that:

• a prototype might not be enough: often you need to run the network over large samples, or several times over a subset of samples (in case of evolving neural networks) to get decent results. If you have to run a network a million times, even a small performance gain might be a huge help and time-saver (i.e. C over matlab);

• if, on the other hand, you need ease of coding you might want to use one of the many pre-packaged libraries (javaNN, etc);

• what kind of neural network are you using? continuous-time recurring neural networks (CTRNN)? Backprop? How do you do train them? How do you verify their results? Is accuracy important? (i.e. are you running them on a small device, such as a memory-limited robotic control board, like Arduinos?)

If you have time to spare, I'd suggest

1. learn the concepts using a higher-level language, or even pseudocode first;
2. once you are familiar with all intricacies, especially if using evolving neural networks, choose a language they are already familiar with
3. then you might want to start researching how to optimize for speed, memory footprint, etc.

Hope this helps.