Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Prototyping neural networks

Tags:

from your experience, which is the most effective approach to implement artificial neural networks prototypes? It is a lot of hype about R (free, but I didn't work with it) or Matlab (not free), another possible choice is to use a language like C++/Java/C#. The question is mainly targeting the people that tried to test some neural networks architectures or learning algorithms.

If your choice is to use a programming language different from the three mentioned above, can you tell me their names and some explanations concerning your choice (excepting: this is the only/most used language known by me).

Thanks.

like image 445
lmsasu Avatar asked Dec 04 '09 12:12

lmsasu


People also ask

What is prototype network?

Prototypical Networks learn a metric space in which classification can be performed by computing distances to prototype representations of each class. Compared to recent approaches for few-shot learning, they reflect a simpler inductive bias that is beneficial in this limited-data regime, and achieve excellent results.

What is prototyping in deep learning?

Model prototyping is the phase in machine learning model development lifecycle where data scientists iterate towards building best performing models through data loading, cleansing, preparation, feature engineering, model training, tuning and scoring so that it can be used in production environment to meet a business ...

What is a prototype in AI?

Prototypes are incredible, low-investment tools for learning and validation. These preliminary experiences are necessary when it comes to AI given its enormous overhead (e.g. data collection, model building, reliability testing and maintenance). So it makes a lot of sense to prototype the product ahead of time.

What are neural networks in Illustrator?

A neural network is a method in artificial intelligence that teaches computers to process data in a way that is inspired by the human brain. It is a type of machine learning process, called deep learning, that uses interconnected nodes or neurons in a layered structure that resembles the human brain.


2 Answers

Since neural networks are used a lot by scientists, and not so much by programmers, choose a programming language that targets scientists and provides good libraries for neural networks.

Python is a reasonable choice since it is widely used by scientists. Two distributions to start with are

http://www.pythonxy.com/

http://code.enthought.com/

These Python distributions include a lot of add-on modules that are not in the standard Python library but which are very useful for the type of programming that scientists do. Searching the Python package index comes up with a few neural network packages that may be OK if you are just learning about neural networks.

http://pypi.python.org/pypi?:action=search&term=neural&submit=search

However, if you are doing serious work with Neural Networks you will want something like the Fast Neural Network library. This comes with Python bindings so that you can do your programming in Python, use the wide variety of Python modules for graphing, visualisation, data manipulation and so on. But, your neural networks will run using optmised compiled code from the FANN library. The best of both worlds.

In other words, for running the actual neural network code, you need C, not Java. Since C libraries don't integrate that well with Java, choose a language that does integrate smoothly with C libraries. Python does that and is also rather more productive than Java since there is a lot less lines of code required to explain your algorithms. Some people have found a 10 times increase in productivity over Java.

You mentioned R, perhaps because it has statistical functions that you would need to use, or perhaps you have people available who can write R code. Again, choosing Python versus R is not an either-or decision. You can use both.

The RPY library allows Python programs to access R libraries and code. Using this you would write your main programs in Python and treat R as a tool to provide libraries, in the same way that you make use of the FANN library which was written in C.

https://rpy2.github.io/

There is another module called RSPython that works both ways, so that R programs can access libraries written in Python. This would be useful if you have an expert R programmer helping you.

http://www.omegahat.org/RSPython/

And that's not all. You can leverage Python to simplify Java programming. If you have a Java neural network engine, you can still write most of your program in Python using the Jython version which runs on the Java VM and allows you to use any Java libraries and classes in your code. And you can still use the fast FANN library as well since they provide bindings for Java programs.

The main reason that I recommend Python for your work is that it is used by an awful lot of scientists which is why there are two science oriented distributions available. A second reason is that Python is very easy for novice programmers to get started with, and in exploring neural networks you will probably start with simpler simulations and work up to more complex ones with more data manipulation and analysis of results. Python will allow you to build up your own library of code, and become an expert Python programmer so that you can focus more of your attention on the neural network problems. If you bounce around with a bit of Java, and a bit of C++ and a bit of R, then you will be able to spend less time on neural networks. That strategy might be good for someone who wants a career in programming, but not for someone who wants to achieve some significant results with neural networks.

Even if your neural network work extends into specialised hardware, so-called neuromorphic chips, you can still leverage Python as this paper from the NIH demonstrates:

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2701676/

like image 145
Michael Dillon Avatar answered Nov 15 '22 11:11

Michael Dillon


I tried using both higher-abstraction level languages (matlab, java) and lower ones (C). Both using toolboxes and libraries and coding them myself. The overall response? Neither is the perfect tool. Please keep in mind that:

  • a prototype might not be enough: often you need to run the network over large samples, or several times over a subset of samples (in case of evolving neural networks) to get decent results. If you have to run a network a million times, even a small performance gain might be a huge help and time-saver (i.e. C over matlab);

  • if, on the other hand, you need ease of coding you might want to use one of the many pre-packaged libraries (javaNN, etc);

  • what kind of neural network are you using? continuous-time recurring neural networks (CTRNN)? Backprop? How do you do train them? How do you verify their results? Is accuracy important? (i.e. are you running them on a small device, such as a memory-limited robotic control board, like Arduinos?)

If you have time to spare, I'd suggest

  1. learn the concepts using a higher-level language, or even pseudocode first;
  2. once you are familiar with all intricacies, especially if using evolving neural networks, choose a language they are already familiar with
  3. then you might want to start researching how to optimize for speed, memory footprint, etc.

Hope this helps.

like image 28
lorenzog Avatar answered Nov 15 '22 11:11

lorenzog