Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Prerequisites Needed to Read Books on Neural Networks (and understand them)

I've been trying to learn about Neural Networks for a while now, and I can understand some basic tutorials online, and I've been able to get through portions of Neural Computing - An Introduction but even there, I'm glazing over a lot of the math, and it becomes completely over my head after the first few chapters. Even then its the least book "math-y" I can find.

Its not that I'm afraid of the math or anything, its just I haven't learned what I need, and I'm not sure what I need exactly. I'm currently enrolled at my local university, working on catching up on classes I need to enter the MS in Comp. Sci program (my BA is in Business/Info. Sys.) and I haven't gotten very far. According to the university's little course descriptions, NN's are actually covered in a Electrical Engineering course on Pattern Recognition (seems odd to me that this course is EE), which has a few EE prereq's that I don't need to get into the MS Comp. Sci. Program.

I'm extremely interested in this topic, and know I eventually want to learn a lot more about it, the problem is, I don't know what I need to know first. Here are topics I think I might need, but this is just speculation from ignorance:

  • Single Variable Calculus (I've had Calc I and II, so I think I'm covered here, just listing for completeness)
  • Multi Variable Calculus
  • Linear Algebra (I've not taken this formally yet, but can actually understand many of the concepts from what I've managed to grok on Wikipedia and other sites)
  • Discrete Mathematics (Another I've not taken formally, but learned a portion of on my own
  • Graph Theory
  • Probability Theory
  • Bayesian Statistics
  • Circuit Design
  • Other maths?
  • Other comp sci topics

Obviously there is a neuroscience component here as well, but I actually haven't had any trouble understanding books when they talk about it as applied to NN's, largely because its conceptual

In short, Can someone lay out a semi-clear path that one needs to really understand, read book on and eventually implement Neural Networks?

like image 989
Tristan Havelick Avatar asked Dec 03 '08 07:12

Tristan Havelick


People also ask

What are the prerequisites for learning neural networks?

There are no prerequisites to learn neural networks. However, it is recommended that learners have a basic understanding of statistics, mathematics, and machine learning concepts.

What math is needed to understand neural networks?

The math component would likely include advanced algebra, trig, linear algebra, and calculus at minimum.

Is it hard to learn neural networks?

Training deep learning neural networks is very challenging. The best general algorithm known for solving this problem is stochastic gradient descent, where model weights are updated each iteration using the backpropagation of error algorithm. Optimization in general is an extremely difficult task.


2 Answers

If you want a list of college courses that you'll need to understand the book, here it is:

  • Calculus (I, II and III)
  • Differential Equations
  • Linear Algebra
  • Statistics (or a good covering of Bayes)

However, I did just fine in my NN classes without Diff. Eq. and just had to look up concepts I hadn't studied yet.

You can take the black box approach as above, but if you really want to understand the math and implementation of the networks, you'll have to study. It's going to be a steep learning curve to fully grasp the more advanced networks no matter what you do. You can either take the above classes first, or you can start reading the book and look up everything you don't grasp on wikipedia, and then from those articles read whatever you have to read to understand them, etc. You will find that, either way, you'll eventually get past that initial peek and things will be easier.

It would be good if you told us why you want to learn neural networks. I've not found a single use for them in my professional career, though I'm not a game developer or telecommunications developer.

like image 96
i_am_jorf Avatar answered Oct 08 '22 21:10

i_am_jorf


You can't implement "neural networks" -- you'll end up implementing a specific kind of NN (e.g. perceptron). There are many different kinds of NNs, each more suitable for some specific kind of task, and each kind uses some math (and not only math) concepts that are specifically only to that particular kind. For example, Boltzmann machines use concepts from statistical thermodynamics (founded by Boltzmann).

As for your question: without a clear goal, there is no clear (not even "semi-clear") path.

like image 38
zvrba Avatar answered Oct 08 '22 22:10

zvrba