Advice for a C, CUDA, & ANN Newbie? [closed]

Question

I'm a business major, two-thirds of the way through my degree program, with a little PHP experience, having taken one introductory C++ class, and now regretting my choice of business over programming/computer science.

I am interested in learning more advanced programming; specifically C, and eventually progressing to using the CUDA architecture for artificial neural network data analysis (not for AI, vision, or speech processing, but for finding correlations between data-points in large data sets and general data/statistical analysis).

Any advice about how I should start learning C? As well as ANN/Bayesian technology for analyzing data? There are so many books out there, I don't know what to choose.

Since CUDA is fairly new, there doesn't seem to be much learner-friendly (i.e. dumbed-down) material for it. Are there learning resources for CUDA beyond the NVIDIA documentation?

Further, what resources would you recommend to me that talk about GPGPU computing and massively parallel programming that would help me along?

Answer 1

I don't recommend trying to learn CUDA first since it's a new technology and you don't have much background in programming.

Since you don't have much experience in C (or C++), CUDA will be a pain to learn since it lacks maturity, libs, nice error messages, etc.

CUDA is meant for people who are familiar with C (C++ experience helps too) and have a problem which needs performance improvement by recoding or rethinking the solution of a well known problem.

If you're trying to solve "ANN/Bayesian" problems I would recommend creating your solution in C++ or C, your choice. Don't bother about creating threads or multithreading. Then, after evaluation the response times of your serial solution try to make it parallel by using OpenMP, Boost threads, w/e. After this, if you still need more performance, then I would recommend learning CUDA.

I think these are valid points because CUDA has some pretty cryptic errors, hard to debug, totally different architecture, etc.

If you're still interested, these are some links to learn CUDA:

Online courses:

• GPGP
• CIS 665
• Richard Edgar's GPU Computing Pages

Forum (the best source of information):

• NVIDIA CUDA Forum

Tools:

• CUDPP

Problems solved in CUDA:

• gpuDG
• Histogram Computation

Answer 2

You've expressed 3 different goals:

• Learning to program in C
• Learning to write code for the CUDA platform
• Learning to use Bayes' Nets and/or Neural nets for data analysis

Firstly: these things are not easy for people who already have several degrees in the field. If you only do one, make sure to learn about Bayesian inference. It's by far the most powerful framework available for reasoning about data, and you need to know it. Check out MacKay's book (mentioned at the bottom). You certainly have set yourself a challenging task - I wish you all the best!

Your goals are all fairly different kettles of fish. Learning to program in C is not too difficult. I would if at all possible to take the "Intro to Algorithms & Data Structures" (usually the first course for CS majors) at your university (it's probably taught in Java). This will be extremely useful for you, and basic coding in C will then simply be a matter of learning syntax.

Learning to write code for the CUDA platform is substantially more challenging. As recommended above, please check out OpenMPI first. In general, you will be well-served to read something about computer architecture (Patterson & Hennessy is nice), as well as a book on parallel algorithms. If you've never seen concurrency (i.e. if you haven't heard of a semaphore), it would be useful to look it up (lectures notes from an operating systems course will probably cover it - see MIT Open Courseware). Finally, as mentioned, there are few good references available for GPU programming since it's a new field. So your best bet will be to read example source code to learn how it's done.

Finally, Bayesian nets and Neural nets. First, please be aware that these are quite different. Bayesian networks are a graphical (nodes & edges) way of representing a joint probability distribution over a (usually large) number of variables. The term "neural network" is somewhat vaguer, but generally refers to using simple processing elements to learn a nonlinear function for classifying data points. A book that gives a really nice introduction to both Bayes' nets and Neural nets is David J.C. MacKay's Information Theory, Inference and Learning algorithms. The book is available for free online at http://www.inference.phy.cam.ac.uk/mackay/itila/. This book is by far my favorite on the topic. The exposition is extremely clear, and the exercises are illuminating (most have solutions).