Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Machine Learning and Natural Language Processing [closed]

Assume you know a student who wants to study Machine Learning and Natural Language Processing.

What specific computer science subjects should they focus on and which programming languages are specifically designed to solve these types of problems?

I am not looking for your favorite subjects and tools, but rather industry standards.

Example: I'm guessing that knowing Prolog and Matlab might help them. They also might want to study Discrete Structures*, Calculus, and Statistics.

*Graphs and trees. Functions: properties, recursive definitions, solving recurrences. Relations: properties, equivalence, partial order. Proof techniques, inductive proof. Counting techniques and discrete probability. Logic: propositional calculus, first-order predicate calculus. Formal reasoning: natural deduction, resolution. Applications to program correctness and automatic reasoning. Introduction to algebraic structures in computing.

like image 268
Stephano Avatar asked Feb 09 '10 23:02

Stephano


People also ask

Does natural language processing fall under machine learning?

Natural Language Processing is the practice of teaching machines to understand and interpret conversational inputs from humans. NLP based on Machine Learning can be used to establish communication channels between humans and machines. Although continuously evolving, NLP has already proven useful in multiple fields.

Why is NLP not machine learning?

Because they are different: One does not include the other. Yes modern NLP (Natural Language Processing) does make use of a lot of ML (Machine Learning), but that is just one group of techniques in the arsenal. For example, graph theory and search algorithms are also used a lot.

Why is machine learning important in NLP?

Machine Learning gives the system the ability to learn from past experiences and examples. General algorithms perform a fixed set of executions according to what it has been programmed to do so and they do not possess the ability to solve unknown problems.

What is main challenge of NLP in AI?

The main challenge is information overload, which poses a big problem to access a specific, important piece of information from vast datasets. Semantic and context understanding is essential as well as challenging for summarisation systems due to quality and usability issues.


2 Answers

This related stackoverflow question has some nice answers: What are good starting points for someone interested in natural language processing?

This is a very big field. The prerequisites mostly consist of probability/statistics, linear algebra, and basic computer science, although Natural Language Processing requires a more intensive computer science background to start with (frequently covering some basic AI). Regarding specific langauges: Lisp was created "as an afterthought" for doing AI research, while Prolog (with it's roots in formal logic) is especially aimed at Natural Language Processing, and many courses will use Prolog, Scheme, Matlab, R, or another functional language (e.g. OCaml is used for this course at Cornell) as they are very suited to this kind of analysis.

Here are some more specific pointers:

For Machine Learning, Stanford CS 229: Machine Learning is great: it includes everything, including full videos of the lectures (also up on iTunes), course notes, problem sets, etc., and it was very well taught by Andrew Ng.

Note the prerequisites:

Students are expected to have the following background: Knowledge of basic computer science principles and skills, at a level sufficient to write a reasonably non-trivial computer program. Familiarity with the basic probability theory. Familiarity with the basic linear algebra.

The course uses Matlab and/or Octave. It also recommends the following readings (although the course notes themselves are very complete):

  • Christopher Bishop, Pattern Recognition and Machine Learning. Springer, 2006.
  • Richard Duda, Peter Hart and David Stork, Pattern Classification, 2nd ed. John Wiley & Sons, 2001.
  • Tom Mitchell, Machine Learning. McGraw-Hill, 1997.
  • Richard Sutton and Andrew Barto, Reinforcement Learning: An introduction. MIT Press, 1998

For Natural Language Processing, the NLP group at Stanford provides many good resources. The introductory course Stanford CS 224: Natural Language Processing includes all the lectures online and has the following prerequisites:

Adequate experience with programming and formal structures. Programming projects will be written in Java 1.5, so knowledge of Java (or a willingness to learn on your own) is required. Knowledge of standard concepts in artificial intelligence and/or computational linguistics. Basic familiarity with logic, vector spaces, and probability.

Some recommended texts are:

  • Daniel Jurafsky and James H. Martin. 2008. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics and Speech Recognition. Second Edition. Prentice Hall.
  • Christopher D. Manning and Hinrich Schütze. 1999. Foundations of Statistical Natural Language Processing. MIT Press.
  • James Allen. 1995. Natural Language Understanding. Benjamin/Cummings, 2ed.
  • Gerald Gazdar and Chris Mellish. 1989. Natural Language Processing in Prolog. Addison-Wesley. (this is available online for free)
  • Frederick Jelinek. 1998. Statistical Methods for Speech Recognition. MIT Press.

The prerequisite computational linguistics course requires basic computer programming and data structures knowledge, and uses the same text books. The required articificial intelligence course is also available online along with all the lecture notes and uses:

  • S. Russell and P. Norvig, Artificial Intelligence: A Modern Approach. Second Edition

This is the standard Artificial Intelligence text and is also worth reading.

I use R for machine learning myself and really recommend it. For this, I would suggest looking at The Elements of Statistical Learning, for which the full text is available online for free. You may want to refer to the Machine Learning and Natural Language Processing views on CRAN for specific functionality.

like image 78
Shane Avatar answered Sep 20 '22 04:09

Shane


My recommendation would be either or all (depending on his amount and area of interest) of these:

The Oxford Handbook of Computational Linguistics:

The Oxford Handbook of Computational Linguistics
(source: oup.com)

Foundations of Statistical Natural Language Processing:

Foundations of Statistical Natural Language Processing

Introduction to Information Retrieval:

Introduction to Information Retrieval

like image 31
Fabian Steeg Avatar answered Sep 23 '22 04:09

Fabian Steeg