Multidimensional polynomial regression (preferably C/C++, Java or Scala)

Question

Given a collection of (N+1)-dimensional real valued vectors with N independent and 1 dependent value, I would like to compute a polynomial of degree 1 (linear), 2 (quadratic) or higher that provides a reasonably good fit (e.g. as determined by least squares error). In other words, when applied to the elements of the collection, the polynomial should map the independent values of each one to the associated dependent value (with some reasonable margin of error).

I expect the dimensionality of the independent variables to be in the range 2..8, and to work on collections of 20..200 elements. I am hoping to fit a polynomial in milliseconds rather than seconds. :-)

I quickly found algorithms for polynomial regression for one-dimensional data, but I have not been able to come up with anything practical for multi-dimensional data. I am interested primarily in algorithm descriptions or source code. Any pointers?

Answer 1

You might want to explore the Weka data mining and machine learning platform - it's extremely comprehensive and includes all kinds of different regression algorithms.

A big bonus is that it is all open source so you can also study the code if you like.