Wrong solution of a linear equation, or why does A*(A\B) not equal B?

Question

How is it possible that the result of A*(A\D) below is not equal to D ?

It should yield D - here is an extract of the Octave documentation :

Systems of linear equations are ubiquitous in numerical analysis. To solve the set of linear equations Ax = b, use the left division operator, ‘\’: x = A \ b

Below is the code for those who want to try it:

A = [1,1,1;0,0,0;2,1,2;2,1,2;3,5,6]
D = [1;2;3;4;5]
% A is of rank 3:
rank(A)
% therefore the system Ax=D has a unique solution
x = A\D
% but Octave has not given the good solution:
A*x

Somebody says me that Matlab yields exactly the same result.

EDIT 10/10/2012: After having read the answers, let me point where I did a serious mistake: the claim "A is of rank 3 therefore the system Ax=D has a unique solution" is absolutely wrong ! By the way the piece of documentation shown above is quite disconcerting.

Answer 1

A has 5 rows, and so is D. Both of them have 3 columns. Therefore, you have an overdetermined system of 5 equations with 3 variables. In most of the cases, it means that you cannot solve the equations exactly, because you have too many constraints.

Once you do

x = A\D;

you get the least squares solution.

 0.8333
-1.5000
 1.6667

What is this solution? It is a solution that minimizes the sum of squares of errors. Let's calculate the error:

  r = A*x-D;
  totalError = sum( r.^2);

It means that you will not be able to find any x such that sum(sqr(A*x-D)) has smaller error.

Small remark : In your case, you also have a row of zeroes - which causes the actual number of equations to become 4

Let's take a look again at A*(A\D):

>> A* (A\D)

ans =

    1.0000
         0
    3.5000
    3.5000
    5.0000

That looks familiar! Very close to [1;2;3;4;5]. The first and the last rows are the same. The second is zero, because you put a line of zeros. In the 3rd and 4th rows, you had exactly the same lines in A, but different value in B, that corresponds to

2*x+ 1*y + 2*z  = 3;
2*x+ 1*y + 2*z  = 4;

And you've got their average! It makes sense, because the average is the value that minimizes the sum of distances to 3 and 4.

---

Here is a simpler example, suppose you want to solve the following system of equations:

   x = 1;
   x = 2;

Obviously, x cannot be 1 and 2 at the same time. The solution that minimizes the sum of squares of errors is 1.5

   A = [1;1];
   b = [1;2];
   A\b
   ans =
    1.5000