Sort in ascending order, but keep zeros at last

Question

Suppose I have a matrix A, on the following form.

A =
    35     1     6
     3    32     0
     0     9     0
     0     0     0

I want to sort it in ascending order, but keep the zeros at last.

I know I can subsitute all zeros with inf, sort it, and replace the infs with zeros again, as proposed in this question.

I would think there was a simpler way. At least since my zeros are already in the bottom rows. Can I do this in a single line?

What I want:

A =
     3     1     6
     35    9     0
     0     32    0
     0     0     0

Thanks!

UPDATE

There was a question regarding the overhead of Eitan's answer. Here are the results (averaged, and after warm up):

B =  kron(A,ceil(rand(2000)*1000));  % 8000x6000 matrix
C = B;

%% Eitan's solution:
t1 = tic; B(B ~= 0) = nonzeros(sort(B)); toc(t1)
Elapsed time is  1.768782 seconds.

%% From question text:
B = C;
t1 = tic; B(B==0)=Inf; B = sort(B); B(B==Inf)=0; toc(t1) 
Elapsed time is 1.938374 seconds.

%% evading's solution (in the comments):
B = C;
t1 = tic; for i = 1:size(B,2)  index = B(:,i) ~= 0; B(index, i) = sort(B(index, i)); end
toc(t1)
Elapsed time is 1.954454 seconds.

%% Shai's solution (in the comments):
B = C;
t1 = tic; sel = B==0; B(sel)=inf;B=sort(B);B(sel)=0; toc(t1)
Elapsed time is 1.880054 seconds.

Answer 1

If you can guarantee that the zeros are only at the bottom of each column, you can do:

A(A ~= 0) = nonzeros(sort(A));