Matlab formula optimization: Radial Basis Function

Question

• z - matrix of doubles, size Nx2;
• x - matrix of doubles, size Nx2;

sup = x(i, :);

phi(1, i) = {@(z) exp(-g * sum((z - sup(ones([size(z, 1) 1]),:)) .^ 2, 2))};

this is a Radial Basis Function (RBF) for logistic regression. Here is the formula:

I need your advice, can i optimize this formula? coz it calls millions times, and it takes a lot of time...

Answer 1

It seems in your recent edits, you introduced some syntax errors, but I think I understood what you were trying to do (from the first version).

Instead of using REPMAT or indexing to repeat the vector x(i,:) to match the rows of z, consider using the efficient BSXFUN function:

rbf(:,i) = exp( -g .* sum(bsxfun(@minus,z,x(i,:)).^2,2) );

The above obviously loops over every row of x

---

You can go one step further, and use the PDIST2 to compute the euclidean distance between every pair of rows in z and x:

%# some random data
X = rand(10,2);
Z = rand(10,2);
g = 0.5;

%# one-line solution
rbf = exp(-g .* pdist2(Z,X,'euclidean').^2);

Now every value in the matrix: rbf(i,j) corresponds to the function value between z(i,:) and x(j,:)

---

EDIT:

I timed the different methods, here is the code I used:

%# some random data
N = 5000;
X = rand(N,2);
Z = rand(N,2);
g = 0.5;

%# PDIST2
tic
rbf1 = exp(-g .* pdist2(Z,X,'euclidean').^2);
toc

%# BSXFUN+loop
tic
rbf2 = zeros(N,N);
for j=1:N
    rbf2(:,j) = exp( -g .* sum(bsxfun(@minus,Z,X(j,:)).^2,2) );
end
toc

%# REPMAT+loop
tic
rbf3 = zeros(N,N);
for j=1:N
    rbf3(:,j) = exp( -g .* sum((Z-repmat(X(j,:),[N 1])).^2,2) );
end
toc

%# check if results are equal
all( abs(rbf1(:)-rbf2(:)) < 1e-15 )
all( abs(rbf2(:)-rbf3(:)) < 1e-15 )

The results:

Elapsed time is 2.108313 seconds.     # PDIST2
Elapsed time is 1.975865 seconds.     # BSXFUN
Elapsed time is 2.706201 seconds.     # REPMAT

Answer 2

Amro has mentioned some really good methods. But the bsxfun can be further exploited by reshaping one of the matrices.

>> type r.m

N = 5000;
X = rand(N,2);
Z = rand(N,2);
g = 0.5;

%BSXFUN+loop
tic
rbf2 = zeros(N,N);
for j=1:N
    rbf2(:,j) = exp( -g .* sum(bsxfun(@minus,Z,X(j,:)).^2,2) );
end
toc

tic
diffs = bsxfun(@minus, reshape(X', [1, 2, N]), Z);
dist = reshape(sum(diffs.^2, 2), [N, N]);
rbf3 = exp(-g .* dist);
toc

>> r
Elapsed time is 2.235527 seconds.
Elapsed time is 0.877833 seconds.
>> r
Elapsed time is 2.253943 seconds.
Elapsed time is 1.047295 seconds.
>> r
Elapsed time is 2.234132 seconds.
Elapsed time is 0.856302 seconds.
>> max(abs(rbf2(:) - rbf3(:)))

ans =

     0

You want to subtract every row of X from every row of Z. This usually is straight forward when one of them is a vector and the other is a matrix. But if both of them are matrices, we can do this by making sure that each matrix in the volume contains just one vector. Here I chose X, but Z can be used interchangeably with X.