Allocatable arrays performance

Question

There is an mpi-version of a program which uses COMMON blocks to store arrays that are used everywhere through the code. Unfortunately, there is no way to declare arrays in COMMON block size of which would be known only run-time. So, as a workaround I decided to move that arrays in modules which accept ALLOCATABLE arrays inside. That is, all arrays in COMMON blocks were vanished, instead ALLOCATE was used. So, this was the only thing I changed in my program. Unfortunately, performance of the program was awful (when compared to COMMON blocks realization). As to mpi-settings, there is a single mpi-process on each computational node and each mpi-process has a single thread. I found similar question asked here but don't think (don't understand :) ) how it could be applied to my case (where each process has a single thread). I appreciate any help.

Here is a simple example which illustrates what I was talking about (below is a pseudocode):

"SOURCE FILE":

SUBROUTINE ZEROSET()
   INCLUDE 'FILE_1.INC'
   INCLUDE 'FILE_2.INC'
   INCLUDE 'FILE_3.INC'
   ....
   INCLUDE 'FILE_N.INC'

   ARRAY_1 = 0.0
   ARRAY_2 = 0.0
   ARRAY_3 = 0.0
   ARRAY_4 = 0.0
   ...
   ARRAY_N = 0.0
END SUBROUTINE

As you may see, ZEROSET() has no parallel or MPI stuff. FILE_1.INC, FILE_2, ... , FILE_N.INC are files where ARRAY_1, ARRAY_2 ... ARRAY_N are defined in COMMON blocks. Something like that

REAL ARRAY_1
COMMON /ARRAY_1/ ARRAY_1(NX, NY, NZ)

Where NX, NY, NZ are well defined parameters described with help of PARAMETER directive. When I use modules, I just destroyed all COMMON blocks, so FILE_I.INC looks like

REAL, ALLOCATABLE:: ARRAY_I(:,:,:)

And then just changed "INCLUDE 'FILE_I.INC'" statement above to "USE FILE_I". Actually, when parallel program is executed, one particular process does not need a whole (NX, NY, NZ) domain, so I calculate parameters and then allocate ARRAY_I (only ONCE!).

Subroutine ZEROSET() is executed 0.18 seconds with COMMON blocks and 0.36 with modules (when array's dimensions are calculated runtime). So, the performance worsened by two times.

I hope that everything is clear now. I appreciate you help very much.

Answer 1

Using allocatable arrays in modules can often hurt performance because the compiler has no idea about sizes at compile time. You will get much better performance with many compilers with this code:

   subroutine X
   use Y  ! Has allocatable array A(N,N) in it
   call Z(A,N)
   end subroutine

   subroutine Z(A,N)
   Integer N
   real A(N,N)
   do stuff here
   end

Then this code:

   subroutine X
   use Y  ! Has allocatable array A(N,N) in it
   do stuff here
   end subroutine

The compiler will know that the array is NxN and the do loops are over N and be able to take advantage of that fact (most codes work that way on arrays). Also, after any subroutine calls in "do stuff here", the compiler will have to assume that array "A" might have changed sizes or moved locations in memory and recheck. That kills optimization.

This should get you most of your performance back.

Common blocks are located in a specific place in memory also, and that allows optimizations also.