Purpose of LDA argument in BLAS dgemm?

Question

The Fortran reference implementation documentation states:

*  LDA    - INTEGER.
*           On entry, LDA specifies the first dimension of A as declared
*           in the calling (sub) program. When  TRANSA = 'N' or 'n' then
*           LDA must be at least  max( 1, m ), otherwise  LDA must be at
*           least  max( 1, k ).
*           Unchanged on exit.

However, given m and k shouldn't I be able to derive LDA? When is LDA permitted to be bigger than n (or k)?

Answer 1

The LDA parameter in BLAS is effectively the stride of the matrix as it is laid out in linear memory. It is perfectly valid to have an LDA value which is larger than the leading dimension of the matrix which is being operated on. Typical cases where it is either useful or necessary to use a larger LDA value are when you are operating on a sub matrix from a larger dense matrix, and when hardware or algorithms offer performance advantages when storage is padded to round multiples of some optimal size (cache lines or GPU memory transaction size, or load balance in multiprocessor implementations, for example).

Answer 2

The distinction is between the logical size of the first dimensions of the arrays A and B and the physical size. The first is the size of the array that you are using, the second is the value in the declaration, or the physical amount of memory used. Since Fortran is a column major language, the declared sizes of all indices except the last must be known in order to calculate the location of an array element. Notice the FORTRAN 77 style declarations of "A(LDA,),B(LDB,),C(LDC,*)". The declared size of the array can be larger than the portion that you are using; of course it can't be smaller.

Answer 3

Another way to look at it is LDA is the y-stride, meaning in a row-major layout your address for element A[y,x] is computed as x+LDA*y. For a "packed" memory layout without gaps between adjacent lines of x-data LDA=xSize.