understanding Lapack_row_major and Lapack_col_major with lda

Question

I have three questions regarding lapack_row_major and lapack_col_major:

1) Is my understanding correct that if A = [1,2,3,4,5,6] with dimensions 2*3 then the lapack_row_major results in [ [1,2,3],[4,5,6] ], while lapack_col_major results in [ [1,3,5],[2,4,6] ] ?

2) does it matter which one of row_major or col_major I choose when I want to pass A to a function?

3) how are lapack_row_major and lda (leading dimension of array) related? Is it true that if I want to pass a m*n matrix A in lapack_row_major format to a function, then its lda is n ? and if I pass A as lapack_col_major then its lda is m ?

Thanks

Answer 1

It's really too bad that these concepts are so opaque in the LAPACK and LAPACKE documentation. For reference:

For LAPACK_{ROW,COL}_MAJOR: http://www.netlib.org/lapack/lapacke.html#_array_arguments

For LD args: http://www.netlib.org/lapack/lug/node116.html

So, to answer your questions:

1. You are on the right track, but you also need one other piece of information (the shape).

LAPACK_{ROW,COL}_MAJOR is used by LAPACKE (the C interface to LAPACK) to determine whether the block of memory you are passing (via a pointer in C) is referencing memory that is organized by row major (all of one row comes before the next row) or column major (all of one column comes before the next column). Also note that these ideas generalize (for more than 2D arrays) with row --> "outer most dimension" and col --> "inner most dimension".

One added note: C uses row-major storage and Fortran uses col-major storage. When talking between the two systems, you need to either translate or be happy working with a transpose. More on that below.

So, the LAPACK_{ROW,COL}_MAJOR value determines the interpretation of memory, but you still have to tell LAPACK/E routines about shape (via an argument usually called int N in LAPACK and LAPACKE).

So, your A array [1,2,3,4,5,6] (just numbers, listed flatly in memory)

// interpreted as row-major, 2 rows, (note: LDA=3)
A = [[1,2,3],  // a 2x3 matrix
     [4,5,6]]

// interpreted as row-major, 3 rows (note: LDA=2)
A = [[1,2],    // a 3x2 matrix
     [3,4],
     [5,6]]

// interpreted as col-major, 2 col (note: LDA=3)
A = [[1, 4]    // a (different) 3x2 matrix
     [2, 5]
     [3, 6]]

Also note that if you are writing C code, the data you enter to fill an array constant will be placed into memory in row-major order. If you pass that data "as is" to LAPACK/E, but set LAPACK_COL_MAJOR, you will effectively be working with the transpose of the array (if you set the appropriate int N and int LDA).

2. Yes. See above.

3. In the simple case with row-major data, with an array A interpreted as having r rows and c cols, LDA = c. Yes, c. The number of columns.

That seems a bit confusing. Why is it c? Well, for one reason, we already know the row count from the int N parameter. So, the next question is "why isn't it the argument just called nCols, or some such"?

The reason is this: we might use some value bigger than c to select only part of the array. For example, if we let LDA = 2 * c, we would end up using "every other row" in the LAPACK routine -- because when we say "go forward by LDA amount to get to the next thing", thing would be equivalent to jumping forward and skipping a row.

A more common phrasing for this concept -- how big a step do we need to take? -- is the stride size in the "major" (outer) dimension. Note the similarity with the LAPACK documentation term: leading dimension (the LD in LDA).

With row-major data, this is essentially answering the question: "how many elements do I need to move forward to get from row i to row i+1. A non-standard LDA (i.e., LDA != c) will answer a slightly different question.

If you have col-major data, LDA will need to be r, the number of rows (unless you are doing something fancy). The equivalent question here is "how many elements do I need to move forward to get from col i to col i+1.