Using CMake, how can I concat files and install them

Question

I'm new to CMake and I have a problem that I can not figure out a solution to. I'm using CMake to compile a project with a bunch of optional sub-dirs and it builds shared library files as expected. That part seems to be working fine. Each of these sub-dirs contains a sql file. I need to concat all the selected sql files to one sql header file and install the result. So one file like:

sql_header.sql
sub_dir_A.sql
sub_dir_C.sql
sub_dir_D.sql

If I did this directly in a make file I might do something like the following only smarter to deal with only the selected sub-dirs:

cat sql_header.sql  > "${INSTALL_PATH}/somefile.sql"
cat sub_dir_A.sql  >> "${INSTALL_PATH}/somefile.sql"
cat sub_dir_C.sql  >> "${INSTALL_PATH}/somefile.sql"
cat sub_dir_D.sql  >> "${INSTALL_PATH}/somefile.sql"

I have sort of figured out pieces of this, like I can use:

LIST(APPEND PACKAGE_SQL_FILES "some_file.sql")

which I assume I can place in each of the sub-dirs CMakeLists.txt files to collect the file names. And I can create a macro like:

CAT(IN "${PACKAGE_SQL_FILES}" OUT "${INSTALL_PATH}/somefile.sql")

But I am lost between when the CMake initially runs and when it runs from the make install. Maybe there is a better way to do this. I need this to work on both Windows and Linux.

I would be happy with some hints to point me in the right direction.

Answer 1

You can create the concatenated file mainly using CMake's file and function commands.

First, create a cat function:

function(cat IN_FILE OUT_FILE)
  file(READ ${IN_FILE} CONTENTS)
  file(APPEND ${OUT_FILE} "${CONTENTS}")
endfunction()

Assuming you have the list of input files in the variable PACKAGE_SQL_FILES, you can use the function like this:

# Prepare a temporary file to "cat" to:
file(WRITE somefile.sql.in "")

# Call the "cat" function for each input file
foreach(PACKAGE_SQL_FILE ${PACKAGE_SQL_FILES})
  cat(${PACKAGE_SQL_FILE} somefile.sql.in)
endforeach()

# Copy the temporary file to the final location
configure_file(somefile.sql.in somefile.sql COPYONLY)

The reason for writing to a temporary is so the real target file only gets updated if its content has changed. See this answer for why this is a good thing.

You should note that if you're including the subdirectories via the add_subdirectory command, the subdirs all have their own scope as far as CMake variables are concerned. In the subdirs, using list will only affect variables in the scope of that subdir.

If you want to create a list available in the parent scope, you'll need to use set(... PARENT_SCOPE), e.g.

set(PACKAGE_SQL_FILES
    ${PACKAGE_SQL_FILES}
    ${CMAKE_CURRENT_SOURCE_DIR}/some_file.sql
    PARENT_SCOPE)

All this so far has simply created the concatenated file in the root of your build tree. To install it, you probably want to use the install(FILES ...) command:

install(FILES ${CMAKE_BINARY_DIR}/somefile.sql
        DESTINATION ${INSTALL_PATH})

So, whenever CMake runs (either because you manually invoke it or because it detects changes when you do "make"), it will update the concatenated file in the build tree. Only once you run "make install" will the file finally be copied from the build root to the install location.

Answer 2

As of CMake 3.18, the CMake command line tool can concatenate files using cat. So, assuming a variable PACKAGE_SQL_FILES containing the list of files, you can run the cat command using execute_process:

# Concatenate the sql files into a variable 'FINAL_FILE'.
execute_process(COMMAND ${CMAKE_COMMAND} -E cat ${PACKAGE_SQL_FILES}
    OUTPUT_VARIABLE FINAL_FILE
    WORKING_DIRECTORY ${CMAKE_CURRENT_LIST_DIR}
)
# Write out the concatenated contents to 'final.sql.in'.
file(WRITE final.sql.in ${FINAL_FILE})

The rest of the solution is similar to Fraser's response. You can use configure_file so the resultant file is only updated when necessary.

configure_file(final.sql.in final.sql COPYONLY)

You can still use install in the same way to install the file:

install(FILES ${CMAKE_CURRENT_BINARY_DIR}/final.sql
    DESTINATION ${INSTALL_PATH})