How to use redirection in cmake add_test

Question

I have a console application called "foo", which takes a reference text file as input (in.txt) and generates text at standard output (I want to keep this behaviour).

In make (not cmake), I use a test target, which calls foo and redirects the output to a file (out.txt) as follows. Then, I use diff to compare the file out.txt with the expected refernece (ref.txt)

test:
    ./foo -a test/in.txt > test/out.txt
    diff test/out.txt test/ref.txt

This works fine using make. Now my question is; how can I use cmake to create a similar Makefile?

From within a subdrectory called build, I tried

project(foo)
...
add_test(NAME test1 COMMAND ./foo ../test/in.txt > ../test/out.txt)
enable_testing()

Using cmake version 3.5, I get a Makefile without errors, but when I call make test, the test itself fails. It seems the cmake command add_test supports command line arguments, but not the redirection. I tried quotes and escaping witout success. Since I could not pass this part, I didn't try to use diff. I just imagine that I could pack foo and diff in one line using & as you can do with bash. That would be the second step.

Answer 1

Turning my comment into an answer

As @Tsyvarev has stated, CTest commands are not run in a shell's context. But you could just add the shell needed yourself and use e.g. sh as the command to be called with add_test().

I've run some tests with your example code and the following did work successfully:

add_test(NAME test1 COMMAND sh -c "$<TARGET_FILE:foo> ../test/in.txt > ../test/out.txt")

This solution is not platform independent (it depends on sh to be available in the search paths).

---

So if you want to be more flexible you could do something like:

include(FindUnixCommands)

file(TO_NATIVE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/test/in.txt" _in)
file(TO_NATIVE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/test/out.txt" _out)
if (BASH)
    add_test(
        NAME test1 
        COMMAND ${BASH} -c "$<TARGET_FILE:foo> ${_in} > ${_out}"
    )
else()
    if (WIN32)
        add_test(
            NAME test1 
            COMMAND ${CMAKE_COMMAND} -E chdir $<TARGET_FILE_DIR:foo> $ENV{ComSpec} /c "$<TARGET_FILE_NAME:foo> ${_in} > ${_out}"
        )
    else()
        message(FATAL_ERROR "Unknown shell command for ${CMAKE_HOST_SYSTEM_NAME}")
    endif()
endif()

---

Additionally there is the possibility to execute a more platform independent diff with ${CMAKE_COMMAND} -E compare_files <file1> <file2>. So you could simplify your complete makefile based example in CMake with:

add_custom_command(
    TARGET foo
    POST_BUILD
    COMMAND ${CMAKE_COMMAND} -E echo "Running $<TARGET_FILE_NAME:foo> ..."
    COMMAND foo in.txt > out.txt
    WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/test
)

add_test(
    NAME test1 
    COMMAND ${CMAKE_COMMAND} -E compare_files in.txt out.txt
    WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/test
)

References

• Integrate bash test scripts in cmake
• CMake: piping commands to executable
• cmake: make tests successfully passing part of the build process