Undefined references when mixing Intel C++ and Fortran in OpenMPI

Question

I have a compilation problem using Open MPI 1.8.4 and Intel Compiler v15.2. This is a large code that uses Fortran and C++. The code was previously compiled using Open MPI 1.6 And the issue was not there.

Here is the content of the make file:

ifdef TAUDEBUG
    FC=tau_f90.sh
    COMP=tau_f90.sh
    CXX=tau_cxx.sh
    CXXFLAGS=
    CXXOPT=
    LINKER=tau_cxx.sh
else
    FC=mpif90
    COMP=mpif90
    CXX=mpic++
    CXXFLAGS=
    CXXOPT=
    LINKER=$(CXX)
endif


CXXLINKFLAGS += -I/cm/shared/apps/openmpi/intel/1.8.4/lib -Wl,-rpath -Wl,/cm/shared/apps/openmpi/intel/1.8.4/lib -Wl,--enable-new-dtags -L/cm/shared/apps/openmpi/intel/1.8.4/lib -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi

FORTFLAGS= -DFORTRAN_UNDERSCORE -DFORTRAN_LOWERCASE

The issue I am having is that C++ code is using some of the Fortran routines and mpic++ needs to be linked with Fortran libraries.

With Open MPI 1.6 it was done with:

CXXLINKFLAGS += -L/usr/mpi/intel/openmpi-1.6/lib64 -lmpi_f90 

And it worked.

Starting with OpenMPI v.1.7 mpif90 was deprecated. and libmpi_f90 no longer exists. All Fortran related routines are handled by libmpi_usempif08* and libmpi_mpifh*.

And that is where I am struggling.

I tried

CXXLINKFLAGS += -L/cm/shared/apps/openmpi/intel/1.8.4/lib -lmpi_usempif08 -lmpi_mpifh -lmpi -lmpi_usempi_ignore_tkr 

And the Fortran portion of the code compiles fine but when it gets to mpic++ I start getting a lot of "undefined reference to" errors:

mpic++  rflump.o main.o -o ../rflump -O .././build_lib/libflu.a      -L/cm/shared/apps/openmpi/intel/1.8.4/lib -lmpi_usempif08 -lmpi_mpifh -lmpi -lmpi_usempi_ignore_tkr 

rflump.f90:(.text+0x56): undefined reference to `for_cpystr'
.././build_lib/libflu.a(RFLU_EndFlowSolver.o): In function `rflu_endflowsolver_':
RFLU_EndFlowSolver.f90:(.text+0x2a): undefined reference to `for_cpystr'
.././build_lib/libflu.a(RFLU_FlowSolver.o): In function `rflu_flowsolver_':
RFLU_FlowSolver.f90:(.text+0x2d): undefined reference to `for_cpystr'
.././build_lib/libflu.a(RFLU_InitFlowSolver.o): In function `rflu_initflowsolver_':
RFLU_InitFlowSolver.f90:(.text+0x4e): undefined reference to `for_cpystr'
RFLU_InitFlowSolver.f90:(.text+0x1b3): undefined reference to `for_inquire'
RFLU_InitFlowSolver.f90:(.text+0x1c59): undefined reference to `for_cpstr'
RFLU_InitFlowSolver.f90:(.text+0x2cc4): undefined reference to `for_write_seq_lis'
.././build_lib/libflu.a(RFLU_OpenConverFile.o): In function `rflu_openconverfile_':
RFLU_OpenConverFile.f90:(.text+0x2d): undefined reference to `for_cpystr'
RFLU_OpenConverFile.f90:(.text+0x1ca): undefined reference to `for_trim'
RFLU_OpenConverFile.f90:(.text+0x20a): undefined reference to `for_concat'
.././build_lib/libflu.a(RFLU_OpenPMFile.o): In function `rflu_openpmfile_':
RFLU_OpenPMFile.f90:(.text+0x2d): undefined reference to `for_cpystr'
RFLU_OpenPMFile.f90:(.text+0x1ca): undefined reference to `for_trim'
RFLU_OpenPMFile.f90:(.text+0x20a): undefined reference to `for_concat'
.././build_lib/libflu.a(RFLU_OpenStatsFileOLES.o): In function `rflu_openstatsfileoles_':
RFLU_OpenStatsFileOLES.f90:(.text+0x2d): undefined reference to `for_cpystr'
RFLU_OpenStatsFileOLES.f90:(.text+0x1c0): undefined reference to `for_trim'
RFLU_OpenStatsFileOLES.f90:(.text+0x200): undefined reference to `for_concat'
.././build_lib/libflu.a(RFLU_OpenTotalMassFile.o): In function `rflu_opentotalmassfile_':
RFLU_OpenTotalMassFile.f90:(.text+0x2d): undefined reference to `for_cpystr'
RFLU_OpenTotalMassFile.f90:(.text+0x1ca): undefined reference to `for_trim'
RFLU_OpenTotalMassFile.f90:(.text+0x20a): undefined reference to `for_concat'
.././build_lib/libflu.a(RFLU_PrintHeader.o): In function `rflu_printheader_':
RFLU_PrintHeader.f90:(.text+0x26): undefined reference to `for_cpystr'
RFLU_PrintHeader.f90:(.text+0x6e): undefined reference to `for_cpystr'
RFLU_PrintHeader.f90:(.text+0x7d): undefined reference to `for_len_trim'
RFLU_PrintHeader.f90:(.text+0xc2): undefined reference to `for_cpystr'
RFLU_PrintHeader.f90:(.text+0x324): undefined reference to `for_trim'
RFLU_PrintHeader.f90:(.text+0x622): undefined reference to `for_trim'
RFLU_PrintHeader.f90:(.text+0x77a): undefined reference to `for_trim'

I know that OpenMPI is using right compiler:

$ mpifort -V
Intel(R) Fortran Intel(R) 64 Compiler XE for applications running on Intel(R) 64, Version 15.0.2.164 Build 20150121
Copyright (C) 1985-2015 Intel Corporation.  All rights reserved.

$ mpic++ -V
Intel(R) C++ Intel(R) 64 Compiler XE for applications running on Intel(R) 64, Version 15.0.2.164 Build 20150121
Copyright (C) 1985-2015 Intel Corporation.  All rights reserved.

Also according to OpenMPI documentation to link to Fortran I need to use:

$ mpifort --showme:link
-I/cm/shared/apps/openmpi/intel/1.8.4/lib -Wl,-rpath -Wl,/cm/shared/apps/openmpi/intel/1.8.4/lib -Wl,--enable-new-dtags -L/cm/shared/apps/openmpi/intel/1.8.4/lib -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi

I copied the whole line and pasted it to Makefile:

CXXLINKFLAGS += -I/cm/shared/apps/openmpi/intel/1.8.4/lib -Wl,-rpath -Wl,/cm/shared/apps/openmpi/intel/1.8.4/lib -Wl,--enable-new-dtags -L/cm/shared/apps/openmpi/intel/1.8.4/lib -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi

The result was identical to the previous output. It looks like that C++ compiler is trying to use Fortran but is not finding it.

What am I missing here?

Answer 1

Like Hristo suggested, I added -lifcore (that was the library I found in Intel compilers) to the CXXLINKFLAGS and I do not see the undefined reference errors any more. I did not realize that you can add icpc and ifort options to mpic++ and mpifort that way.