Call to function (unknown) through pointer to incorrect function type

Question

I have a program that dynamically links against a library. The program passes a function pointer to that library, to execute.

But the ubsan (Undefined Behavior Sanitizer) specified that the pointer is on an incorrect function type. And that occurs only

• if the callback function has a class as parameter
• if the callback function has a class as parameter, but only forward declared
• if I specify the compilation flags: -fvisibility=hidden.

I use clang to compile my project.

Is it a bug in clang undefined behavior sanitizer?

The following code is reduced to a simple test case. Check the comments to see where we can act to remove some warnings

The code of the application:

Main.cxx

#include "Caller.h"
#include "Param.h"

static void FctVoid()
{
}
static void FctInt(int _param)
{
   static_cast<void>(&_param);
}
static void FctCaller(Caller &_caller)
{
   static_cast<void>(&_caller);
}
static void FctParam(Param const &_param)
{
   static_cast<void>(&_param);
}

int main()
{
   Param param;
   Caller::CallVoid(&FctVoid);
   Caller::CallInt(&FctInt);
   Caller::CallThis(&FctCaller);
   Caller::CallParam(&FctParam, param);
   return 0;
}

The code of the library's files are:

Caller.cxx:

#include "Caller.h"
// To uncomment to fix one warning
//#include "Param.h"
void Caller::CallVoid(FctVoidT _fct)
{
   _fct();
}
void Caller::CallInt(FctIntT _fct)
{
   _fct(32);
}
void Caller::CallThis(FctThisT _fct)
{
   Caller caller;
   _fct(caller);
}
void Caller::CallParam(FctParamT const &_fct, Param const &_param)
{
   _fct(_param);
}

Caller.h

#ifndef __Caller_h_
#define __Caller_h_
#include "owExport.h"

class Param;
class EXPORT_Library Caller
{
public:
   typedef void(*FctVoidT)();
   static void CallVoid(FctVoidT _fct);
   typedef void(*FctIntT)(int);
   static void CallInt(FctIntT _fct);
   typedef void(*FctThisT)(Caller &);
   static void CallThis(FctThisT _fct);
   typedef void(*FctParamT)(Param const &);
   static void CallParam(FctParamT const &_fct, Param const &_param);
};
#endif

Param.h

#ifndef __Param_h_
#define __Param_h_
#include "owExport.h"
class EXPORT_Library Param
{
public:
};
#endif

owExport.h

#ifndef __owExport_h_
#define __owExport_h_
#define OW_EXPORT __attribute__ ((visibility("default")))
#define OW_IMPORT
// Use this one to fix one warning
#define OW_IMPORT __attribute__ ((visibility("default")))
#ifdef Library_EXPORTS
#  define EXPORT_Library OW_EXPORT
#else
#  define EXPORT_Library OW_IMPORT
#endif
#endif

CMakeLists.txt that configures the project:

cmake_minimum_required(VERSION 3.0.0)
project(TestFunction)
set(BUILD_SHARED_LIBS ON)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -fsanitize=undefined ")

# Act here to for the call of function through pointer to incorrect function type
# set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fvisibility=hidden")

add_library(Library Caller.cxx Param.cxx)
add_executable(TestWithLib Main.cxx)
target_link_libraries(TestWithLib Library)

Answer 1

First: It is not good if you edit the question to add fixes already. This makes it hard to answer.

For your problem: You basically have 2 issues: First with the Ssymbol Caller, second withParam`, both are basically the same.

For the source of the issue: UBSAN compares the typeinfo of the pointer to the expected typeinfo. If the typeinfo differs, it shows the error. typeinfo comparison is done by pointer comparison. This is great for speed but introduces a subtle issue: Even when the actual types are literally the same, they might not share the same typeinfo. This is also important when you throw a type from a shared library and want to catch it in the executable (or vice-versa): Catching is done by typeinfo comparison and if the two types are not exactly the same (share the same typeinfo) you won't catch it.

So your first issue is class EXPORT_Library Caller: You conditionally define EXPORT_Library to either be "exported" or not. If it is exported from multiple DSOs then the typeinfos will be merged. In your case you export it in the shared library but not in the executable which prevents merging them. You can use BOOST_SYMBOL_EXPORT or OW_EXPORT for this.

Second issue is the other way round (assuming EXPORT_Library==OW_EXPORT): Param is exported when the Param.h header is included which is only done by the executable not by the shared library. Again typeinfos not merged -> different types to the RTTI system.

Bottom line: Export all your classes you want to use over DSO boundaries.