What is the appropriate library for linking Postgres server-side C functions

Question

I am trying to compile some C extensions, on platform with Ubuntu 14.04, for Postgres 9.5.

In my case, I want to write my C code and compile it first to a standalone executable (as you can see from my Makefile below). This is because I am also using the NumPy API and writing functions that convert Postgres ArrayType arrays into NumPy PyArray objects, and then use some NumPy array functions. It's very tricky to get the details right, deallocate NpyIter objects correctly, etc., so I definitely need to compile, run, observe errors, and test all before finalizing the details of how the library is built for the final part where I say CREATE EXTENSION in Postgres.

When compiling, I get several undefined reference issues, such as:

tmp.c:(.text+0x2d6): undefined reference to `get_typlenbyvalalign'
tmp.c:(.text+0x346): undefined reference to `deconstruct_array'
tmp.c:(.text+0x41f): undefined reference to `DatumGetFloat8'
tmp.c:(.text+0x4ae): undefined reference to `pfree'
tmp.c:(.text+0x4ba): undefined reference to `pfree'

These are server-side functions from the Postgres C API, but with lots of Googling and lots of strongarming of pgxs I cannot figure out how to obtain the name or path of the backend Postgres library that I'm failing to link.

Almost all searches just mention libpq, but these functions are not defined in that client-side API library, so I'm looking for something else.

For reference, here's the Makefile I'm currently using. Including the library directory from pg_config --libdir must also be incorrect, as it does not cause any changes in the undefined reference errors.

INCLUDEDIRS := -I.
INCLUDEDIRS += -I/usr/include/python2.7
INCLUDEDIRS += -I/home/username/anaconda/lib/python2.7/site-packages/numpy/core/include
INCLUDEDIRS += -I$(shell pg_config --includedir-server)
INCLUDEDIRS += -I$(shell pg_config --includedir)

LIBS := -L$(shell pg_config --libdir)
LIBS += -lpython2.7

tmp: tmp.c Makefile
    gcc tmp.c -o tmp $(INCLUDEDIRS) $(LIBS)

Output of pg_config --libdir is:

user@computer:~/programming$ pg_config --libdir
/usr/lib/x86_64-linux-gnu

In that library directory I also found libpgcommon and when I add it to the Makefile, some of the undefined references disappear, but not all. These still remain:

tmp.c:(.text+0x2d6): undefined reference to `get_typlenbyvalalign'
tmp.c:(.text+0x346): undefined reference to `deconstruct_array'
tmp.c:(.text+0x41f): undefined reference to `DatumGetFloat8'

So pfree was found by linking with libpgcommon, but nothing else.

Digging even further, inside of postgres.h I can see where the DatumGetFloat8 macro is defined (line 662):

#ifdef USE_FLOAT8_BYVAL
extern float8 DatumGetFloat8(Datum X);
#else
#define DatumGetFloat8(X) (* ((float8 *) DatumGetPointer(X)))
#endif

So, it must be the case that Postgres was installed in some way that made use of the USE_FLOAT8_BYVAL flag. Is that standard? Would you expect that from a plain Postgres install using package repos for a popular OS like Ubuntu?

Given this, what is the other source code or library from which DatumGetFloat8 is extern'd from? For example, searching Google for "postgres DatumGetFloat8" sheds pretty much no light on this. The best I can find is a message thread from 2011 stating (not sure if correctly):

A missing reference to DatumGetFloat8 implies that the server was built
with float8 pass by value and pljava was built with float8 pass by
reference.

(The pljava bit is irrelevant for me).

Answer 1

A PostgreSQL extension is a shared library and not a standalone executable, so you got to use -shared -fpic -o tmp.so in your Makefile. Don't link with -lpq or -lpgcommon.

Then gcc won't complain about undefined references – these will be resolved with the executable postgres when the shared library is loaded into PostgreSQL, either with the SQL command LOAD or when a PostgreSQL C function defined with that library is called.

Of course you'll have to use the same -D defines as the PostgreSQL server when it was built, so that for example USE_FLOAT8_BYVAL is set or unset in both. Otherwise your extension may fail to load or could crash in interesting ways.
(Since you were asking: there is nothing bad about passing float8 by value; it is actually more efficient that way if your architecture supports it. DatumGetFloat8 is defined and exported in postgres if USE_FLOAT8_BYVAL is defined.)

To make all that easy, it is best to use the PostgreSQL extension building infrastructure PGXS. That will take care of all these problems for you.

All you have to do is create an appropriate Makefile, in your case it might be as simple as

MODULES = tmp

PG_CONFIG = pg_config
PGXS := $(shell $(PG_CONFIG) --pgxs)
include $(PGXS)

Then run make and make install, and you are ready to go!

Since your problem is to test and debug your code that uses the NumPy API, you should modularize your PostgreSQL extension so that you can do that more easily. You could for example write numpy.c which contains all the code that accesses that API and pgxs.c which contains the PostgreSQL extension code and calls functions in numpy.c.
Then you can write special testing code and link it with numpy.o to test your NumPy functions.