Const correctness of Python's C API

Question

It seems that the Python C API is not consistent with the const correctness of character arrays. For example, PyImport_ImportFrozenModule accepts a char*, whereas PyImport_ImportModule accepts a const char*.

The implication of all this is that in my C++ application that I am writing with an embedded Python interpreter, I sometimes have to cast the string literal that I pass to a Python API call as just a char* (as opposed to const char*), and sometimes I don't. For example:

PyObject *os = PyImport_ImportModule("os"); // Works without the const_cast
PyObject *cwd = PyObject_CallMethod(os, const_cast<char*>("getcwd"), NULL); // Accepts char*, not const char*

If I don't do the const_cast<char*> (or (char*)) on the string literal, I get a compiler warning about casting string literals to char*.

Here are my questions:

1. Is there an advantage/reason to having some of the functions not take a const char* (and/or why would the Python API not be consistent in this)? My understanding is that if the function can take a string literal, it cannot change the char* so the const modifier would just be reinforcing this. I also believe that the const distinction is not as important for C (for which the API was written) than it is in C++ (correct me if I am wrong... my strength is python, not C/C++). Is the lack of "const correctness" of the Python API because it's simply not as important in C? (There is an old thread on the python mailing list from 2000 asking the same question, but it didn't seem to go anywhere and it is implied the reason might be due to some compilers not supporting const. Since many functions now have const char*, this doesn't seem to apply anymore)

2. Because my understanding of C++ is limited, I am unsure if I am going about casting string literals properly. The way I see it, I can either one of the following (I am currently doing the first):

   // Method 1) Use const_cast<char*>
   PyImport_ImportFrozenModule(const_cast<char*>("mymodule"));
   
   // Method 2) Use (char*)
   PyImport_ImportFrozenModule((char*) "mymodule");
   
   // Method 3) Use char array
   char mod[] = "mymodule";
   PyImport_ImportFrozenModule(mod);
   

   Which is the best method do use?

---

Update:

It looks like the Python3 branch is slowly trying to fix the const correctness issue. For example, the PyImport_ImportFrozenModule function I use as an example above now takes a const char* in Python 3.4, but there are still functions that take only a char*, such as PyLong_FromString.

Answer 1

Based on some mailing list conversations from python-dev, it looks like the initial API just simply wasn't created with const correctness in mind, probably just because Guido didn't think about it. Dating all the way back to 2002, someone asked if there was any desire to address that by adding const-correctness, complaining that it's a pain to always have to do this:

somefunc(const char* modulename, const char* key)
{
    ... PyImport_ImportModule(const_cast<char*>(modulename)) ...

Guido Van Rossum (the creator of Python) replied (emphasis mine):

I've never tried to enforce const-correctness before, but I've heard enough horror stories about this. The problem is that it breaks 3rd party extensions left and right, and fixing those isn't always easy. In general, whenever you add a const somewhere, it ends up propagating to some other API, which then also requires a const, which propagates to yet another API needing a const, ad infinitum.

There was a bit more discussion, but without Guido's support the idea died.

Fast forward nine years, and the topic came up again. This time someone was simply wondering why some functions were const-correct, while others weren't. One of the Python core developers replied with this:

We have been adding const to many places over the years. I think the specific case was just missed (i.e. nobody cared about adding const there).

It seems that when it could be done without breaking backwards compatibility, const-correctness has been added to many places in the C API (and in the case of Python 3, in places where it would break backwards compatibility with Python 2), but there was never a real global effort to fix it everywhere. So the situation is better in Python 3, but the entire API is likely not const correct even now.

I'm don't think that the Python community has any preferred way to handle casting with calls that are not const-correct (there's no mention of it in the official C-API style guide), probably because there aren't a ton of people out there interfacing with the C-API from C++ code. I would say the preferred way of doing it from a pure C++ best-practices perspective would be the first choice, though. (I'm by no means a C++ expert, so take that with a grain of salt).

Answer 2

Is there an advantage/reason to having some of the functions not take a const char*?

No. Looks like an oversight in the library's design or, like you say, legacy issues. They could at least have made it consistent, though!

My understanding is that if the function can take a string literal, it cannot change the char* so the const modifier would just be reinforcing this.

Exactly. Their documentation should also specify that the function argument (or, rather, the argument's pointee) shall not be modified during the function call; alas it currently does not say this.

I also believe that the const distinction is not as important for C (for which the API was written) than it is in C++.

Well, not really, at least as far as I know.

The way I see it, I can either one of the following (I am currently doing the first)

(good)

Which is the best method do use?

Well the const_cast will at least make sure that you are only modifying the const-ness, so if you had to choose I'd go with that. But, really, I wouldn't be too bothered about this.