Django database charset issue

Question

I have the following issue for which i haven't get proper solution after an hour of searching.

I have a MySQL database table which has 'Long Text' column. To use less space for storing content of files in that text column the following compression approach has been used in PHP to store the content.

$compressed_content = bzcompress($content);
$db_compressed_content = addslashes($compressed_content);

The 'db_compressed_content' is stored in database using PHP itself.

Now i am in a position to utilize the database contents using Django. I was able to come up with the model class to represent the table. 'TextField' is used to represent that particular column.

Here is my exact issue, i used 'bz2.decompress()' of python to decompress and to get the text content but am getting 'UnicodeEncodeError' under django when i tried to do that.

FYI, the charset used to store content in database using PHP was 'latin-1'.

Answer 1

Answer: Django uses 'utf-8' as the default charset for the database, so if your database uses any other charset (mostly legacy database would be configured with 'latin1') then the charset need to be explicitly mentioned under database settings. Settings Ex:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'test_db',
        'USER': 'root',
        'PASSWORD': '',
        'HOST': '',
        'PORT': '3306',
        'OPTIONS': {
                    'charset': 'latin1',
                    'use_unicode': True, },
    }, 
}

In addition to this, if you don't need to use unicode then you can set the 'use_unicode' to False but i guess that is not recommended.

Cheers !!!