Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Django dumpdata UTF-8 (Unicode)

Is there a easy way to dump UTF-8 data from a database?

I know this command:

manage.py dumpdata > mydata.json

But the data I got in the file mydata.json, Unicode data looks like:

"name": "\u4e1c\u6cf0\u9999\u6e2f\u4e94\u91d1\u6709\u9650\u516c\u53f8"

I would like to see a real Unicode string like 全球卫星定位系统 (Chinese).

like image 799
icn Avatar asked Jan 26 '10 04:01

icn


4 Answers

After struggling with similar issues, I've just found, that xml formatter handles UTF8 properly.

manage.py dumpdata --format=xml > output.xml

I had to transfer data from Django 0.96 to Django 1.3. After numerous tries with dump/load data, I've finally succeeded using xml. No side effects for now.

Hope this will help someone, as I've landed at this thread when looking for a solution..

like image 97
Tisho Avatar answered Nov 18 '22 20:11

Tisho


django-admin.py dumpdata yourapp could dump for that purpose.

Or if you use MySQL, you could use the mysqldump command to dump the whole database.

And this thread has many ways to dump data, including manual methods.

UPDATE: because OP edited the question.

To convert from JSON encoding string to human readable string you could use this:

open("mydata-new.json","wb").write(open("mydata.json").read().decode("unicode_escape").encode("utf8"))
like image 27
YOU Avatar answered Nov 18 '22 22:11

YOU


You need to either find the call to json.dump*() in the Django code and pass the additional option ensure_ascii=False and then encode the result after, or you need to use json.load*() to load the JSON and then dump it with that option.

like image 6
Ignacio Vazquez-Abrams Avatar answered Nov 18 '22 20:11

Ignacio Vazquez-Abrams


Here I wrote a snippet for that. Works for me!

like image 5
dir01 Avatar answered Nov 18 '22 20:11

dir01