Unable to retrieve Chinese texts while scraping

Question

I have created a script that scrape website: 1688.com and the problem is, the site is in Chinese so whenever i try to retrieve the text, it gives me a bunch of unicode and when i export to a CSV file, there's nothing in the file. My code:

# -*- coding: utf-8 -*-
import csv
from urllib import urlopen
from bs4 import BeautifulSoup as BS

csv_content = open('content.csv', 'w+')
writer_content = csv.writer(csv_content)

url = urlopen('https://fuzhuang.1688.com/nvzhuang?
spm=a260k.635.1998214976.1.7eqUGT')
html = BS(url, 'lxml')
container = html.find('ul', {'class' : 'ch-box fd-clr'})
offers = container.find_all('div', {'class' : 'ch-offer-body'})
lst = []

for offer in offers:
    offer_box = offer.find('div', {'component-name' : '@alife/ocms-
component-1688-pc-ch-offer-pic'})
    images = offer_box.find('img')['src']
    title = offer.find('div', {'class' : 'ocms-component-1688-pc-ch-offer-
title-0-1-11'}).text
    price = offer.find('div', {'class' : 'ocms-component-1688-pc-ch-offer-
price-0-1-14'}).text
    lst.append(price)

for item in lst: writer_content.writerow([item])

print lst 

The output is

[u'\n\n\n\xa5\n109.00\n\n\n\u6210\u4ea4\n329\n\u4ef6\n\n\n', u'\n\n\n\xa5\n56.00\n\n\n\u6210\u4ea4\n195\n\u4ef6\n\n\n', u'\n\n\n\xa5\n83.00\n\n\n\u6210\u4ea4\n109\n\u4ef6\n\n\n', u'\n\n\n\xa5\n69.00\n\n\n\u6210\u4ea4\n208\n\u4ef6\n\n\n', u'\n\n\n\xa5\n46.00\n\n\n\u6210\u4ea4\n204\n\u4ef6\n\n\n', u'\n\n\n\xa5\n45.00\n\n\n\u6210\u4ea4\n54\n\u4ef6\n\n\n', u'\n\n\n\xa5\n82.00\n\n\n\u6210\u4ea4\n38\n\u4ef6\n\n\n', u'\n\n\n\xa5\n48.90\n\n\n\u6210\u4ea4\n318\n\u4ef6\n\n\n']

And i have already tried encoding and decoding utf-8, i would really appreciate it if you show me how to solve this problem.

Answer 1

this code will save chinese symbols to txt:

for Python3:

         ...
(all your code above)
for i in range(len(lst)):    
    lst[i]=lst[i].replace('\n','') #getting rig of `'\n'` newlines

writing to txt:

with open(r'C:\Users\Username\list.txt','w',newline='',encoding='utf-8-sig') as f:
    for i in lst:
        f.write(i+'\t')

for Python2:

import unicodecsv as ucsv
with open(r'C:\Users\Username\list1.txt','wb') as f:
    w = ucsv.writer(f,encoding='utf-8-sig')
    for i in lst:
        w.writerow([i+'\t'])