Python 3 email body encoding

Question

I am working on setting up a script that forwards incoming mail to a list of recipients.

Here's what I have now:

I read the email from stdin (that's how postfix passes it):

email_in = sys.stdin.read()

incoming = Parser().parse(email_in)

sender = incoming['from']
this_address = incoming['to']

I test for multipart:

if incoming.is_multipart():
    for payload in incoming.get_payload():
        # if payload.is_multipart(): ...
        body = payload.get_payload()
else:
    body = incoming.get_payload(decode=True)`

I set up the outgoing message:

msg = MIMEMultipart()
msg['Subject'] = incoming['subject']
msg['From'] = this_address
msg['reply-to'] = sender
msg['To'] = "[email protected]"
msg.attach(MIMEText(body.encode('utf-8'), 'html', _charset='UTF-8'))

s = smtplib.SMTP('localhost')
s.send_message(msg)
s.quit()

This works pretty well with ASCII characters (English text), forwards it and all.

When I send non-ascii characters though, it gives back gibberish (depending on email client bytes or ascii representations of the utf-8 chars)

What can be the problem? Is it on the incoming or the outgoing side?

Answer 1

The problem is that many email clients (including Gmail) send non-ascii emails in base64. stdin on the other hand passes everything into a string. If you parse that with Parser.parse(), it returns a string type with base64 inside.

Instead the optional decode argument should be used on the get_payload() method. When that is set, the method returns a bytes type. After that you can use the builtin decode() method to get utf-8 string like so:

body = payload.get_payload(decode=True)
body = body.decode('utf-8')

There is great insight into utf-8 and python in Ned Batchelder's talk.

My final code works a bit differently, you can check that, too here.