Python 3 Base64 decode messing up newline characters

Question

I'm trying to decode a base64 multi-line file through the standard python library, however only the first line gets decoded, and the rest gets dumped for no reason.

Why is this?

The file before it gets encoded (what I'm trying to achieve after decoding):

dataFile.dat

VERSION: BenWin+ Version: 3.0.12.1[CR]

[CR][LF]

CREATED: 01 September 2016 12:56:27 PM[CR]

[CR][LF]

TIME CODE: 0x907e0, 0x10004, 0x38000c, 0x242001b[CR]

[CR][LF]

...

[CR] and [LF] are the character codes for Carriage Return (\r) and Line Feed (\n) respectively

I base64 encode the file using base64.b64encode and want to decode it later. Here is my code snippet.

encodedData = b'VkVSU0lPTjogQmVuV2luKyBWZXJzaW9uOiAzLjAuMTIuMQo=Cg==Q1JFQVRFRDogMDEgU2VwdGVtYmVyIDIwMTYgMTI6NTY6MjcgUE0KCg==VElNRSBDT0RFOiAweDkwN2UwLCAweDEwMDA0LCAweDM4MDAwYywgMHgyNDIwMDFiCg==Cg=='

data = base64.b64decode(encodedData)
print(data)

Which returns

b'VERSION: BenWin+ Version: 3.0.12.1\n'

Thanks in advance. Using Python 3.5

Answer 1

The problem appears to be that you are encoding each line separately and then joining those encoded strings together. A Base-64 encoded string may end in padding characters, and when the decoder sees those padding characters it assumes that's the end of the valid data, so any following data is ignored.

Here's how to Base64 encode multi-line text in Python 3. First, we need to convert the Unicode text to bytes. Then we Base64 encode all those bytes in one go. To decode, we reverse the process: first Base64 decode, then decode the resulting bytes to a Unicode string. Notice that the \r and \n have been preserved properly.

import base64

s = 'VERSION: BenWin+ Version: 3.0.12.1\r\r\nCREATED: 01 September 2016 12:56:27 PM\r\r\nTIME CODE: 0x907e0, 0x10004, 0x38000c, 0x242001b\r\r\n'
print(s)

b = base64.b64encode(s.encode('utf8'))
print(b)

z = base64.b64decode(b).decode('utf8')
print(repr(z))

output

VERSION: BenWin+ Version: 3.0.12.1
CREATED: 01 September 2016 12:56:27 PM
TIME CODE: 0x907e0, 0x10004, 0x38000c, 0x242001b

b'VkVSU0lPTjogQmVuV2luKyBWZXJzaW9uOiAzLjAuMTIuMQ0NCkNSRUFURUQ6IDAxIFNlcHRlbWJlciAyMDE2IDEyOjU2OjI3IFBNDQ0KVElNRSBDT0RFOiAweDkwN2UwLCAweDEwMDA0LCAweDM4MDAwYywgMHgyNDIwMDFiDQ0K'
'VERSION: BenWin+ Version: 3.0.12.1\r\r\nCREATED: 01 September 2016 12:56:27 PM\r\r\nTIME CODE: 0x907e0, 0x10004, 0x38000c, 0x242001b\r\r\n'