How to convert an array of hex numbers into array of bytes-python

Question

I have an array of hex values in python, and I want to convert it into an array of those values broken up into individual bytes, stored as strings.

For example:

Given the input

hexValueAr = [0x085F80, 0x0109A53, 0x0188C6D]

I would like the output

bytesAr = ['08', '5F', '80', ...]

I was wondering if there is a built-in python function to do this conversion, or some other simple way to do it?

EDIT: My input values are integers stored as hex values in an array. My input array has 48 of these integers, I just shortened it to make it easier to read. Hope this helps.

Answer 1

The fact that your ints are encoded in hex is irrelevant to the question.

hexValueAr = [0x085F80, 0x0109A53, 0x0188C6D]

and

hexValueAr = [548736, 1088083, 1608813]

result in the same list.

Python integers have a method to convert to bytes already: int.to_bytes. Converting to a string afterwards is pretty easy (just use str.format or hex). The only catch is that you need to know the number of bytes to use up-front. In your case, you can either use the number 3, or you can use the method int.bit_length to calculate it.

Here is one possible implementation of the above:

from math import ceil

bytesAr = []
for num in hexValueAr:
    count = ceil(num.bit_length() / 8)
    bytes = num.to_bytes(count, 'big')
    bytesAr.extend('{:02X}'.format(x) for x in bytes)

This produces the result

['08', '5F', '80', '10', '9A', '53', '18', '8C', '6D']

The order here is big-endian: the highest byte of the integer is in the lowest index of the array.

If you want to do it as a one-liner, you can do:

bytesAr = ['{:02X}'.format(x) for num in hexValueAr for x in num.to_bytes(ceil(num.bit_length() / 8), 'big')]

One possible caveat here is that different numbers in the input may give you different numbers of elements in the output. For example, if your input is [0x00AB, 0x109C], the first element will result in one string, while the second will result in 2. If you want to avoid this, find the maximum number of bytes required to encode one of your integers, and use that instead of ceil(num.bit_length() / 8):

byte_count = ceil(max(hexValueAr).bit_length() /  8))
bytesAr = ['{:02X}'.format(x) for num in hexValueAr for x in num.to_bytes(byte_count, 'big')]

In this case, an input like [0x00AB, 0x109C] will result in a bunch of zeros, but equal number of bytes per number: ['00', ''AB', '10', '9C'].

Answer 2

def int_to_hexbytes(i):
    s = '%X' % i
    if len(s) % 2:
        s = '0' + s
    return [s[i:i+2] for i in range(0, len(s), 2)]

def words_to_hexbytes(a):
    ret = []
    for i in a:
        ret += int_to_hexbytes(i)
    return ret

assert int_to_hexbytes(0x0123456789ABCDEF) == ['01', '23', '45', '67', '89', 'AB', 'CD', 'EF']
assert int_to_hexbytes(0x123456789ABCDEF0) == ['12', '34', '56', '78', '9A', 'BC', 'DE', 'F0']

assert words_to_hexbytes([0x123, 0x456, 0x78]) == ['01', '23', '04', '56', '78']
assert words_to_hexbytes([0x085F80, 0x0109A53, 0x0188C6D]) == ['08', '5F', '80', '10', '9A', '53', '18', '8C', '6D']

int_to_hexbytes uses the % formatting operator to convert the integer parameter into a string containing its representation in hexadecimal; and then uses a list comprehension to split that string into a list every second character (i.e. into each byte). Note that since the leftmost byte might be lower than 0x10, the code prepends a '0' beforehand (i.e. when the string is of odd length).

words_to_hexbytes simply iterates through the array parameter and appends the result of int_to_hexbytes.

The asserts afterwards are just a quick way to test the code and, at the same time, show the reader the expected results.