how to store data from std::vector<short> in std::vector<uint8_t>

Question

What I want to do is store the data in a std::vector<short> in a std::vector<uint8_t>, splitting each short into two uint8_t values. I need to do this because I have a network application that will only send std::vector<uint8_t>'s, so I need to convert to uint8_t to send and then convert back when I receive the uint8_t vector.

Normally what i would do (and what I saw when I looked up the problem) is:

std::vector<uint8_t> newVec(oldvec.begin(),oldvec.end());

However, if i understand correctly this will take each individual short value, truncate to the size of a uint8_t, and make a new vector of half the amount of data and the same number of entries, when what i want is the same amount of data with twice as many entries.

solutions that include a way to reverse the process and that avoid copying as much as possible would help a lot. Thanks!

Answer 1

to split something at the 8 bit boundary, you can use right shifts and masks, i.e.

uint16_t val;
uint8_t low = val & 0xFF;
uint8_t high = (val >> 8) & 0xFF;

now you can put your high and low into the second vector in your order.

Answer 2

For splitting and merging, you would have the following:

unsigned short oldShort;
uint8_t char1 = oldShort & 0xFF; // lower byte
uint8_t char2 = oldShort >> 8; // upper byte

Then push the two parts onto the vector, and send it off to your network library. On the receiving end, during re-assembly, you would read the next two bytes off of the vector and combine them back into the short.

Note: Make sure that there are an even number of elements on the received vector such that you didn't obtain corrupted/modified data during transit.

// Read off the next two characters and merge them again
unsigned short mergedShort = (char2 << 8) | char1;

Answer 3

I need to do this because I have a network application¹ that will only send std::vector's

Besides masking and bit shifting you should take endianess into account when sending stuff over the wire.

The network representation of data is usually big endian. So you can always put the MSB first. Provide a simple function like:

std::vector<uint8_t> networkSerialize(const std::vector<uint16_t>& input) {
    std::vector<uint8_t> output;
    output.reserve(input.size() * sizeof(uint16_t)); // Pre-allocate for sake of
                                                     // performance
    for(auto snumber : input) {
        output.push_back((snumber & 0xFF00) >> 8); // Extract the MSB
        output.push_back((snumber & 0xFF)); // Extract the LSB
    }
    return output;
}

and use it like

std::vector<uint8_t> newVec = networkSerialize(oldvec);

See live demo.

---

¹⁾_{Emphasis mine}

Answer 4

Disclaimer: People are talking about "network byte order". If you send something huger than 1 byte, of course you need to take network endiannes into account. However, as far as I understand the limitation "network application that will only send std::vector<uint8_t>" explicitly states that "I don't want to mess with any of that endianness stuff". uint8_t is already a one byte and if you send a sequence of bytes in an one order, you should get them back in the exactly same order. This is helpful: sending the array through a socket.
There can be different system endianness on client and server machines but OP said nothing about it so that is a different story...

Regarding the answer: Assuming all "endianness" questions are closed. If you just want to send a vector of shorts, I believe, VTT`s answer will perform the best. However, if std::vector<short> is just a particular case, you can use pack() function from my answer to a similar question. It packs any iterable container, string, C-string and more... into a vector of bytes and does not perform any endiannes shenanigans.
Just include byte_pack.h and then you can use it like this:

#include "byte_pack.h"

void cout_bytes(const std::vector<std::uint8_t>& bytes)
{
    for(unsigned byte : bytes) {
        std::cout << "0x" << std::setfill('0') << std::setw(2) << std::hex
                   << byte << " ";
    }
    std::cout << std::endl;
}


int main()
{
    std::vector<short> test = { (short) 0xaabb, (short) 0xccdd };
    std::vector<std::uint8_t> test_result = pack(test);

    cout_bytes(test_result); // -> 0xbb 0xaa 0xdd 0xcc (remember of endianness)

    return 0;
}