Convert byte in hex to actual byte [duplicate]

Question

I have a file that is written in bytes like this

\r\x00\x00\x00\xd0{"a": "test"}

which has the following bytes

[13, 0, 0, 0, -48, 123, 34, 97, 34, 58, 32, 34, 116, 101, 115, 116, 34, 125]

when this file is read into Java I'm getting everything escaped

\\r\\x00\\x00\\x00\\xd0{"a": "test"}

when I do a .getBytes() on this string I get

[92, 114, 92, 120, 48, 48, 92, 120, 48, 48, 92, 120, 48, 48, 92, 120, 100, 48, 123, 34, 97, 34, 58, 32, 34, 116, 101, 115, 116, 34, 125]

I have to convert the string to the valid bytes, I don't have the ability to change how a file is read unfortunately. I know in Python, you open a file with the 'rb' mode and you're good to go. If java has that ability, I can't use it.

So in short, how can I convert the string Java reads into the original byte array that was written to the file?

Sorry if this question is stupid simple, but I am so green when it comes to Java.

EDIT: So I believe my question is different than the proposed "duplicate question" link. It's not taking every literal value in the java string and converting that back to a byte. The string in java has been escaped by the reader. \x00 is now \\x00 which is not the same byte value. So I guess I need some way of unescaping the string?

The file as viewed in a hex editor

0000000: 5c72 5c78 3030 5c78 3030 5c78 3030 5c78  \r\x00\x00\x00\x
0000010: 6430 7b22 6122 3a20 2274 6573 7422 7d0a  d0{"a": "test"}.

The string that java gets viewed in a hex editor

0000000: 5c5c 725c 5c78 3030 5c5c 7830 305c 5c78  \\r\\x00\\x00\\x
0000010: 3030 5c5c 7864 307b 2261 223a 2022 7465  00\\xd0{"a": "te
0000020: 7374 227d 0a                             st"}.

Answer 1

In Java, you're going to have to interpret the input string to get the byte values you want.

I wrote a Java application that interprets the input string.

Here's the input string:

\r\x00\x00\x00\xd0{"a": "test"}

Here's the result:

[13, 0, 0, 0, -48, 34, 97, 34, 58, 32, 34, 116, 101, 115, 116, 34, 125]

And here's the code. You'll probably have to modify the code somewhat to handle cases you didn't put in your question.

package com.ggl.testing;

import java.io.BufferedReader;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;

public class ConvertBytes implements Runnable {

    private String fileName;

    public static void main(String[] args) {
        new ConvertBytes("bytes.txt").run();
    }

    public ConvertBytes(String fileName) {
        this.fileName = fileName;
    }

    @Override
    public void run() {
        BufferedReader br = null;

        try {
            br = new BufferedReader(new InputStreamReader(getClass()
                    .getResourceAsStream(fileName)));
            String line = "";
            while ((line = br.readLine()) != null) {
                processLine(line);
            }
        } catch (FileNotFoundException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        } finally {
            try {
                if (br != null) {
                    br.close();
                }
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }

    private void processLine(String line) {
        String[] parts = line.split("(?=\\\\)");
        List<Byte> byteList = new ArrayList<Byte>();

        for (int i = 0; i < parts.length; i++) {
            if (parts[i].equals("")) {
                continue;
            } else {
                byteList.addAll(getValue(parts[i]));
            }
        }

        Byte[] bytes = byteList.toArray(new Byte[byteList.size()]);
        System.out.println(Arrays.toString(bytes));
    }

    private List<Byte> getValue(String s) {
        List<Byte> byteList = new ArrayList<Byte>();

        if (s.startsWith("\\x")) {
            int value = Integer.valueOf(s.substring(2, 4), 16);
            if (value > 127) {
                value = value - 256;
            }
            byteList.add(Byte.valueOf((byte) value));
            if (s.length() > 4) {
                byteList.addAll(getAsciiValue(s.substring(4)));
            }
        } else if (s.equals("\\r")) {
            byteList.add(Byte.valueOf((byte) 13));
        } else if (s.equals("\\t")) {
            byteList.add(Byte.valueOf((byte) 9));
        } else {
            byteList.addAll(getAsciiValue(s));
        }

        return byteList;
    }

    private List<Byte> getAsciiValue(String s) {
        List<Byte> byteList = new ArrayList<Byte>();

        for (int i = 0; i < s.length(); i++) {
            int value = (int) s.charAt(i);
            byteList.add(Byte.valueOf((byte) value));
        }

        return byteList;
    }

}

The bytes.txt file has to be in the same directory as the Java application.

Answer 2

Looks like your have to parse the "String" line yourself.

I would have a Map of escaped characters ('\r', '\n', '\b', etc...)

private static Map<String, Byte> escapedCharacters; 
static {
    escapedCharacters = new HashMap<>();
    escapedCharacters.put("\\b", (byte)'\b');
    escapedCharacters.put("\\f", (byte)'\f');
    escapedCharacters.put("\\n", (byte)'\n');
    escapedCharacters.put("\\r", (byte)'\r');
    escapedCharacters.put("\\t", (byte)'\t');
    // Add more if needed
};

Then the following to process your file:

public static void main(String[] args) throws Exception {
    String myFile = "PathToYourFile";

    // Read your file in
    List<String> myFileLines = Files.readAllLines(Paths.get(myFile));

    // List to hold all the lines as translated bytes
    List<byte[]> myFileLinesAsBytes = new ArrayList<>();
    for (String line : myFileLines) {
        myFileLinesAsBytes.add(translateEscapedBytes(line));
    }

    // Displays all translated lines
    for (byte[] byteLine : myFileLinesAsBytes) {
        System.out.println(Arrays.toString(byteLine));
    }
    System.out.println();
}

private static byte[] translateEscapedBytes(String line) throws UnsupportedEncodingException {
    List<Byte> translatedBytes = new ArrayList<>();
    for (int i = 0; i < line.length();) {
        if (line.charAt(i) == '\\') { // Escaped byte
            String escapedByte = line.substring(i, i + 2);
            if (escapedByte.endsWith("x")) { // Hexidecimal number
                escapedByte = line.substring(i + 2, i + 4); // + 4 to get the two numbers after \x
                translatedBytes.add(hexStringToByte(escapedByte));
                i += 4;
            } else { // Escaped character
                translatedBytes.add(escapedCharacters.get(escapedByte));
                i += 2;
            }
        } else { // Non Escapted Character
            translatedBytes.add((byte)(line.charAt(i)));
            i++;
        }
    }

    // Copy List to actual byte[] to return
    byte[] result = new byte[translatedBytes.size()];
    for (int i = 0; i < translatedBytes.size(); i++) {
        result[i] = translatedBytes.get(i);
    }
    return result;
}

private static byte hexStringToByte(String s) {
    return (byte) ((Character.digit(s.charAt(0), 16) << 4) + Character.digit(s.charAt(1), 16));
}

translatedEscapedBytes() looks for the "\" character in the string and identifies that combined with the next character, you'll have an escaped character. If the escaped character is an \x, then you know the next two numbers is a hexadecimal number that needs to be converted into a byte (hexStringToByte(String s)), otherwise covert the escaped character to a byte using the map of escaped characters. All other characters are treated as non-escaped characters and just get converted to their byte value.

Results (Using the data you provided):