Go - How to decode/convert a txt file contains hex chars into readable string

Question

I've got a log file, where each line is a JSON. Due to some Nginx security reasons, the logs are being saved in a hexadecimal format (e.g. the char " will be converted to \x22). Here is an example of a JSON line:

 { "body_bytes_sent": "474", "params": {\x22device_id\x22: \x221234567890\x22} }

My goal:

1. Read the file line by line.

2. Convert each line to a readable format

   { "body_bytes_sent": "474", "params" : {"device_id": "1234567890"} }
   

3. Convert this string into a JSON object so I could manipulate it.

Any help will be appreciated.

Answer 1

You can use strconv.Unquote to convert the string to a normal one:

package main

import (
    "encoding/json"
    "fmt"
    "strconv"
)

func main() {

    // this is what your input string looks like...
    qs := "{\\x22device_id\\x22: \\x221234567890\\x22}"

    // now let's convert it to a normal string
    // note that it has to look like a Go string literal so we're
    // using Sprintf
    s, err := strconv.Unquote(fmt.Sprintf(`"%s"`, qs))
    if err != nil {
        panic(err)
    }
    fmt.Println(s)

    // just for good measure, let's see if it can actually be decoded.
    // SPOILER ALERT: It decodes just fine!
    var v map[string]interface{}
    if err := json.Unmarshal([]byte(s), &v); err != nil {
        panic(err)

    }
    fmt.Println(v)
}

Playground