wireshark lua string:byte() error

Question

I have a string problem when writing my lua dissector. My packet looks like:

0000   00 00 00 69 00 10 00 01 00 00 00 ed 00 00 00 0c
0010   bf a6 5f ...

When debugging, the tvb looks the same

The byte at offset 0x10 is 0xbf, but in my dissector function I got different result, here' my code:

local str = buf(0x10):string()
local x = string.byte(str, 1)

the variable x should be 0xbf, but it's 0xef, and some other offset are also 0xef:

local str = buf(0x11):string()
local x = string.byte(str, 1) -- also get 0xef, should be 0xa6

local str = buf(11):string()
local x = string.byte(str, 1) -- also get 0xef, should be 0xed

Seems big values will always get 0xef as result, like 0xa6/0xbf/0xed...

And small values will be correct, like 0x69/0x5f/0x0c...

I'm using the latest wireshark 2.0, is this a bug?

Answer 1

I don't know much about Wireshark in particular, but I have a pretty good idea what's going on.

You are using Wireshark's tvbrange:string([encoding]) function. The documentation I have found on the Wireshark website says that the default encoding is ENC_ASCII. Bytes in the range of 0x80-0xFF (for which you have reported problems) are not valid ASCII.

What Wireshark is probably doing is converting these to U+FFFD, Unicode's "Replacement Character". This is a standard practice for representing an unknown character in a Unicode string.

Then, Wireshark is probably encoding this string as UTF-8 when returning to Lua. The first byte of U+FFFD's UTF-8 encoding is 0xEF, so that's what you see.

If you want to get the raw byte values from a TVB, maybe try the tvbrange:bytes([encoding]) function to get the values. e.g.

local bytes = buf(0x10):bytes()
local x = bytes:get_index(0) -- maybe 1, I'm not sure if it would be 0 or 1 indexed

There also may be some encoding you can pass to tvbrange:string that would do what you want, but I couldn't find any good reference for this.