VB.net returns chr(255) as chr(63)

Question

I am writing a program in visual basic and have run into an odd problem. I am sending strings via serial port to a telescope mount. When I send the check string, the scope can return either chr(0) or chr(255). This works fine in python and c++ returning chr(255). However when I run the script in visual basic it returns chr(0) or chr(63).

below are two identical functions one in python and one in visual basic.

Can anyone give me an idea why visual basic returns 63 instead of 255?

function in python (returns correct values 0 and 255):

global d, check
d=chr(80)+chr(4)+chr(16)+chr(2)+chr(1)+chr(112)+chr(252)+chr(0)
check=chr(80) + chr(1) + chr(16) + chr(19) + chr(0) + chr(0) + chr(0)+chr(1)


def test():
    ser.write(d)
    time.sleep(.1)
    print ser.readline()
    ser.write(check)
    time.sleep(.1)
    out=ser.readline()[0]
    print "out=",ord(out)
    while out == chr(0):
            print "out = ", ord(out)
            ser.write(check)
        time.sleep(.1)
            out=ser.readline()[0]
        print "out=",ord(out)
    print "out is now", ord(out)
    ser.readline()

script in visual basic (returns incorrect values 0 and 63)

Public Sub test()
    Dim out As Char
    Dim d As String = Chr(80) + Chr(4) + Chr(16) + Chr(2) + Chr(1) + Chr(112) + Chr(252) + Chr(0)
    Dim check As String = Chr(80) + Chr(1) + Chr(16) + Chr(19) + Chr(0) + Chr(0) + Chr(0) + Chr(1)
    port.Write(d)
    Threading.Thread.Sleep(100)
    Console.Write(port.ReadTo("#"))
    port.Write(check)
    Threading.Thread.Sleep(100)
    out = port.ReadTo("#")
    Console.Write(vbNewLine & "out=" & out)
    While out = Chr(0)
        Console.Write("out = " & Convert.ToInt16(out))
        port.Write(check)
        Threading.Thread.Sleep(0.1)
        out = port.ReadTo("#")
        Console.Write("out=" & Convert.ToInt16(out))
    End While
    Console.Write("out is now" & Convert.ToInt16(out))
    port.ReadLine()


End Sub

Answer 1

The .NET SerialPort class has an Encoding associated with it that is used for conversion between text and bytes sent on the port. The documentation says that it's used for "pre- and post-transmission conversion of text". It's not exactly clear that the Encoding is used to convert bytes received into text for the string returned fromt ReadTo() (but that might be what is meant by "post-transmission").

In any case, I'm pretty sure that that is what's happening to your 255 character. The ReadTo() method uses the Encoding to convert the data received to text to put in the string. The default Encoding is ASCIIEncoding, which only deals with characters in the range 0x00 - 0x7f. Bytes outside of that range will be encoded into '?' characters, which happens to have an ASCII value of 63.

Use the ReadByte() method instead to read the data from the port without any text encoding getting involved.

Update: I have found a blog posting from someone on the BCL team that confirms this and also points out an encoding object that will pass 8-bit data unmodified. Ryan Byington wrote in an article called "SerialPort Encoding" on the Base Class Library (BCL) Blog on 26 May 2006:

The SerialPort class uses the encoding specified by the SerialPort.Encoding property to convert strings and character arrays to bytes to send across the serial port. This encoding also gets used when reading from the serial port to convert bytes received by the serial port to strings and character arrays.

By default the SerialPort class uses ASCII encoding which converts all characters with the value of 0-127 to a single byte with the same value. So "Hello" gets converted to the byte array {72, 101, 108, 108, 111}. However every other character gets converted to a byte with the value of 63 or the '?' character. This applies for reading as well so if the serial port received the byte array {72, 101, 108, 108, 111} it will convert this to "Hello" and any byte with a value greater then 127 will get converted to the '?' character.

The ASCII encoding works perfectly fine for most serial devices but there are still quite a few device that either send bytes with values beyond 127 or you need to send bytes with values greater then 127. You have two options; one would be to use the API’s that deal with byte[] and bypass the Encoding, or to use the correct Encoding. However finding the correct encoding is a bit tricky as most encodings that deal with values above 127 will use multiple bytes which probably won’t work with the device attached to the serial port. The only encoding that converts all characters with a value 0-255 to a single byte with the corresponding value and vice versa when converting bytes to characters is the "Western European (ISO)" encoding. Use the following code to use this Encoding with the SerialPort class:

SerialPort mySerialPort = new SerialPort("COM1");
mySerialPort.Encoding = Encoding.GetEncoding(28591);