Writing large strings with DataOutputStream

Question

I've been doing some socket programming to transmit information across the wire. I've run into a problem with DataOutputStream.writeUTF(). It seems to allow strings of up to 64k but I have a few situations where I can run over this. Are there any good alternatives that support larger strings or do I need to roll my own?

Answer 1

It actually uses a two bytes to write the length of the string before using an algorithm that compacts it into one, two or three bytes per character. (See the documentation on java.io.DataOutput) It is close to UTF-8, but even though documented as being so, there are compatibility problems. If you are not terribly worried about the amount of data you will be writing, you can easily write your own by writing the length of the string first, and then the raw data of the string using the getBytes method.

// Write data
String str="foo";
byte[] data=str.getBytes("UTF-8");
out.writeInt(data.length);
out.write(data);

// Read data
int length=in.readInt();
byte[] data=new byte[length];
in.readFully(data);
String str=new String(data,"UTF-8");

Answer 2

ObjectOutputStream.writeObject() properly handles long strings (verified by looking at the source code). Write the string out this way:

ObjectOutputStream oos = new ObjectOutputStream(out);
... other write operations ...
oos.writeObject(myString);
... other write operations ...

Read it this way:

ObjectInputStream ois = new ObjectInputStream(in);
... other read operations ...
String myString = (String) ois.readObject();
... other read operations ...

Another difference with DataOutputStream is that using ObjectOutputStream automatically writes a 4-byte stream header when instantiated, but its usually going to be a pretty small penalty to pay.

Answer 3

You should be able to use OutputStreamWriter with the UTF-8 encoding. There's no explicit writeUTF method, but you can set the charset in the constructor. Try

Writer osw = new OutputStreamWriter(out, "UTF-8");

where out is whatever OutputStream you're wrapping now.