Checking size of SQS message batches

Question

There are limits on the size of published SQS messages. A single message can not be larger than 256 KB and a batch of messages (max of 10) also cannot exceed 256 KB. When adding messages to a collection to later publish to an SQS queue, how can I keep track of the message sizes to ensure my request stays under the limit?

I have looked at methods to get the size of an object and I know that the IDs of the failed messages will be available to me after calling sendMessageBatch(). I don't feel like either of these are very good solutions because the size of the object itself omits the overhead data from the message (that I assume also counts), and I would really like to not have to manage failed messages simply because the batch was too large. I really don't expect my messages to ever be that large but you never know.

Code snippet to batch and send:

List<SendMessageBatchRequestEntry> entries = new LinkedList<>();

And then in a loop:

SendMessageBatchRequestEntry entry = new SendMessageBatchRequestEntry();

entry.setMessageBody(gson.toJson(message));
entry.setMessageAttributes(attributes);             
entry.setId(messageId);

// How to make sure `entry` is not larger than 256 KB, and how to
// make sure adding this entry won't cause the batch to exeed 256 KB?

entries.add(entry);

And lastly:

client.sendMessageBatch(queueUrl, entries);

Answer 1

According to AWS SQS Batch notes,

The total size of all messages that you send in a single SendMessageBatch call can't exceed 262,144 bytes (256 KB).

According to this the SendBatchMessage call should not exceed 256KB. And when looking at your code snippet, you have used a List object named entries. All list objects are serializable in Java. See this answer.

You can get a good estimation of the size of the object by serializing it. The code for it is given below.

ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(baos);
oos.writeObject(entries);
oos.close();
System.out.println(baos.size());

With this you can get an approximate size in bytes, of the entries LinkedList. After that you can simply check whether the entries list exceeds 256Kb, after addding a new entry object. If so discard the entry object and call SendBatchMessage with previous entries object to SQS. The code snippet is given below.

    List<SendMessageBatchRequestEntry> entries = new LinkedList<>();
    SendMessageBatchRequestEntry entry = new SendMessageBatchRequestEntry();

    entry.setMessageBody(gson.toJson(message));
    entry.setMessageAttributes(attributes);
    entry.setId(messageId);

    entries.add(entry);
    if ( getObjectSize(entries) > 262144) {
      entries.remove(entry);
      client.sendMessageBatch(queueUrl, entries);
    } else {
      //add another entry to the entries List 
    }

  private static int getObjectSize(List object) throws IOException {
    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    ObjectOutputStream oos = new ObjectOutputStream(baos);
    oos.writeObject(object);
    oos.close();
    return baos.size();
  }