Kotlin - How to split list of strings based on the total length of characters

Question

I am writing a program in Kotlin that is posting messages to a rest endpoint.

val messages : List<String> = getMessageToSend();
webClient
    .post()
    .uri { builder -> builder.path("/message").build() }
    .bodyValue(PostMessageRequest(
        messages.joinToString("\n")
    ))
    .exchange()
    .block()

However, the rest endpoint has a limit on the maximum size of messages sent. I'm fairly new to Kotlin, but I was looking for a functional way to achieve this, and i'm struggling. I know how I would write this in Java, but i'm keen to do it right. I want to split the messages list into a list of lists, with each list limited to the maximum size allowed and only whole strings added, and then post them individually. I've had a look at methods like chunked, but that doesn't seem flexible enough to achieve what i'm trying to do.

For example, if my message was [this, is, an, example] and the limit was 10, i'd expect my list of lists to be [[this, is an], [example]]

Any suggestions would be massively appreciated.

Answer 1

This looks rather like a situation I've hit before.  To solve it, I wrote the following general-purpose extension function:

/**
 * Splits a collection into sublists not exceeding the given size.  This is a
 * generalisation of [List.chunked]; but where that limits the _number_ of items in
 * each sublist, this limits their total size, according to a given sizing function.
 *
 * @param maxSize None of the returned lists will have a total size greater than this
 *                (unless a single item does).
 * @param size Function giving the size of an item.
 */
inline fun <T> Iterable<T>.chunkedBy(maxSize: Int, size: T.() -> Int): List<List<T>> {
    val result = mutableListOf<List<T>>()
    var sublist = mutableListOf<T>()
    var sublistSize = 0L
    for (item in this) {
        val itemSize = item.size()
        if (sublistSize + itemSize > maxSize && sublist.isNotEmpty()) {
            result += sublist
            sublist = mutableListOf()
            sublistSize = 0
        }
        sublist.add(item)
        sublistSize += itemSize
    }
    if (sublist.isNotEmpty())
        result += sublist

    return result
}

The implementation's a bit hairy, but it's pretty straightforward to use.  In your case, I expect you'd do something like:

messages.chunkedBy(1024){ length + 1 }
        .map{ it.joinToString("\n") }

to give a list of strings, each no more than 1024 chars^*. (The + 1 is of course to allow for the newline characters.)

I'm surprised something like this isn't in the stdlib, to be honest.

^{(* Unless any of the initial strings is longer.)}