I have a Vertx request and I need to calculate an externally visible (public) URL

Question

I am using Vertx 3 with Kotlin, and at times I need to return a specific URI from the perspective of the public URL which is not the same as what the Vertx-web request thinks my URL is. This is likely due to my load balancer or proxy receiving one URL, and then forwarding to my application on an internal URL.

So if I do this:

val publicUrl = context.request().absoluteURI() 

I end up with a URL like http://10.10.103.22:8080/some/page instead of https://app.mydomain.com/some/page. Everything is wrong about that URL!

I found a header that supposedly tell me more about the original request such as X-Forwarded-Host but it only includes app.mydomain.com or sometimes it has the port app.mydomain:80 but that isn't enough to figure out all parts of the URL, I end up with something like http://app.mydomain.com:8080/some/page which is still not the correct public URL.

I also need to handle not just my current URL, but peer URL's, like while on page "something/page1" go to "something/page2" on same server. The same problems mentioned about when I try to resolve to another URL because important parts of the public URL are unobtainable.

Is there a method in Vertx-web I'm missing to determine this public URL, or some idiomatic way to solve this?

I'm coding in Kotlin, so any examples for that language are great!

Note: this question is intentionally written and answered by the author (Self-Answered Questions), so that solutions for interesting problems are shared in SO.

Answer 1

This is a more complicated issue, and the logic is the same for most App servers if they do not already provide an URL externalization function.

To do this correctly, you need to handle all of these headers:

• X-Forwarded-Proto (or X-Forwarded-Scheme: https, and maybe oddballs like X-Forwarded-Ssl: on, Front-End-Https: on)
• X-Forwarded-Host (as "myhost.com" or "myhost.com:port")
• X-Forwarded-Port

And if you want to resolve and return a URL that is not the current one you need to also consider:

• partial without host, for example "/something/here" or "under/me" resolving to the servers public protocol, host, port as well as that abosolute or relative path
• partial with host/port, for example "//somehost.com:8983/thing" would add the same scheme (http/https) as this server and keep the rest
• full, URL's that are fully qualified are returned untouched, so they are safe to pass to this function ("http://...", "https://...") and won't be modified

Here is a pair of extension functions to RoutingContext that will handle all these cases and fall back when the load balancer / proxy headers are not present so will work in both cases of direct connections to the server and those going through the intermediary. You pass in the absolute or relative URL (to the current page) and it will return a public version of the same.

// return current URL as public URL
fun RoutingContext.externalizeUrl(): String {
    return externalizeUrl(URI(request().absoluteURI()).pathPlusParmsOfUrl())
}

// resolve a related URL as a public URL
fun RoutingContext.externalizeUrl(resolveUrl: String): String {
    val cleanHeaders = request().headers().filterNot { it.value.isNullOrBlank() }
            .map { it.key to it.value }.toMap()
    return externalizeURI(URI(request().absoluteURI()), resolveUrl, cleanHeaders).toString()
}

Which call an internal function that does the real work (and is more testable since there is no need to mock the RoutingContext):

internal fun externalizeURI(requestUri: URI, resolveUrl: String, headers: Map<String, String>): URI {
    // special case of not touching fully qualified resolve URL's
    if (resolveUrl.startsWith("http://") || resolveUrl.startsWith("https://")) return URI(resolveUrl)

    val forwardedScheme = headers.get("X-Forwarded-Proto")
            ?: headers.get("X-Forwarded-Scheme")
            ?: requestUri.getScheme()

    // special case of //host/something URL's
    if (resolveUrl.startsWith("//")) return URI("$forwardedScheme:$resolveUrl")

    val (forwardedHost, forwardedHostOptionalPort) =
            dividePort(headers.get("X-Forwarded-Host") ?: requestUri.getHost())

    val fallbackPort = requestUri.getPort().let { explicitPort ->
        if (explicitPort <= 0) {
            if ("https" == forwardedScheme) 443 else 80
        } else {
            explicitPort
        }
    }
    val requestPort: Int = headers.get("X-Forwarded-Port")?.toInt()
            ?: forwardedHostOptionalPort
            ?: fallbackPort
    val finalPort = when {
        forwardedScheme == "https" && requestPort == 443 -> ""
        forwardedScheme == "http" && requestPort == 80 -> ""
        else -> ":$requestPort"
    }

    val restOfUrl = requestUri.pathPlusParmsOfUrl()
    return URI("$forwardedScheme://$forwardedHost$finalPort$restOfUrl").resolve(resolveUrl)
}

And a few related helper functions:

internal fun URI.pathPlusParmsOfUrl(): String {
    val path = this.getRawPath().let { if (it.isNullOrBlank()) "" else it.mustStartWith('/') }
    val query = this.getRawQuery().let { if (it.isNullOrBlank()) "" else it.mustStartWith('?') }
    val fragment = this.getRawFragment().let { if (it.isNullOrBlank()) "" else it.mustStartWith('#') }
    return "$path$query$fragment"
}

internal fun dividePort(hostWithOptionalPort: String): Pair<String, Int?> {
    val parts = if (hostWithOptionalPort.startsWith('[')) { // ipv6
        Pair(hostWithOptionalPort.substringBefore(']') + ']', hostWithOptionalPort.substringAfter("]:", ""))
    } else { // ipv4
        Pair(hostWithOptionalPort.substringBefore(':'), hostWithOptionalPort.substringAfter(':', ""))
    }
    return Pair(parts.first, if (parts.second.isNullOrBlank()) null else parts.second.toInt())
}

fun String.mustStartWith(prefix: Char): String {
    return if (this.startsWith(prefix)) { this } else { prefix + this }
}