Proxy password containing `@` in JVM

Question

I have some Scala code that has managed to successfully negotiate the (NTLM) proxy and access the internet, specifying the username and password as so:

// Based upon http://rolandtapken.de/blog/2012-04/java-process-httpproxyuser-and-httpproxypassword
Authenticator.setDefault(new Authenticator() {

  override protected def getPasswordAuthentication: PasswordAuthentication = {

    if (getRequestorType eq RequestorType.PROXY) {
      val prot = getRequestingProtocol.toLowerCase

      // This allows us to only return a PasswordAuthentication when we have
      // all four of the host, port, user and password _and_ the host and
      // port match the actual proxy wanting authentication.
      for {
        host <- Option(System.getProperty(prot + ".proxyHost"))
        if getRequestingHost.equalsIgnoreCase(host)
        port <- Try(augmentString(System.getProperty(prot + ".proxyPort")).toInt).toOption
        if port == getRequestingPort
        user <- Option(System.getProperty(prot + ".proxyUser"))
        pass <- Option(System.getProperty(prot + ".proxyPassword"))
      } yield return new PasswordAuthentication(user, pass.toCharArray)
    }

    // One of the if-statements failed.  No authentication for you!
    null
  }
})

However, I've now been given a new system username/password combination to use, and the password contains an @ in it. I've tried using the password directly, escaping it (with both \ and \\ in case a double-level of escaping was needed), url-encoding it (i.e. replacing @ with %40) and even HTML-encoding (@ and @) to no avail.

I know the password works, as it's used on other systems for non-JVM applications to access the internet by setting the http_proxy variable, but it doesn't work here.

Any ideas?

---

EDIT

To try and clear some things up, I've tried simplifying my code to this:

Authenticator.setDefault(new Authenticator() {

  def urlEncode(str: String): String = {
    val res = URLEncoder.encode(str, "UTF-8")
    // To confirm it's working
    println(s"${str} -> ${res}")
    res
  }

  override protected def getPasswordAuthentication: PasswordAuthentication = {

    if (getRequestorType eq RequestorType.PROXY) {

      return new PasswordAuthentication(urlEncode("username"), urlEncode("p@ssword").toCharArray);
    }
    null
  }

})

The environment that this is being run in is in a Spark cluster (run with spark-submit) on a Linux box. The proxy is a corporate NTLM one.

If I use known a username and password combination that doesn't contain @ then this works. If I change it to one containing an @ then it fails.

I've tried making val res = str inside the urlEncode function (in case it doesn't need to be URL encoded), tried having \\@ (with and without URL encoding) and ^@ (with and without URL encoding). Every single time I get an exception of Unable to tunnel through proxy. Proxy returns "HTTP/1.1 407 Proxy Authorization Required".

I know that the username and password are valid as they are currently set in the https_proxy variable that is successfully used by curl, etc.

So unless the fact that the proxy is being set within a running Spark server somehow affects what happens to it, it appears to me that the JVM libraries do not support having @ in the authenticator for proxies (at the very least).

Answer 1

The problem is not within the java.net library code (EDIT: certainly for HTTP Basic proxies, I have not yet been able to test NTLM proxies). The java.net code can connect fine using passwords with "@" in them. I have put demo code below that allows you to verify this claim.

You do not need to escape the string value that you pass into java.net.PasswordAuthentication, you should pass your password in plaintext there. The java.net library code will take care of encoding your password as necessary when sending it over the network to the proxy (see demo code below to verify this claim).

I believe that your problem must be in the way that you are configuring your system outside of the code that you have included in the question so far.

For example, have you passed the proxy hostname to the JVM or a nearby system in such a way that it is confused by the "@" symbol?

Please can you provide more context?

Demo code to verify that the java.net library code can cope with "@" in the password

This code includes instructions on setting up Fiddler2 as an HTTP proxy on your local machine, configuring Fiddler2 to require a password, and connecting through that proxy using the java.net library classes.

The code runs successfully for me as-is, and fails if I change the "password" variable to an incorrect password.

import java.io.BufferedReader;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.Authenticator;
import java.net.PasswordAuthentication;
import java.net.URL;
import java.util.Base64;

public class q45749081 {

    public static void main(String[] args) throws Exception {

        // Start Fiddler HTTP proxy https://www.telerik.com/download/fiddler
        // Click "rules" -> "require proxy authentication"
        // Type into the "QuickExec" box below Fiddler's Web Sessions list:
        //    prefs set fiddler.proxy.creds dXNlcm5hbWU6cEBzc3dvcmQ=
        //
        // This sets Fiddler to require auth with "username", "p@ssword"
        //
        // See https://stackoverflow.com/questions/18259969/changing-username-and-password-of-fiddler-proxy-server

        // Note that you must start a new process each time you change the password
        // here, as sun.net.www.protocol.http.HttpURLConnection caches the proxy password
        // for the lifetime of the JVM process
        String password = "p@ssword";

        System.out.println(
            "prefs set fiddler.proxy.creds " +
            Base64.getEncoder().encodeToString("username:p@ssword".getBytes()));

        Authenticator.setDefault(new Authenticator() {
            @Override
            protected PasswordAuthentication getPasswordAuthentication() {
                return new PasswordAuthentication(
                    "username",
                    password.toCharArray());
            }
        });


        System.setProperty("http.proxyHost", "localhost");
        System.setProperty("http.proxyPort", "8888");

        System.out.println("Connecting to Google via authenticated proxy with password '"
            + password + "'");
        try (InputStream conn = new URL("http://www.google.com/").openStream()) {
            try (BufferedReader r = new BufferedReader(new InputStreamReader(conn))) {
                System.out.println(r.readLine());
                System.out.println("OK");
            }
        } catch (Exception e) {
            System.out.println("Failed: " + e);
        }
    }
}

First answer:

The code you have shown is taking the password from a JVM System property. How are you putting your password into that property? I suspect the problem lies there, rather than in the code you have shown.

If you are on Windows, and if you are setting the password as a command-line argument, you will need to use the DOS escape char "^", i.e.

java -Dhttp.proxyPassword=foo^@bar -jar myapp.jar

If you use another mechanism to provide the password to Java, you may need a different escaping scheme.