Getting 500 error from HttpClient, works in browser

Question

I'm using Apache HttpClient to try to submit some post data to a server. Unfortunately, I don't have access to the server to get any log information so that won't be possible.

If I go through this process with Firefox, it works fine. (I do get a 302 warning on this particular page)

I have matched the Request headers of both Firefox and my program.

Firefox Request Headers:

Host: server ip
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Referer: https://server ip/
Content-Type: application/x-www-form-urlencoded
Content-Length: 407
Cookie: sessionId=blahblah
Connection: keep-alive
Upgrade-Insecure-Requests: 1

My Programs Request Headers shown from context.getRequest().getAllHeaders();

Host: server ip
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:55.0) Gecko/20100101 Firefox/55.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Referer: https://server ip/
Content-Type: application/x-www-form-urlencoded
Connection: keep-alive
Upgrade-Insecure-Requests: 1
Content-Length: 406
Cookie: sessionId=blahblah

I have matched the body of the request by comparing the output of EntityUtils.toString(httpPost.getEntity(), "UTF-8"); and the built in tool for Firefox's tool to look at the request body, and they match almost character for character. (Just a slight difference in the session id which is expected as it's not using the same session.)

I'm not sure what else to check. What could be causing the server to behave differently between the Browser and the program?

Below is my code for the POST request.

HttpPost httpPost = new HttpPost("https://" + getIp() + "");

List<NameValuePair> params = new ArrayList<NameValuePair>();
params.add(new BasicNameValuePair("FTPUsername", "blah"));
params.add(new BasicNameValuePair("FTPPassword", "blah"));
params.add(new BasicNameValuePair("FormButtonSubmit", "OK"));
httpPost.setEntity(new UrlEncodedFormEntity(params));

httpPost.setHeader("Host", ip);
httpPost.setHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:55.0) Gecko/20100101 Firefox/55.0");
httpPost.setHeader("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"); 
httpPost.setHeader("Accept-Language", "en-US,en;q=0.5"); 
httpPost.setHeader("Accept-Encoding", "gzip, deflate, br"); 
httpPost.setHeader("Referer", referer);
httpPost.setHeader("Content-Type", "application/x-www-form-urlencoded");
httpPost.setHeader("Connection", "keep-alive");
httpPost.setHeader("Upgrade-Insecure-Requests", "1");

//Response
HttpResponse response = getHttpClient().execute(httpPost, LoginRequest.context);
int statusCode = response.getStatusLine().getStatusCode();
httpPost.releaseConnection();

I realize this could probably be many things since 500 is a server error, but it's got to be something I'm submitting wrong or I'm missing something as it works perfectly in the browser.

Answer 1

302 "warning" is actually a redirect. HTTP Client does do redirect automatically, you must flag the RedirectStrategy, For HttpClient 4.3:

HttpClient instance = HttpClientBuilder.create()
                     .setRedirectStrategy(new LaxRedirectStrategy()).build();

see examples in answer and w3 docs:

If the 302 status code is received in response to a request other than GET or HEAD, the user agent MUST NOT automatically redirect the request unless it can be confirmed by the user