Menu
I'm using Angular 5 HttpClient and I'm trying to get an HTML response from a page that provides both json data and HTML. (Spotify auth).
When I curl, I get the expected HTML and the json payload. No matter what I try with HttpClient, all I can get is the json, which in this case is not helpful. I want to get to the HTML. I have verified my headers are identically between the command line curl and the HttpClient.
curl -vvv https://accounts.spotify.com/authorize/?client_id=xxxxxxxxxxxxxxx81ad8fc&response_type=code&redirect_uri=http://reaver.xxxxxx.com:4200/callback&state=34fFs29kd09
<!DOCTYPE html>
<html ng-app="accounts" ng-csp>
<head>
<meta charset="utf-8">
<title ng-bind="(title && (title | localize) + ' - ') + 'Spotify'">Spotify</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no">
<base href="/">
<link href="https://d2d1dxiu3v1f2i.cloudfront.net/a4a5157/css/index.css" media="screen" rel="stylesheet">
<script async defer src="https://www.google.com/recaptcha/api.js"></script>
<script async defer src="https://d2d1dxiu3v1f2i.cloudfront.net/a4a5157/js/index.js" sp-bootstrap></script>
<meta ng-non-bindable sp-bootstrap-data='{"client":{"name":"Playlist Reaver"},"country":"US","useCaptcha":false,"locales":["*"],"BON":["0","0",-795429514]}'>
</head>
<body ng-view></body>
</html>
This part of that payload is all I can get HttpClient to expose to me.
{"client":{"name":"Playlist Reaver"},"country":"US","useCaptcha":false,"locales":["*"],"BON":["0","0",-795429514]}
Which normally I would say great, but I really need access to the HTML.
How do I get the raw html back from a response like this that contains both json data and HTML?
My get call looks like this:
return this.http.get(this.apiGeneric, { params: params, observe: 'response'});
Additional info: My http headers do not appear to be added. I have made the following change and I see no XFF header in the http request headers.
// function returns Observable UserResponse object
getUrl() {
this.httpHeaders = new HttpHeaders();
//.set('Content-Type'
this.httpHeaders.set('Accept', 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8');
this.httpHeaders.set('XFF', 'testing123');
let params = new HttpParams();
const params = new HttpParams()
.set('client_id', this.clientId)
.set('response_type', 'code')
.set('redirect_uri', this.redirectUri)
console.log(params);
return this.http.get(this.apiGeneric, { headers: this.httpHeaders, params: params, observe: 'response' });
}
719
Ok, I found the issue. This is probably something I misunderstood about generating headers, but what I found was that:
this.httpHeaders = new HttpHeaders();
this.httpHeaders.set('Accept', 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8');
(Both .set and .add) would result in an empty httpHeaders
Changing how I constructed the header solved my problem and the content-type header (And some customer test headers) were inserted and I got the full HTML output. If anyone else runs into this, here is the proper way to construct a header that solved my problem:
const headers = new HttpHeaders({
Accept:'text/html',
XFF:'testing123'
});
const params = new HttpParams()
.set('client_id', this.clientId)
.set('response_type', 'code')
.set('redirect_uri', this.redirectUri)
console.log(headers.get('Accept'));
console.log(this.apiGeneric);
return this.http.get(this.apiGeneric, { headers: headers, params:params });
}
Evidence:XFF Header and Accept headers are what we set them to:
Accept:text/html
Accept-Encoding:gzip, deflate, br
Accept-Language:en-US,en;q=0.9
Connection:keep-alive
Host:accounts.spotify.com
Origin:http://evil.com/
Referer:http://192.168.1.29:4200/
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36
XFF:testing123
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With