I've exported a request from my Chrome browser using "Copy as PowerShell", trimmed it to a minimum and the script runs successfully:
Invoke-WebRequest -Uri "https://www.sec.gov/data-research/sec-markets-data/financial-statement-data-sets" `
-Headers @{
"Accept"="application/signed-exchange"
"Accept-Encoding"="gzip"
"sec-fetch-site"="none"
}
Now, I want to reproduce this behaviour in C#:
using var client = new HttpClient();
using var request = new HttpRequestMessage(HttpMethod.Get, "https://www.sec.gov/data-research/sec-markets-data/financial-statement-data-sets");
request.Headers.Add("Accept", "application/signed-exchange");
request.Headers.Add("Accept-Encoding", "gzip");
request.Headers.Add("sec-fetch-site", "none");
var response = await client.SendAsync(request);
But here I'm getting a 403 forbidden error.
What is the difference between the 2 ?
Is one of them doing something hidden ? Implements HTTP differently ? This is driving me nuts
Found it right after posting, PowerShell adds a hidden User-Agent to the request with a value of:
Mozilla/5.0 (Windows NT; Windows NT 10.0; en-GB) WindowsPowerShell/5.1.19041.5607
Adding it to the C# code fixes the issue. I was sure I tried that before posting, but I guess I made a mistake. Hopefully this'll help people
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With