Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

urllib3 HTTPResponse.read() returns empty bytes

Tags:

python

urllib3

I'm trying to read a website's content but I get an empty bytes object, b''.

import urllib3
from urllib3 import PoolManager
urllib3.disable_warnings()
https = PoolManager()

r = https.request('GET', 'https://minemen.club/leaderboards/practice/')

print(r.status)
print(r.read())

When I open the URL in a web browser I see the website, and r.status is 200 (success).

Why does r.read() not return the content?

like image 272
sef sf Avatar asked Sep 03 '26 23:09

sef sf


2 Answers

What makes you think it is wrong? Try the following, you'll have much more output:

print(r.data)

Check HTTPResponse to see how to use the r object you got.

like image 167
Keldorn Avatar answered Sep 06 '26 14:09

Keldorn


This is how urllib3.response.HTTPResponse.read is supposed to work.

It is explained for example here by one of the contributors to urllib3:

This is about documentation. You cannot use read() by default, because by default all the content is consumed into data. If you want read() to work, you need to set preload_content=True on the call to urlopen. Want to give that a try?

So you can simply use r.data.

like image 36
mkrieger1 Avatar answered Sep 06 '26 14:09

mkrieger1