I've got a Python script which downloads data in json format through HTTP. If I run the script through command-line using the requests module, the HTTP connection is successful and data is downloaded without any issues. But when I try to launch the script as a crontab job, the HTTP connection throws a timeout after a while. Could anyone please tell me what is going on here? I am currently downloading data via a bash script first and then running the Python script from within that bash. But this is nonsense! Thank you so much!
Using: 3.6.1 |Anaconda custom (64-bit)| (default, May 11 2017, 13:09:58) \n[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
P.S.: I haven't found any posts regarding this issue. If there is already an answer for this on some other post, then please accept my apologies.
This is an excerpt from my code. It times out when running requests.get(url):
try:
response = requests.get(url)
messages = response.json()["Messages"]
except requests.exceptions.Timeout:
logging.critical("TIMEOUT received when connecting to HTTP server.")
except requests.exceptions.ConnectionError:
logging.critical("CONNECTION ERROR received when connecting to HTTP server.")
You'll learn in a bit what this table refers to. You should use Cron any time you want to automate something, like an OS job or a Python script. Needless to say, but an automated Python script can do basically anything. On Linux and macOS, the Crontab consists of six fields.
Print a process tree by ps axjf and look over the "COMMAND" column for the script name respectively the "PID" column to get its PID. kill the corresponding process: kill placeholder_for_pid_of_your_script.
It will wait until the response arrives before the rest of your program will execute. If you want to be able to do other things, you will probably want to look at the asyncio or multiprocessing modules. Chad S. Chad S.
The crontab is a list of commands that you want to run on a regular schedule, and also the name of the command used to manage that list. cron is the system process that will automatically perform tasks for you according to a set schedule.
I just found the answer to my question. I've defined the proxy being used and then used it like this in my code:
HTTP_PROXY="http://your_proxy:proxy_port"
PROXY_DICT={"http":HTTP_PROXY}
response = requests.get(url, proxies=PROXY_DICT)
Reference:
Proxies with Python 'Requests' module
Thank you all for your comprehension. I guess I should have done a thorough search before posting. Sorry.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With