Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to perform oauth when doing twitter scraping with python requests

I am trying to retrieve 100 recent tweets of a user. It is working well with tweepy module in Python. But how can I do the same with requests in python. I want to do:

import requests
r = requests.get('https://api.twitter.com/1.1/statuses/user_timeline.json?screen_name=xxxx&count=100')

Here how to do the authentication with client key, client secret, access token and access secret, before sending this request?

like image 685
sudheesh ks Avatar asked Oct 23 '15 17:10

sudheesh ks


People also ask

Can you scrape Twitter with Python?

Tweepy is a Python library for integrating with the Twitter API. Because Tweepy is connected with the Twitter API, you can perform complex queries in addition to scraping tweets. It enables you to take advantage of all of the Twitter API's capabilities.

Does Twitter have oauth2?

App settings You can also enable an App to access both OAuth 1.0a and OAuth 2.0. OAuth 2.0 can be used with the Twitter API v2 only. If you have selected OAuth 2.0 you will be able to see a Client ID in your App's Keys and Tokens section.

What is OAuth in Twitter API?

OAuth 1.0a allows an authorized Twitter developer App to access private account information or perform a Twitter action on behalf of a Twitter account.


1 Answers

You can use requests-oauthlib as described in requests docs.

OAuth:

import requests
from requests_oauthlib import OAuth1
url = 'https://api.twitter.com/1.1/account/verify_credentials.json'
auth = OAuth1(API_KEY, API_SECRET, ACCESS_TOKEN, ACCESS_TOKEN_SECRET)
requests.get(url, auth=auth)

Get tweets:

r = requests.get('https://api.twitter.com/1.1/statuses/user_timeline.json?screen_name=stackoverflow&count=100', auth=auth)
for tweet in r.json():
  print tweet['text']
like image 161
miles82 Avatar answered Oct 14 '22 06:10

miles82