I wanna crawl 200 results for about 2000 queries but it gives me a "daily limit exceeded" error.
I want to confirm how many results can we crawl per day. Is there any solution that can solve this problem? Or the only way is to crawl a small part of queries each day...?
My code to crawl google is as follow:
def crawl(query_list):
http = httplib2.Http()
# Construct the service object for the interacting with the CustomSearch API.
service = discovery.build('customsearch', 'v1', developerKey='my api key', http=http)
res_list = []
for query in query_list:
json_res = service.cse().list(q = query, cx = 'my search engine id', num = 200,).execute()
res_list.append(json_res)
Thank you!
According to this post:
The first 100 queries per day are free. Any more, then you have to pay $5 per 1000 queries, for up to 10,000 queries per day, just enable billing to do so. Each query returns a maximum of 10 results, so you can retrieve 1000 URL’s from your search per day for free.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With