I want to use different API keys for data scraping each time my program is run.
For instance, I have the following 2 keys:
apiKey1 = "123abc"
apiKey2 = "345def"
and the following URL:
myUrl = http://myurl.com/key=...
When the program is run, I would like myUrl
to be using apiKey1
. Once it is run again, I would then like it to use apiKey2
and so forth... i.e:
First Run:
url = "http://myurl.com/key=" + apiKey1
Second Run:
url = "http://myurl.com/key=" + apiKey2
Sorry if this doesn't make sense, but does anyone know a way to do this? I have no idea.
EDIT:
To avoid confusion, I've had a look at this answer. But this doesn't answer my query. My target is to cycle between the variables between executions of my script.
I would use a persistent dictionary (it's like a database but more lightweight). That way you can easily store the options and the one to visit next.
There's already a library in the standard library that provides such a persistent dictionary: shelve
:
import shelve
filename = 'target.shelve'
def get_next_target():
with shelve.open(filename) as db:
if not db:
# Not created yet, initialize it:
db['current'] = 0
db['options'] = ["123abc", "345def"]
# Get the current option
nxt = db['options'][db['current']]
db['current'] = (db['current'] + 1) % len(db['options']) # increment with wraparound
return nxt
And each call to get_next_target()
will return the next option - no matter if you call it several times in the same execution or once per execution.
The logic could be simplified if you never have more than 2 options:
db['current'] = 0 if db['current'] == 1 else 1
But I thought it might be worthwhile to have a way that can easily handle multiple options.
Here is an example of how you can do it with automatic file creation if no such file exists:
import os
if not os.path.exists('Checker.txt'):
'''here you check whether the file exists
if not this bit creates it
if file exists nothing happens'''
with open('Checker.txt', 'w') as f:
#so if the file doesn't exist this will create it
f.write('0')
myUrl = 'http://myurl.com/key='
apiKeys = ["123abc", "345def"]
with open('Checker.txt', 'r') as f:
data = int(f.read()) #read the contents of data and turn it into int
myUrl = myUrl + apiKeys[data] #call the apiKey via index
with open('Checker.txt', 'w') as f:
#rewriting the file and swapping values
if data == 1:
f.write('0')
else:
f.write('1')
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With