I'm trying to download files based on given list. My script does it well for me. However, I have one issue. It downloads just the first one. It doesn't iterate over the list. I've modified the code a little and now it's considering all the elements in the list to be one element and returns an error saying 'No such file or directory data\item1\nitem2\nitem3\nitem4\nitem5.txt' Here's a part of my code that parses the input list.
def get_data(name):
file_name = os.path.join("data", name + ".txt")
if not os.path.exists(file_name):
sys.exit(-1)
inF = open(file_name, "r")
lines = inF.readlines()
data = ''.join(lines)
return data
EDIT:
def download_final_data_for_data(data):
url = "http://www.example.com/"+ data
url_file = urlopen(url)
soup = BeautifulSoup(url_file)
soup1 = str(soup)
pattern=re.compile(r'''>final_data(.*?)</a>''')
data = pattern.findall(soup1)
final_data_number = ''.join(data)
return final_data_number
def get_data(name):
data_list = []
file_name = os.path.join("data", name + ".txt")
if not os.path.exists(file_name):
sys.exit(-1)
inF = open(file_name, "r")
lines = inF.readlines()
for line in lines:
data = line.strip()
if len(data) > 1:
data_list.append(data)
else:
data_list.append(sys.argv[1])
return data_list
I see where the problem is. The problematic part is:
file_name = os.path.join("data", name + ".txt")
in order to get the correct file-name you would have to iterate over the names somehow. In order to get a list of your names as you read them (and as they are present in the code right now) do a
namelist = name.split("/n") #this gives you a list that you can work on.
#alternatively read the file line-by-line (which you don't at the moment)
what your code does is joining the strings containing "data" , all the names you read with the newline characters and a ".txt" suffix. Anyway, then just do a
for name in namelist:
#do stuff with name
file_name = os.path.join("data",name+".txt")
....
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With