Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Download files based on input list Python

I'm trying to download files based on given list. My script does it well for me. However, I have one issue. It downloads just the first one. It doesn't iterate over the list. I've modified the code a little and now it's considering all the elements in the list to be one element and returns an error saying 'No such file or directory data\item1\nitem2\nitem3\nitem4\nitem5.txt' Here's a part of my code that parses the input list.

def get_data(name):
    file_name = os.path.join("data", name + ".txt")
    if not os.path.exists(file_name):
        sys.exit(-1)

    inF = open(file_name, "r") 
    lines = inF.readlines()
    data = ''.join(lines)

    return data 

EDIT:

def download_final_data_for_data(data):
    url = "http://www.example.com/"+ data
    url_file = urlopen(url)
    soup = BeautifulSoup(url_file)
    soup1 = str(soup)
    pattern=re.compile(r'''>final_data(.*?)</a>''')  
    data = pattern.findall(soup1)
    final_data_number = ''.join(data)
    return final_data_number

def get_data(name):
    data_list = []
    file_name = os.path.join("data", name + ".txt")
    if not os.path.exists(file_name):
        sys.exit(-1)

    inF = open(file_name, "r") 
    lines = inF.readlines()
    for line in lines:
        data = line.strip()
        if len(data) > 1:
            data_list.append(data)
        else:
            data_list.append(sys.argv[1])
    return data_list
like image 778
abn Avatar asked Jun 13 '26 06:06

abn


1 Answers

I see where the problem is. The problematic part is:

file_name = os.path.join("data", name + ".txt")

in order to get the correct file-name you would have to iterate over the names somehow. In order to get a list of your names as you read them (and as they are present in the code right now) do a

namelist = name.split("/n") #this gives you a list that you can work on.
                            #alternatively read the file line-by-line (which you don't at the moment)

what your code does is joining the strings containing "data" , all the names you read with the newline characters and a ".txt" suffix. Anyway, then just do a

for name in namelist:
    #do stuff with name
    file_name = os.path.join("data",name+".txt")
    ....
like image 51
user1941126 Avatar answered Jun 14 '26 18:06

user1941126