Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Checking if a list of files exists before proceeding?

I have some pandas code running for 9 different files each day. Currently, I have a scheduled task to run the code at a certain time but sometimes the files have not been uploaded to the SFTP by our client on time which means that the code will fail. I want to create a file checking script.

like image 654
ldacey Avatar asked Dec 24 '22 08:12

ldacey


1 Answers

import os, time

filelist = ['file1','file2','file3']

while True:
    list1 = []

    for file in filelist:
        list1.append(os.path.isfile(file))

    if all(list1):
        # All elements are True. Therefore all the files exist. Run %run commands
        break
    else:
        # At least one element is False. Therefore not all the files exist. Run FTP commands again
        time.sleep(600) # wait 10 minutes before checking again

all() checks if all the elements in the list are True. If at least one element is False it returns False.

like image 153
Farhan.K Avatar answered Feb 13 '23 04:02

Farhan.K