Issue with writing multiple lines into a file in python

Question

I want to download multiple specific links(images´ urls) into a txt file(or any file where all links can be listed underneath each others).

I get them but the code wrtite each link on the top of the other one and at the end it stays only a link :(. Also I want not repeated urls

def dlink(self, image_url):
        r = self.session.get(image_url, stream=True)
        with open('Output.txt','w') as f:
            f.write(image_url + '\n')

Answer 1

The issue is most simply that opening a file with mode 'w' truncates any existing file. You should change 'w' to 'a' instead. This will open an existing file for writing, but append instead of truncating.

More fundamentally, the problem may be that you are opening the file over and over in a loop. This is very inefficient. The only time the approach you use could be really useful is if your program is approaching the OS-imposed limit on number of open files. If this is not the case, I would recommended putting the loop inside the with block, keeping the mode as 'w' since you open the file just once now, and passing the open file to your dlink function.

Answer 2

Edit

Huge mistake of my part, as it is a method, and you will call it several times, if you open it in write mode ('w') or similar, it will Overwrites the existing file if the file exists. So, if you use the 'a' way, you can see that:

Opens a file for appending. The file pointer is at the end of the file if the file exists. That is, the file is in the append mode. If the file does not exist, it creates a new file for writing.

The other problem radics in image_url is a list, so you need to write it line by line:

def dlink(self, image_url):
        r = self.session.get(image_url, stream=True)
        with open('Output.txt','a') as f:
            for url in list(set(image_url)):
                f.write(image_url + '\n')

another way to do it:

your_file = open('Output.txt', 'a')
r = self.session.get(image_url, stream=True)
for url in list(set(image_url)):
  your_file.write("%s\n" % url)
your_file.close() #dont forget close it :)