Why does my text file keep overwriting the data on it?

Question

I'm trying to pull some data from Facebook pages for a product and dump it all into a text file, but I find that the file keeps overwriting itself with the data. I'm not sure if it's a pagination issue or if I have to make several files.

Here's my code:

#Modules
import requests
import facebook
import json

def some_action(post):
    print posts['data']
    print post['created_time']

#Token
access_token = 'INSERT ACCESS TOKEN'
user = 'walkers'

#Posts
graph = facebook.GraphAPI(access_token)
profile = graph.get_object(user)
posts = graph.get_connections(profile['id'], 'posts')

#Write
while True:
    posts = requests.get(posts['paging']['next']).json()
    #print posts
        
    with open('test121.txt', 'w') as outfile:
        json.dump(posts, outfile)

Any idea as to why this is happening?

Answer 1

w overwrites, open with a to append or open the file once outside the loop:

append:

while True:
    posts = requests.get(posts['paging']['next']).json()
    #print posts
    with open('test121.txt', 'a') as outfile:
        json.dump(posts, outfile)

Open once outside the loop:

with open('test121.txt', 'w') as outfile:
    while True:
        posts = requests.get(posts['paging']['next']).json()
        #print posts
        json.dump(posts, outfile)

It makes more sense to use the second option, if you are going to be running the code multiple times then you can open with a outside the loop also, if the file does not exist it will be created, if it does data will be appended

Answer 2

This is because you are using the file operator with w mode, you are overwriting the content. You can use the a append mode:

It can be done like this

Modification:

with open('test121.txt', 'w') as outfile:
   while True:
       posts = requests.get(posts['paging']['next']).json() 
       json.dump(posts, outfile)

w overwrites on the existing file

i.e)

File1.txt:

123

code:

with open("File1.txt","w") as oup1:
    oup1.write("2")

File1.txt after python run:

2

Its value is overwritten

a appends to the existing file

i.e)

File1.txt:

123

code:

with open("File1.txt","a") as oup1:
    oup1.write("2")

File1.txt after python run:

1232

The written content is appended to the end.