Saving a Python dictionary in external file?

Question

I'm working on a code that is essentially a super basic AI system (basically a simple Python version of Cleverbot).

As part of the code, I've got a starting dictionary with a couple keys that have lists as the values. As the file runs, the dictionary is modified - keys are created and items are added to the associated lists.

So what I want to do is have the dictionary saved as an external file in the same file folder, so that the program doesn't have to "re-learn" the data each time I start the file. So it will load it at the start of running the file, and at the end it will save the new dictionary in the external file. How can I do this?

Do I have to do this using JSON, and if so, how do I do it? Can I do it using the built-in json module, or do I need to download JSON? I tried to look up how to use it but couldn't really find any good explanations.

I have my main file saved in C:/Users/Alex/Dropbox/Coding/AI-Chat/AI-Chat.py

The phraselist is saved in C:/Users/Alex/Dropbox/Coding/AI-Chat/phraselist.py

I'm running Python 2.7 through Canopy.

When I run the code, this is the output:

In [1]: %run "C:\Users\Alex\Dropbox\Coding\AI-Chat.py"
  File "C:\Users\Alex\Dropbox\Coding\phraselist.py", line 2
    S'How are you?'
    ^
SyntaxError: invalid syntax

EDIT: I got it now. I had to specify the sys.path to import phrase frome phraselist.py

Here's the full code I have:

############################################
################ HELPER CODE ###############
############################################
import sys
import random
import json
sys.path = ['C:\\Users\\Alex\\Dropbox\\Coding\\AI-Chat'] #needed to specify path
from phraselist import phrase



def chooseResponse(prev,resp):
    '''Chooses a response from previously learned responses in phrase[resp]    
    resp: str
    returns str'''
    if len(phrase[resp])==0: #if no known responses, randomly choose new phrase
        key=random.choice(phrase.keys())
        keyPhrase=phrase[key]
        while len(keyPhrase)==0:
            key=random.choice(phrase.keys())
            keyPhrase=phrase[key]
        else:
            return random.choice(keyPhrase)
    else:
        return random.choice(phrase[resp])

def learnPhrase(prev, resp):
    '''prev is previous computer phrase, resp is human response
    learns that resp is good response to prev
    learns that resp is a possible computer phrase, with no known responses

    returns None
    '''
    #learn resp is good response to prev
    if prev not in phrase.keys():
        phrase[prev]=[]
        phrase[prev].append(resp)
    else:
        phrase[prev].append(resp) #repeat entries to weight good responses

    #learn resp is computer phrase
    if resp not in phrase.keys():
        phrase[resp]=[]

############################################
############## END HELPER CODE #############
############################################

def chat():
    '''runs a chat with Alan'''
    keys = phrase.keys()
    vals = phrase.values()

    print("My name is Alan.")
    print("I am an Artifical Intelligence Machine.")
    print("As realistic as my responses may seem, you are talking to a machine.")
    print("I learn from my conversations, so I get better every time.")
    print("Please forgive any incorrect punctuation, spelling, and grammar.")
    print("If you want to quit, please type 'QUIT' as your response.")
    resp = raw_input("Hello! ")

    prev = "Hello!"

    while resp != "QUIT":
        learnPhrase(prev,resp)
        prev = chooseResponse(prev,resp)
        resp = raw_input(prev+' ')
    else:
        with open('phraselist.py','w') as f:
            f.write('phrase = '+json.dumps(phrase))
        print("Goodbye!")

chat()

And phraselist.py looks like:

phrase = {
    'Hello!':['Hi!'],
    'How are you?':['Not too bad.'],
    'What is your name?':['Alex'],
}

Answer 1

You can use pickle module for that. This module have two methods,

1. Pickling(dump): Convert Python objects into string representation.
2. Unpickling(load): Retrieving original objects from stored string representstion.

https://docs.python.org/3.3/library/pickle.html code:

>>> import pickle
>>> l = [1,2,3,4]
>>> with open("test.txt", "wb") as fp:   #Pickling
...   pickle.dump(l, fp)
... 
>>> with open("test.txt", "rb") as fp:   # Unpickling
...   b = pickle.load(fp)
... 
>>> b
[1, 2, 3, 4]

---

Following is sample code for our problem:

1. Define phrase file name and use same file name during create/update phrase data and also during get phrase data.
2. Use exception handling during get phrase data i.e. check if file is present or not on disk by os.path.isfile(file_path) method.
3. As use dump and load pickle methods to set and get phrase.

code:

import os
import pickle
file_path = "/home/vivek/Desktop/stackoverflow/phrase.json"

def setPhrase():
    phrase = {
        'Hello!':['Hi!'],
        'How are you?':['Not too bad.'],
        'What is your name?':['Alex'],
    }
    with open(file_path, "wb") as fp:
        pickle.dump(phrase, fp)

    return 

def getPhrase(): 
    if os.path.isfile(file_path):
        with open(file_path, "rb") as fp: 
            phrase = pickle.load(fp)
    else:
        phrase = {}

    return phrase

if __name__=="__main__":
    setPhrase()

    #- Get values.
    phrase = getPhrase()
    print "phrase:", phrase

output:

vivek@vivek:~/Desktop/stackoverflow$ python 22.py
phrase: {'How are you?': ['Not too bad.'], 'What is your name?': ['Alex'], 'Hello!': ['Hi!']}

Answer 2

You can dump it in json (built into python so you don't need to install it)

import json 
json.dump(your_dictionary, open('file_name.json', 'wb'))

You can use pickle but the file will not be human readable. Pickling is useful when you need to store python (or custom) objects.