python csv to dictionary using csv or pandas module

Question

I am using Python's csv.DictReader to read in values from a CSV file to create a dictionary where keys are first row or headers in the CSV and other rows are values. It works perfectly as expected and I am able to get a dictionary, but I only want certain keys to be in the dictionary rather than all of the column values. What is the best way to do this? I tried using csv.reader but I don't think it has this functionality. Maybe this can be achieved using pandas?

Here is the code I was using with CSV module where Fieldnames was the keys that I wanted to retain in my dict. I realized it isn't used for what I described above.

import csv
with open(target_path+target_file) as csvfile:
    reader = csv.DictReader(csvfile,fieldnames=Fieldnames)
    for i in reader:
        print i

Answer 1

You can do this very simply using pandas.

import pandas as pd

# get only the columns you want from the csv file
df = pd.read_csv(target_path + target_file, usecols=['Column Name1', 'Column Name2'])
result = df.to_dict(orient='records')

Sources:

• pandas.read_csv
• pandas.DataFrame.to_dict

Answer 2

You can use the to_dict method to get a list of dicts:

import pandas as pd

df = pd.read_csv(target_path+target_file, names=Fieldnames)

records = df.to_dict(orient='records')

for row in records:
    print row

to_dict documentation:

In [67]: df.to_dict?
Signature: df.to_dict(orient='dict')
Docstring:
Convert DataFrame to dictionary.

Parameters
----------
orient : str {'dict', 'list', 'series', 'split', 'records', 'index'}
    Determines the type of the values of the dictionary.

    - dict (default) : dict like {column -> {index -> value}}
    - list : dict like {column -> [values]}
    - series : dict like {column -> Series(values)}
    - split : dict like
      {index -> [index], columns -> [columns], data -> [values]}
    - records : list like
      [{column -> value}, ... , {column -> value}]
    - index : dict like {index -> {column -> value}}

      .. versionadded:: 0.17.0

    Abbreviations are allowed. `s` indicates `series` and `sp`
    indicates `split`.

Returns
-------
result : dict like {column -> {index -> value}}
File:      /usr/local/lib/python2.7/dist-packages/pandas/core/frame.py
Type:      instancemethod