Prepending instead of appending NaNs in pandas using from_dict

Question

I have a pandas dataframe that I am reading from a defaultdict in Python, but some of the columns have different lengths. Here is what the data might look like:

Date      col1    col2    col3    col4    col5
01-01-15  5       12      1      -15      10
01-02-15  7       0       9       11      7
01-03-15          6       1       2       18
01-04-15          9       8       10
01-05-15         -4               7
01-06-15         -11             -1
01-07-15          6               

And I am able to pad the blanks with NaNs like so:

pd.DataFrame.from_dict(pred_dict, orient='index').T

Which gives:

Date      col1    col2    col3    col4    col5
01-01-15  5       12      1      -15      10
01-02-15  7       0       9       11      7
01-03-15  NaN     6       1       2       18
01-04-15  NaN     9       8       10      NaN
01-05-15  NaN    -4       NaN     7       NaN
01-06-15  NaN    -11      NaN    -1       NaN
01-07-15  NaN     6       NaN     NaN     NaN

However, what I'm really looking for is a way to prepend NaNs instead of appending them to the end, so that the data looks like this:

Date      col1    col2    col3    col4    col5
01-01-15  NaN     12      NaN     NaN     NaN
01-02-15  NaN     0       NaN    -15      NaN
01-03-15  NaN     6       NaN     11      NaN
01-04-15  NaN     9       1       2       NaN
01-05-15  NaN    -4       9       10      10
01-06-15  5      -11      1       7       7
01-07-15  7       6       8      -1       18

Is there an easy way to do this?

You can recreate the dictionary with this code:

import pandas as pd
from collections import defaultdict

d = defaultdict(list)
d["Date"].extend([
    "01-01-15", 
    "01-02-15", 
    "01-03-15", 
    "01-04-15", 
    "01-05-15",
    "01-06-15",
    "01-07-15"
])
d["col1"].extend([5, 7])
d["col2"].extend([12, 0, 6, 9, -4, -11, 6])
d["col3"].extend([1, 9, 1, 8])
d["col4"].extend([-15, 11, 2, 10, 7, -1])
d["col5"].extend([10, 7, 18])

Answer 1

You can use Series.shift to induce the Series/DataFrame. Unfortunately you can't pass in array of periods - you must shift each column by an integer value.

s = df.isnull().sum()
for col, periods in s.iteritems():
    df[col] = df[col].shift(periods)

Answer 2

A little modification on the itertools solution to your earlier question:

pd.DataFrame(list(itertools.zip_longest(*[reversed(i) for i in d.values()]))[::-1], columns=d.keys()).sort_index(axis=1)
Out[143]: 
       Date  col1  col2  col3  col4  col5
0  01-01-15   NaN    12   NaN   NaN   NaN
1  01-02-15   NaN     0   NaN -15.0   NaN
2  01-03-15   NaN     6   NaN  11.0   NaN
3  01-04-15   NaN     9   1.0   2.0   NaN
4  01-05-15   NaN    -4   9.0  10.0  10.0
5  01-06-15   5.0   -11   1.0   7.0   7.0
6  01-07-15   7.0     6   8.0  -1.0  18.0