Pandas: split list in column into multiple rows [duplicate]

Question

I have a question regarding splitting a list in a dataframe column into multiple rows.

Let's say I have this dataframe:

  Job position   Job type  id
0          [6]        [1]   3
1       [2, 6]  [3, 6, 5]   4
2          [1]        [9]  43

I would like every single combination of numbers, so the final result would be:

   id    Job position  Job type
0   3         6.0       1.0
1   4         2.0       3.0
2   4         2.0       6.0
3   4         2.0       5.0
4   4         6.0       3.0
5   4         6.0       6.0
6   4         6.0       5.0
7  43         1.0       9.0

Because right now I get this result:

   id    Job position  Job type
0   3         6.0       1.0
1   4         2.0       3.0
2   4         6.0       6.0
3   4         NaN       5.0
4  43         1.0       9.0

In order to get the result above, I did:

df = df.set_index(['id'])
(df.apply(lambda x: pd.DataFrame(x.tolist(),index=x.index)
                        .stack()
                        .rename(x.name)).reset_index())

Answer 1

Use a comprehension

pd.DataFrame([
    [p, t, i] for P, T, i in df.values
    for p in P for t in T
], columns=df.columns)

   Job position  Job type  id
0             6         1   3
1             2         3   4
2             2         6   4
3             2         5   4
4             6         3   4
5             6         6   4
6             6         5   4
7             1         9  43

---

Alternatives to iterating over values

pd.DataFrame([
    [p, t, i] for P, T, i in df.itertuples(index=False)
    for p in P for t in T
], columns=df.columns)

---

z = zip(df['Job position'], df['Job type'], df['id'])
pd.DataFrame([
    [p, t, i] for P, T, i in z
    for p in P for t in T
], columns=df.columns)

---

To generalize this solution to accommodate any number of columns

pd.DataFrame([
    [p, t] + a for P, T, *a in df.values
    for p in P for t in T
], columns=df.columns)

   Job position  Job type  id
0             6         1   3
1             2         3   4
2             2         6   4
3             2         5   4
4             6         3   4
5             6         6   4
6             6         5   4
7             1         9  43