Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to concatenate two columns containing list (series) in Pandas

I'd like to concatenate two columns in pandas. Each column consists of a list of floating points of 1x4 elements. I'd like to merge two columns such that the output is a vector of 1x8. The below shows a snippet of the dataframe

ue,bs
"[1.27932459e-01 7.83234197e-02 3.24789420e-02 4.34971932e-01]","[2.97806183e-01 2.32453145e-01 3.10236304e-01 1.69975788e-02]"
"[0.05627587 0.4113416  0.02160842 0.20420576]","[1.64862491e-01 1.35556330e-01 2.59050065e-02 1.42498115e-02]"

To concatenate two columns, I do the following:

df['ue_bs'] = zip(df_join['ue'], df_join['bs'])

With this, I get a new column 'ue_bs' which contains the following for the first row of df['ue_bs']:

(array([1.27932459e-01, 7.83234197e-02, 3.24789420e-02, 4.34971932e-01]),
 array([2.97806183e-01, 2.32453145e-01, 3.10236304e-01, 1.69975788e-02]))

However, they are still two arrays. In order to merge them, I did it as follows:

a = df['ue_bs'][0]
np.concatenate((a[0], a[1]), axis=0)

Then, I got

array([1.27932459e-01, 7.83234197e-02, 3.24789420e-02, 4.34971932e-01,
   2.97806183e-01, 2.32453145e-01, 3.10236304e-01, 1.69975788e-02])

I am wondering is there a neat way of doing this in single line of code, instead of having to loop through df['ue_bs'] and perform np.concatenate()?

like image 639
twfx Avatar asked Aug 16 '18 06:08

twfx


People also ask

How do I merge two columns in pandas list?

Using Series.str.cat() Function to Combine Two Columns of Text. By using series.str.cat() function you can combine two Series by a separator. You can apply this with DataFrame as below. Here df["courses"] & df["Duration"] returns series.

How do I concatenate a series and DataFrame in pandas?

Use pandas. concat() to merge two Series Call pandas. concat(objs, axis=1) with objs as a sequence of two Series to create a DataFrame with objs as columns.

How do I concatenate certain columns in pandas?

Concatenating string columns in small datasets For relatively small datasets (up to 100–150 rows) you can use pandas.Series.str.cat() method that is used to concatenate strings in the Series using the specified separator (by default the separator is set to '' ).


1 Answers

To concatinate two lists in python, the easiest way is to use +. The same is true when concating columns in pandas. You can simply do:

df['ue_bs'] = df['ue'] + df['bs']

If the column type is numpy arrays you can first convert them into normal python lists before the concatination:

df['ue_bs'] = df['ue'].apply(lambda x: x.tolist()) + df['bs'].apply(lambda x: x.tolist())
like image 114
Shaido Avatar answered Oct 13 '22 00:10

Shaido