Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Pandas select unique values from column

I was able to ingest a csv in jupyter notes by doing this :

csvData= pd.read_csv("logfile.csv")

My data looks like this:

event_timestamp ip  url 
2018-01-10 00:00 111.111.111.111 http://webpage1.com
2018-01-10 00:00 222.222.222.222 http://webpage2.com
...
..
.

I got a list of unique ips:

list_ips = csvData("[ip]")

What I'm trying to do is get a unique. Normally I would do:

list_ips.unique()

But in this case I get this error:

AttributeError: 'DataFrame' object has no attribute 'unique'

(I can use list_ips.head() and it will list a few IPs but it's not a unique list)

Thanks

EDIT My problem is I actually had:

list_ips = csvData([["ip"]]) 

So I removed 1 set of brackets so it became:

list_ips = csvData(["ip"]) 

Then I was able to follow Wen's example and do:

list_ips.unique().tolist()

Output:

['111.111.111.111','222.222.222.222'...]
like image 235
chowpay Avatar asked Jan 17 '18 02:01

chowpay


2 Answers

You need to select the column correctly then apply unique

csvData['ip'].unique().tolist()
Out[677]: ['111.111.111.111', '222.222.222.222']
like image 96
BENY Avatar answered Sep 21 '22 09:09

BENY


The reason why you are running into this problem is because pd.read_csv("logfile.csv").unique() is not a valid attribute from DataFrame. What I suggest you do is since csvData comes out as a list, you can search for all ip's by csvData['ip'] then search for unique ip's with csvData['ip'].unique().

like image 45
Julian Rachman Avatar answered Sep 20 '22 09:09

Julian Rachman