Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Reading csv with date and time

Tags:

r

zoo

xts

I am working in R and reading csv which has date and time in its first column. I want to import this csv file in R first and then convert it to zoo obect.

I am using the code in R

EURUSD <- as.xts(read.zoo("myfile.csv",sep=",",tz="",header=T))

My csv file contain data in the format:

Date,Open,Low,High,Close
2006-01-02 10:01:00,2822.9,2825.45,2822.1,2824.9
2006-01-02 10:02:00,2825,2825.9,2824,2824.95
2006-01-02 10:03:00,2824.55,2826.45,2824,2826.45
2006-01-02 10:04:00,2826.45,2826.45,2824.9,2825.5
2006-01-02 10:05:00,2825.15,2825.5,2824,2824.85
2006-01-02 10:06:00,2824.7,2825.5,2823.7,2823.8
2006-01-02 10:07:00,2823.95,2824.45,2823.55,2824
2006-01-02 10:08:00,2824,2824.85,2823.5,2824.85
2006-01-02 10:09:00,2824.25,2825.45,2824,2825.45
2006-01-02 10:10:00,2825.2,2827,2825,2827

When I run the above command to import the data in to R I get the folowwwing error :

Error in as.POSIXlt.character(x, tz, ...) : 
  character string is not in a standard unambiguous format

I tried to find all the ways to sort out the issue. I read so many blogs over net but none of the method works for me.

I hope someone would help me.

like image 899
user395882 Avatar asked Feb 21 '12 04:02

user395882


People also ask

How do I separate a date and time in a CSV file?

Select Text to Columns and choose Space for the Separated By field. By default, the Tab option will be enabled for the Separated By field, so you'll need to uncheck that after choosing Space. Choose the Collection Time column and then select Date (MDY) from the Column type drop-down. Once you're done, click OK.

Is datetime a Dtype?

There is no datetime dtype to be set for read_csv as csv files can only contain strings, integers and floats. Setting a dtype to datetime will make pandas interpret the datetime as an object, meaning you will end up with a string.


1 Answers

Although this seems to be an old post, but I want to share my experience since I went through a similar very frustrating process trying to load time series csv data into R. The problem above is that excel changes the format of the date and time to the following %m/%d/%Y %H:%M, basically it drops the seconds. If you read a file with this format and you have a second resolution data you get multiple date time combinations that are similar. so you cannot simply use the format that ignores seconds because it gives the following error message . "character string is not in a standard unambiguous format"

The solution is to go back to excel and change the format of the date time column to be %m/%d/%Y %H:%M:%S. You can do that by choosing the closest date time default formats to the desired format (in this case it is %m/%d/%Y %H:%M and then manually add :ss at the end. Save the file as a csv file and then read it using the following command:

Data<-read.zoo("file.csv", tz="", header=TRUE,format='%m/%d/%Y %H:%M:%S')

This worked for me and I read a file that has about 900K rows.

like image 78
Ghaleb Abdulla Avatar answered Oct 14 '22 14:10

Ghaleb Abdulla