Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Convert to date MongoDB via mongoimport

I have downloaded huge chunks of data in the format in csv. I am using mongoimport to enter the data into MongoDB for processing. How do I get the date into date format recognized by MongoDB?

sample data with header

Date, Open Price, High Price, Low Price, Last Traded Price , Close Price, Total Traded Quantity, Turnover (in Lakhs)
04-Apr-2014,901,912,889.5,896.75,892.85,207149,1867.08
03-Apr-2014,908,918,897.65,900,900.75,156260,1419.9
02-Apr-2014,916,921.85,898,900.7,900.75,175990,1591.97
like image 610
Ankit Haldar Avatar asked Apr 06 '14 04:04

Ankit Haldar


People also ask

How do I import a CSV file into MongoDB?

You can use the mongoimport command to import CSV files into a collection using the headerline option. Headerline option notifies the mongoimport command of the first line; to be not imported as a document since it contains field names instead of data.

What is ISODate in MongoDB?

ISODate() is a helper function that's built into to MongoDB and wraps the native JavaScript Date object. When you use the ISODate() constructor from the Mongo shell, it actually returns a JavaScript Date object.


2 Answers

As of Mongo version 3.4, you can use --columnsHaveTypes option to specify the type of your field while using mongoimport to import your data. here is the link for reference.

Sample mongoimport syntax below:

mongoimport --db XYZ --collection abc --type tsv --fields id.int32(),client_name.string(),app_name.auto(),date.date() --columnsHaveTypes --file "abc.tsv" --verbose
like image 45
Hatim Stovewala Avatar answered Sep 20 '22 02:09

Hatim Stovewala


As far as I know, there is no way to do this with mongoimport.

But this is achievable by importing the data and then running the following script (note that there is no point of all this hastle with a monthes as in Neil's Lunn script, because mongo can properly convert your date by doing this new Date('04-Apr-2014')):

db.collName.find().forEach(function(el){
    el.dateField = new Date(el.dateField);
    db.collName.save(el)
});

PS If timezone is so important (I assume that it is not, if there are only dates without time information), you can just change timezone on your local machine and then run the query. (Thanks to Neil Lunn for clarification regarding this)

like image 123
Salvador Dali Avatar answered Sep 21 '22 02:09

Salvador Dali