So I have a 70mb .csv file that I wanna parse and convert into a json, trying to do the json convert in a 500kb test csv I found an easy solution with regex.
The problem was when I put my actual data, I can't use fs.readFileSync anymore, so I need to work with streams.
My problem is: how can I work with streams and regex? Supose that my stream cuts the buffer in the middle of a possible regex match, I think if that happens I will lose that data. Also, the data isn't structured so I don't realise other way to parse it than regexs.
Please let me know if I wasn't clear with my explanation, english isn't my main language but also I know the english community is the biggest also the fastest and more reliable.
Thanks in advance.
there is a stable readline core module
and you can do this
let lineReader = require('readline').createInterface({
input: require('fs').createReadStream('file.csv')
})
lineReader.on('line', (line) => {
// do regexs with line
})
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With