Programmatically Downloading CSV Files with Java

Question

Scenario: A website I use to research stock data has a link on the page to Export Data to Spreadsheet. The URL displayed when hovering over the export link is of the form http://www.stocksite.com/historical/export.php?symbol=C .

Question: Rather, that manually visiting the page for each stock I would like to automate the task. From Java, how can I programmatically call the site with a stock symbol and save the exported csv file? The URL and URLConnection class seem like the obvious place to start, but I'm unsure where to go from there.

Answer 1

All you need to do is to get the CSV in flavor of an InputStream.

InputStream input = new URL("http://example.com/file.csv").openStream();

Then you can feed it to any decent Java CSV parser API. Almost any of them take input in flavor of InputStream or Reader. If necessary, you can easily decorate InputStream as a Reader using InputStreamReader which you can then feed to the CSV parser.

Reader reader = new InputStreamReader(input, "UTF-8");