C# NetCDF Library

Question

I am currently working on (or at least planning) a couple of projects that work with large amounts of repetitive data. The kind of data that works well in a spreadsheet or database, but is nasty in XML. :)

NetCDF seems like a great option for a file format. However, my work is in C# and there is no "official" NetCDF implementation available. There is an "official" Python version that I could potentially use with IronPython. Another option is the "NetCDF Library for .Net" on CodePlex but it seems pretty quiet (http://www.codeplex.com/netcdf).

Has anyone worked with either of these libraries? Can you provide any recommendations?

Answer 1

I'm adding this now because this was the top answer when I Googled about this topic.

---

ETA Per the reply below, there is ANOTHER Microsoft NetCDF library now available:

https://www.nuget.org/packages/SDSLite

Scientific DataSet Lite 1.4.0

This is a cross platform library for manipulating netCDF, CSV and TSV files.

---

Since this question was originally asked and answered, Microsoft has released a Scientific DataSet Library that has support for NetCDF

http://research.microsoft.com/en-us/downloads/ccf905f6-34c6-4845-892e-a5715a508fa3/

Project Description

The SDS library makes it easy for .Net developers to read, write and share scalars, vectors, matrices and multidimensional grids which are very common in scientific modelling. It supports CSV, NetCDF and other file format

Programs that use the library store related data and associated metadata in a compact self-describing package. Libraries come with a set of utilities and packages: sds command line utility, DataSet Viewer application and an add-in for Microsoft Excel 2007 (and later versions). See Release page for details.

Answer 2

First, are you sure that NetCDF is the right choice? If you want to interact with other programs that need to read in large amounts of data and they already support NetCDF, then it's probably a great choice. There aren't that many standard and well-supported file formats that support large multidimensional arrays. But if you're only going to be reading and writing files from C#, it may not be such a good choice.

I'm a big fan of the "classic" NetCDF file format. It's compact and extremely simple, but flexible enough to support lots of common kinds of multidimensional well-structured data. It only took me one day to write a complete parser for classic NetCDF, and it only took an hour to write a program to output a well-formed special case of a classic NetCDF file. You could implement a pure C# NetCDF library yourself and it wouldn't be much trouble. You could easily start by implementing only the features you need. Here's the specification.

Unfortunately, NetCDF-4 chose to use HDF-5 as its data format. It adds a lot of complexity and makes it much more difficult to write a complete NetCDF parser in another language. HDF-5 is very general-purpose and in my opinion, it was overengineered - it tries to be too many things to too many people. I would not recommend trying to work with it directly unless you plan to spend a month writing unit tests. If you must use netCDF-4 / HDF-5 from C#, your only realistic option would be to wrap the C library using SWIG or something like that.

Note that NetCDF for Python is just a wrapper around the C code, so it's not really all that helpful; if you're going to use a wrapped C library you may as well just write a C# wrapper rather than use Python as a middle layer.