Processing output from Global Circulation Models 1

Installing CDO and netcdf

Global circulation models that iterate calculations on a global grid over many time steps produce potentially vast amounts of data. Output from climate models and large historical climatic data are often held in either GRIB (WMO) or netCDF (Unidata) format. These formats have been specifically designed for efficient, compact storage of huge volumes of information. Research groups that work on climate modelling and monitoring have developed their own specialized tools for handling data in these formats under Unix. A Linux laptop can carry out many powerful operations on these large data sets using the same tools that super computer users run under Unix.

One of the differences between Linux and Windows users is that Linux users naturally tend to have a naturally clearer idea about how information is being held on their computers. While the Ubuntu GUI can do almost anything the Windows GUI can do Linux users tend to use the command line at least occasionally and are more comfortable editing configuration files.

Thus when work arises that involves moving around and reformatting large amounts of data Linux is a natural platform.

The tools for working with GRIB and netCDF have not been designed with ”user friendly” GUIs. However they fit easily into a scripting framework. At the time of writing the tool that appears to have the best documentation, ease of to installation and general simplicity of use is CDO (Climate Data Operators). The program can be downloaded as source code from

http://www.mpimet.mpg.de/fileadmin/software/cdo/.

This software offers a very strait-forward, powerful command line interface that can be used for a large number of pre-processing and processing tasks. Because both grib and netcdf file formats are in common use it is important to compile the source with netcdf support.

In Ubuntu or Debian install netcdf-bin and the libraries either using Synaptic or by typing

sudo apt-get install netcdf-bin

sudo apt-get install netcdfg-dev

sudo apt-get install libnetcdf3

This installs the programs ncdump and ncgen which convert NetCDF files to ASCII and back, respectively. NetCDF (network Common Data Form) is an interface for scientific data access and a freely-distributed software library that provides an implementation of the interface. The netCDF library also defines a machine-independent format for representing scientific data. Together, the interface, library, and format support the creation, access, and sharing of scientific data.

 

Now cdo can be built with netcdf support. Check the location of the netcdf header with

 

locate netcdf.h

 

In Ubuntu the header is found in /usr/include. So compile the cdo source code and install it with the following option.

 

sudo ./configure –with-netcdf=/usr/include

sudo make

sudo make install

Now if you get output after typing “cdo” in the console the program should be correctly installed. Full documentation and a reference card cdo_refcard.pdf can be found in the doc directory.

Next install the ncdf package for R. This can apparently be installed under Windows for R2.2.+ and includes all that is needed to run. However Windows users would have to use Cygwin to compile and run cdo, thus their lives would be much more difficult. Note that under Linux you must install the netcdf binary and headers before installing the R package. More details can be found here.

In the next post on the topic I will show how to begin analysing output from a major GCM in R.

 

 

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s