Tools we use
Currently, this cookbook features Python Packages for working with data supported by NSIDC DAAC. It also features some applications that are accessible through a web browser or as stand alone packages that need to be installed on your local machine.
The focus on Python not only reflects the expertise of NSIDC DAAC but also reflects the popularity of Python within the Earth and atmospheric science communities. However, we recognize that many of our users are more familiar with other programming langauges such as R and Matlab. We hope that we will be able to include these langauges as the Cookbook develops.
Using web or locally-installed applications is a good way to start to discover and learn about data. We often use these applications to explore datasets, find what data is available, and quickly visualize data. However, many investigations require large numbers of files to be accessed and processed. It is often more efficient to write scripts in Python or some other language to do this. In the cloud, scripts are often the only way to search for and access data. Scripts are also a way to make workflows reproducible, something that is difficult to do with a GUI application.
Below we mention various tools that perform different tasks. To help distinguish which tools can perform which tasks, look for the following symbols:
| Task | Symbol |
|---|---|
| Data access and discovery: |  |
| Data visualization: |  |
| Data reformatting and transformation: |  |
| Data analysis: |  |
| Computation scaling: |  |
Python Packages
There are many Python packages available for working with Earth science data. The packages we use in this Cookbook are an unashamedly opinionated selection; they are the tools we like to use. We also think that these tools are the easiest to use for the types of data mananaged by NSIDC DAAC. Most of the tools have been developed so that researchers do not have to worry about the low-level details of accessing and working with often complicated data used in Earth science. This reduces the amount of code you have to write and also reduces the number of mistakes you will inevitably make.
earthaccess
earthaccess — search for and access NASA Earth science data.
xarray
xarray — work with N-dimensional labeled data (e.g. (time, x, y, z)).
rioxarray
rioxarray — add geospatial awareness to xarray.
rasterio
rasterio — read, write, and process raster datasets.
pandas
pandas — handle tabular data (spreadsheets, databases, CSV).
geopandas
geopandas — extend pandas to work with geospatial vector data.
cartopy
cartopy — map projections, transformations, and plotting for geospatial data.
sliderule
sliderule — on-demand processing and access to ICESat-2 data.
icepyx
icepyx — search, subset, and download ICESat-2 data with built-in metadata handling.
satpy
satpy — read, composite, and visualize meteorological satellite data.
dask
dask — scale computations from laptops to clusters with parallel, out-of-core workflows.
Installing Python Packages
Most Python packages can be installed from the Python Package Index (PyPI) using pip. Some scientific and geospatial packages are easier to install with conda, particularly through the conda-forge community channel.
For more details, see the official Python documentation on installing packages, the pip documentation, and the conda user guide.
Applications
Applications provide graphical interfaces for exploring, visualizing, and working with Earth science data. Some run directly in a web browser, while others require installation on your local machine. They are useful for quickly examining datasets, generating plots or maps, learning about data structures before moving on to scripted workflows. Some are capapble of performing analysis and have built-in mechanisms for repeatable workflows.
Earthdata Search
Earthdata Search — NASA’s primary web application for searching, filtering, and downloading Earth science data.
Open Altimetry
Open Altimetry — a browser-based tool for visualizing and accessing ICESat and ICESat-2 altimetry data.
SlideRule
SlideRule — web client for on-demand processing and access to ICESat-2 data.
HDFView
HDFView — a desktop application for viewing and editing HDF4 and HDF5 files.
Panoply
Panoply — a NASA tool for visualizing netCDF, HDF, and GRIB files with quick map and plot options.
QGIS
QGIS — an open-source geographic information system for analyzing and visualizing spatial data.
ArcGIS
ArcGIS — a commercial GIS platform for advanced mapping, spatial analysis, and data management.