Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

Download Daymet datasets from NASA OPeNDAP

This tutorial will demonstrate how to download Daymet: Daily Surface Weather Data on a 1-km Grid for North America, Version 4 for a bounding box. Daymet dataset is available for North America (NA), including Hawaii (HI) and Puerto Rico (PR).

This tutorial will demonstrate how to directly access and subset Daymet dataset using NASA OPeNDAP Hyrax. Hyrax allows direct access to selected variables for the dataset within the spatial-temporal bounds without having to download the whole data files.

Requirements

Access to NASA OPeNDAP requires NASA Earthdata authentication. We recommend authenticating your Earthdata Login (EDL) information using the earthaccess python library as follows:

Searching for Daymet Files

Daymet data files (or granules) are natively in netCDF4 format, and each file has one year’s worth of data. Data files are organized by variables (each for dayl, prcp, tmin, tmax, srad, swe, vp) and regions (each for US, PR, HI). For this tutorial, we will download data for the year 2024. We will first search for all the granules in the area of interest within the bounds of 170 W, 67.5N and 136.5W, 71.5N degrees.

We will now search for all the Daymet granules overlapping the above bounding box using the Earthaccess Python library. We will also limit the period of our search to the year 2024 (Jan 1 to Dec 31, 2024). The earthaccess’s search_data allows searching dataset granules using NASA’s CMR API. It requires the dataset digital object identifier (DOI), time, and the bounding box.

Total granules found: 7

Now, the day granule information, including data URLs, is stored in granules. The OPeNDAP URLs are in the RelatedUrls fields of the granule metadata. Let’s extract the OPeNDAP URLS.

['dap4://opendap.earthdata.nasa.gov/collections/C2532426483-ORNL_CLOUD/granules/Daymet_Daily_V4R1.daymet_v4_daily_na_tmin_2024.nc', 'dap4://opendap.earthdata.nasa.gov/collections/C2532426483-ORNL_CLOUD/granules/Daymet_Daily_V4R1.daymet_v4_daily_na_vp_2024.nc', 'dap4://opendap.earthdata.nasa.gov/collections/C2532426483-ORNL_CLOUD/granules/Daymet_Daily_V4R1.daymet_v4_daily_na_prcp_2024.nc']

Project the bounds to Daymet CRS

Daymet is natively in lambert_conformal_conic (LCC) coordinate reference system. We can now reproject the bounding coordinates of the above geopandas object (gdf) to bounds_lcc for requesting to OPeNDAP later.

array([-2823222.26105783, 3101471.12476104, -1401567.56219731, 4328450.66251338])

Retrieve Daymet subset with OPeNDAP

Access to NASA OPeNDAP requires NASA Earthdata authentication. Please set up a new account if you do not already have a NASA Earthdata account. We will use earthaccess module to set up the authentication and return a session instance with an authorized bearer token.

NOTE: it is recommended to use pydap’s create_session method, which is also a requests session, but it is configured to stream larger data arrays.

Efficiently communicate with the remote OPeNDAP server

Hint 1: When “opening” multiple remote files with Xarray, Xarray automatically downloads any coordinate data from each remote file. In our case, this means that both lat and lon — each a huge 2D array that is identical across all files — will unnecessarily be downloaded, resulting in a negative impact on our performance. To avoid this we need to drop these coordinates from all but a single remote file. Xarray has a useful method called drop_variables, but it is only usable after “opening” the dataset, i.e. after these have already been downloaded. Luckily, OPeNDAP and the DAP4 protocol can be exploited to drop the variables before downloading. This is done by manipulating the URL so that only the variable of interest is requested from the remote file.

Hint 2: When the remote file is huge, the subset can be large too. It helps to split the download into multiple chunks, download them in parallel, and reassemble them locally. This can be done with Xarray! But to achieve that we need to define a chunk parameter when “opening” the dataset. This chunk parameter should not be larger than our desired subset (otherwise we will download too much data). Below, we make it so that chunk matches the horizontal extent of our subset, but split the download along the time dimension into Nt chunks.

Loading...

Stream data in parallel

Should take only a couple of minutes.

Loading...
From the 645.4 Gbs of remote data, we only downloaded 17.8 Gbs
References
  1. Thornton, M. M., Shrestha, R., Wei, Y., Thornton, P. E., & Kao, S.-C. (2022). Daymet: Daily Surface Weather Data on a 1-km Grid for North America, Version 4 R1. ORNL Distributed Active Archive Center. 10.3334/ORNLDAAC/2129