[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[IDV #ILU-880840]: Help! The old NAM data have also disappeared from the Unidata LEAD server



Hi Dave,

re: our cutting back on the number of days of "rolling archive" on the LEAD 
cluster

> Well, sigh...what a pain.

Yes, and I apologize for you getting caught by our reconfiguration of the
number of days of data being kept.  I want to assure you that our decision
was not made lightly -- we were at 90% capacity before starting to our
cutback, and with the dual polarization NEXRAD Level II data coming online
in the near future, we were forced to make a move.  We all breathed a
sigh of relief when we heard that the GFS 0.25 degree data would not become
available before November, 2011 since it gives us some time to plan for
infrastructure changes that will likely be necessary.

re:
> OK, so I went over to NCDC NOMADS and found a directory full of
> individual 0.5 degree GFS grib files for April 01 at 1200Z.  The link is:
> http://nomads.ncdc.noaa.gov/thredds/catalog/gfs4/201004/20100401/catalog.html

Yup, I use NOMADS routinely.

re:
> The problem is those are individual files, one per forecast time (a
> separate file for 00hr, +03hr, +06hr, +09hr, etc.)  Do you know of a
> place on NOMADS I can get an aggregated grib file containing all the
> forecast files for one forecast cycle?

No, I don't know of an aggregation service in NOMADS.  My opinion,
however, is that you do not need it to accomplish what you want.

re:
> Or, do you know how I can aggregate the separate files using the
> "Aggregate Grids by Time" Data Source Type in the IDV Data Chooser?

Yes, absolutely.  I did much the same thing you are attempting for
January-June 2009 0.5 GFS data for the UCAR Africa Initiative
Meningitis Project.  I put all of the individual files into a
single directory and then told the IDV to aggregate them by time.
This step kicks off the creation of a THREDDS catalog in which
the files/times are represented.

Another approach would be to simply concatenate all of the files
into a single file and then point the IDV at the aggregated file.
Just so you know, the THREDDS processing on the LEAD cluster and
on motherlode.ucar.edu is doing this exact same thing:  concatenating
individual forecast fields in GRIB/GRIB2 and then creating a
catalog for the resulting file.

re: 
> Thanks again!

No worries.

re:
> Well, I'm not sure if my needs are similar to what other people want,
> but here's my recommendation:
> 
> It would be great to have at least one year's worth of something like
> the following:

We do not have the resources to keep a year's worth of all of the data
online.  This is becoming more and more true as the existing datasets
get larger and larger (with no end in sight).

re:
> GFS: global, as high of resolution as you can get (at worst 0.5 degree).
> Or, if you need to save storage: Global, 0.5 degree, and CONUS or
> North America at a higher resolution.
> 
> NAM: North America, again as high of resolution as you can get.  I like
> having the option of 12km, 20km, and 40km datasets.
> 
> RUC: CONUS, highest resolution available
> 
> I haven't done anything with ensemble data yet, but I suspect there's a
> need for these data as well.

I personally am more interested in high resolution satellite imagery
including GOES-East/West/South and Meteosat.  The volume of these data
is staggering!

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: ILU-880840
Department: Support Datastream
Priority: Normal
Status: Closed