[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[IDV #YIL-341951]: Re: 20060412: My good friend; grib to netCDF conversion question

Hi Arthur-

> Sorry about that.  Would it help if I subscribed?  How do I subscribe?

No need to be sorry, you were just answering Jack's note which
cc'd idvusers.  You can subscribe to idvusers through the web 
interface at:

http://www.unidata.ucar.edu/support/mailinglist/mailing-list-form.html#subscribe

> Not sure that Jack can cat something together (In the form of a GRIB
> file) since he starts with a GRIB file with multiple variables and times
> (he might break it up using head / tail).  It's actually GRIB1 rather
> than GRIB2, which could throw IDV off, but I don't know IDV's
> capabilities.  Is IDV using the GRIB2 library from NCEP?  If so, I don't
> think it would handle GRIB1 data, but I can't say that I've tried it.

If he has the entire model run in one file (multiple times and params), 
there is no need to concatenate the files together.  Right now, you have 
to have all GRIB messages in one file so if they weren't there, they
would have to be cat'ed together..  The IDV reads both GRIB1 and GRIB2 
files directly.

The IDV is written in Java and uses Unidata mods the the JGrib package.
This is included with the netCDF for Java library.  That library reads
in the GRIB messages and presents it as a netCDF file so effectively it
does the degrib process in memory.

> Jack did mention in his original email where he was getting the GRIB1
> data from:
> ftp://tgftp.nws.noaa.gov/SL.us008001/DF.gr1/DC.mos/DS.mavcns/RD.20060411/

Okay.  I'll look there if they have problems.
 
> The problem with the file from degrib's NetCDF perspective was that it
> had multiple variables.  Since the NDFD has one variable per file, I
> made that assumption when I created my export to NetCDF capability
> (poor judgment on my part).  With the nameStyle option he should be able
> to get around that and create 1 NetCDF file per variable+surface (with
> multiple valid times).

I understand.

> BTW: I looked at your site, and saw a number of interesting URLs
> (impressive), but I didn't see something labeled "ndfd" or "ndgd".  

The catalog I pointed to was for use in the IDV.  If you access the
HTML interface to the THREDDS Data Server (TDS) catalog (which the
IDV catalog points to) at:

http://motherlode.ucar.edu:8080/thredds/catalog.html

and look under the "NCEP Model Data" link, you'll see a link for
"NWS National Digital Forecast Database" which is where the files
are located (and downloadable).  These are files that are available
through Unidata Internet Data Distribution (IDD) system and come from
the NOAAPort feed (I think).

> I probably just missed it.  Congrats on adding GRIB support, although I
> suppose that may eventually put degrib out of business :-(.  Still one
> of degrib's purposes was to promote use of GRIB so it seems to be
> succeeding.

I agree. We don't yet handle all GRIB products (tiles, etc), but we support
most.  Also, since these are national grids, you can use the IDV to 
subset over a particular region before downloading the data.
 
> Arthur
> 
> >This thread is bouncing to the idvusers list because Jack's message
> >was too long (the attachment made it so) and Arthur is not subscribed
> >to idvusers.
> >
> >Instead of running degrib, can you cat all the files from one
> >timestep into a single file and open it directly into the IDV?
> >Version 1.3b1 (and 1.3b2 which will be out on Friday) support
> >direct reading of GRIB messages.
> >
> >If that doesn't work, can you put some sample files out for us
> >to test?  It might just be a matter of having the necessary GRIB tables.
> >
> >You can access the some NDFD GRIB2 data using the 1.3b1 release through
> >the catalog at:
> >
> >http://www.unidata.ucar.edu/georesources/idvcatalog.xml
> >
> >Just a thought.
> >
> >Don Murray
> >
> >
> >
> >>Looks like you're pushing the envelope again.  The -msg 0 option says
> >>convert all messages and dump them into the same NetCDF file.
> >>Unfortunately I designed the code to allow only one variable per NetCDF
> >>file.  This was intended to make my life easier, but evidently it is not
> >>as flexible as I should have made it.
> >>
> >>The result is that when it hits the second weather (around message 28)
> >>element it "can't find it in the file" and causes a fatal error, and the
> >>.nc file doesn't contain data.  Not sure.
> >>
> >>Various people have been sneaky, and have used degrib -I along with
> >>"head and tail' to split the file into submessages.  One could do that
> >>for messages 1-27 and then 28 - ?? , then feed it through degrib -C
> >>-NetCDF with a -msg 0 or "-msg all".
> >>
> >>Alternatively without the -out at the end, it would create a bunch of
> >>.nc files.  In fact if you got sneaky with the namestyle option, you
> >>could probably have the same effect as the cut option by telling it that
> >>TMPF_2-HTGL goes to one place, and MAXF_2-HTGL goes somewhere else.
> >>
> >>Yet another altenative would be to use some kind of NetCDF tool to merge
> >>the different files together.
> >>
> >>I kind of like the middle option, so try:
> >>degrib cy.06.bin -C -NetCDF 3 -msg all -nameStyle %e_%s.txt
> >>or
> >>degrib cy.06.bin -C -NetCDF 3 -msg all -nameStyle %e_%s.txt -nMet
> >>
> >>Arthur
> >>
> >>Jack Settelmaier wrote:
> >>
> >>
> >>
> >>>Below you'll find a screen capture of the error I get when running a
> >>>command line (yes, this dog can learn new tricks) to convert an MDL
> >>>GRIB dataset (MAV MOS guidance) to CF-1.0 netCDF.  (mav.nc attached)
> >>>
> >>>Running the output file through this CF checker, yields no errors, or
> >>>even warnings:
> >>>http://titania.badc.rl.ac.uk/cgi-bin/cf-checker.pl
> >>>
> >>>My pea-sized brain has me thinking it is not taking my "-msg 0" and is
> >>>only doing the first variable TMPF_2_HTGL and first projection rather
> >>>than ALL the grids?  My ncdump of what it creates appears only to have
> >>>one field.  The second screen capture shows I read the file in
> >>>Unidata's IDV, which can read CF-1.0 compliant netCDF, but it only has
> >>>one field, one time, and note the latitude and longitude at the bottom
> >>>saying NA?!  The file appears to be good enough to read, but it
> >>>doesn't display?!  Any ideas there.
> >>>
> >>>In ArcGIS 9.2 Beta 2, which I'm testing, I can drag and drop the
> >>>mav.nc file onto the desktop (one of two ways to view netCDF data),
> >>>but ArcGIS defaults to the longitude field, rather than the field I'd
> >>>like it to.  I'm guessing since longitude is listed before TMPF in the
> >>>netCDF, that might be why it defaults to it?  Can your code put the
> >>>data field earlier in the netCDF file as a test?
> >>>
> >>>Do I have something wrong in my command line, or, as I vaguely recall,
> >>>the ability to make a netCDF file with more than one variable is just
> >>>beyond reach in your code at this juncture?
> >>>
> >>>The input file I used is a renamed copy of the ones located here:
> >>>ftp://tgftp.nws.noaa.gov/SL.us008001/DF.gr1/DC.mos/DS.mavcns/RD.20060411/
> >>>
> >>>Any help would be appreciated, friend.  Thanks.
> >>>
> >>>I've also sent a version of this email to the ESRI ArcGIS 9.2 Beta
> >>>forum and IDV user's forum.


Ticket Details
===================
Ticket ID: YIL-341951
Department: Support IDV
Priority: Normal
Status: Open