[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDFJava #SSN-631180]: Re: more grib issues



> On 05/31/2012 09:34 AM, Unidata netCDF Java Support wrote:
> >> I've been getting my feet wet with this stuff, and I'm finally coming
> >> back to this issue.  I've tried to throw away the 6-hour intervals using
> >> the method described in the documentation, but it doesn't seem to work
> >> for me.  Have you tested this on the RFC grib format? If so, could you
> >> possibly send a sample my way?  I'm really trying to tackle this mixed
> >> interval problem (by throwing away non 1-hour intervals) in a complete
> >> way, so any tips you have would be greatly appreciated.
> >>
> >> Also, we are currently downloading these grib files from the NPVU ftp
> >> server to build an updating archive.  Our worry however is that the NPVU
> >> losing funding will cause this server to be shut down.  Would it make
> >> sense for us to pursue getting hooked up to the IDD so we can have
> >> access to this (and possibly other datasets) in a more proper way?  We
> >> are not sure what the implications of hooking up to IDD streams is, but
> >> it seems like we should start considering this as an option.
> > motherlode catalogs are in github.
> > this is what i currently have for RFC:
> >
> >      <dataset name="NWS National Precipitation Verification Unit">
> >
> >        <metadata inherited="true">
> >          <dataFormat>GRIB-1</dataFormat>
> >          <documentation xlink:href="http://www.hpc.ncep.noaa.gov/npvu/";
> >                         xlink:title="National Precipitation Verification 
> > Unit Page"/>
> >          <documentation
> >              
> > xlink:href="http://origin.hpc.ncep.noaa.gov/npvu/confpres/rfcworkshop07/NPVUatRFC_Verification_Workshop_2007.ppt";
> >              xlink:title="NPVU Overview (ppt)"/>
> >          <creator>
> >            <name vocabulary="DIF">DOC/NOAA/NWS/NPVU</name>
> >            <contact url="http://www.ncep.noaa.gov/"; 
> > email="http://www.ncep.noaa.gov/mail_liaison.shtml"/>
> >          </creator>
> >        </metadata>
> >
> >        <featureCollection name="RFC" featureType="GRIB" harvest="true" 
> > path="grib/NPVU/RFC">
> >          <collection spec="/data/ldm/pub/native/grid/NPVU/RFC/**/.*grib1$"
> >                      dateFormatMark="yyyyMMdd#.grib1#"
> >                      olderThan="5 min"/>
> >          <tdm startup="true" rescan="0 0/15 * * * ? *" trigger="allow"/>
> >
> >          <gribConfig>
> >            <gdsName hash='-1960629519' groupName='KTUA:Arkansas-Red River 
> > RFC'/>
> >            <gdsName hash='-1819879011' groupName='KFWR:West Gulf RFC'/>
> >            <gdsName hash='-1571856555' groupName='KORN:Lower Mississippi 
> > RFC'/>
> >            <gdsName hash='-1491065322' groupName='KKRF:Missouri Basin RFC'/>
> >            <gdsName hash='-1017807718' groupName='TSJU:San Juan PR WFO'/>
> >            <gdsName hash='-1003775954' groupName='NCEP-QPE National 
> > Mosaic'/>
> >            <gdsName hash='-529497359' groupName='KRHA:Middle Atlantic RFC'/>
> >            <gdsName hash='289752153' groupName='KRSA:California-Nevada 
> > RFC-6hr'/>
> >            <gdsName hash='424971237' groupName='KRSA:California-Nevada 
> > RFC-1hr'/>
> >            <gdsName hash='511861653' groupName='KTIR:Ohio Basin RFC'/>
> >            <gdsName hash='880498701' groupName='KPTR:Northwest RFC'/>
> >            <gdsName hash='1123818409' groupName='KTAR:Northeast RFC'/>
> >            <gdsName hash='1174418106' groupName='KNES-National Satellite 
> > Analysis'/>
> >            <gdsName hash='1193085709' groupName='KMSR:North Central RFC'/>
> >            <gdsName hash='1464276934' groupName='KSTR:Colorado Basin RFC'/>
> >            <gdsName hash='1815048381' groupName='KALR:Southeast RFC'/>
> >          </gribConfig>
> >        </featureCollection>
> >
> >      </dataset>
> >
> > not currently throwing away the 6-hour intervals. which has that problem?
> I've pretty much copied your configuration on our side after I found it
> on github.  I'm not using the TDM (need to read up on that a bit more).
> What I want is for a center like the northeast, where there is only the
> mixed interval variable, to throw away the one hour stuff in order to
> have it look like consistent hourly data.  Where this is really useful
> for me is writing a long archive that should be consistent across time
> (same variable name, same time intervals).
> 
> So I'm using this ncml:
> 
> <?xml version="1.0" encoding="UTF-8"?>
> <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";
> location="QPE.20110101.009.158">
> 
> <iospParam>
> <intvFilter intvLength="1">
> <variable id="9-158-2"/>
> </intvFilter>
> </iospParam>
> </netcdf>

1) its possible i dont have this working for GRIB1, let me check.

2)
look at:

http://www.unidata.ucar.edu/software/netcdf-java/formats/GribFiles.html#intvFilter

<iospParam>
   <intvFilter intvLength="3">
     <variable id="0-1-8"/>
     <variable id="0-1-10"/>
   </intvFilter>
 </iospParam>

Only include the 3 hour intervals for parameters 0-1-8 and 0-1-10, defining the 
parameter using discipline-category-number notation.

so

> <intvFilter intvLength="1">
> <variable id="9-158-2"/>
> </intvFilter>

would include, not exclude, length 1 intervals

3) i dont see a parameter 9-158-2 in ZETA_KTAR.*grib1$, send me a sample file

4) you probably have to delete the *.ncx files to make this work, once you 
change the filter.

> 
> I am interested in your opinion of connecting directly to the IDD, what
> would be involved, and whether it makes sense.

probably does, do you just want the RFC data?



Ticket Details
===================
Ticket ID: SSN-631180
Department: Support netCDF Java
Priority: High
Status: Open


NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.