[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Motherlode clone.



Chris and Mike,

sorry for the delay in getting back to you about the pqacts, etc.

there are three thredd pqacts that i'll attach: pqact.thredds
pqact.threddsconduit  pqact.threddsradar that need to be place in ~ldm/etc
directory. also the etc/ldmd.conf file needs to have these three line
added to reference the pqacts.

exec    "pqact -f ANY-CONDUIT-NNEXRAD-CRAFT /opt/ldm/etc/pqact.thredds"
exec    "pqact -f CONDUIT|SPARE /opt/ldm/etc/pqact.threddsconduit"
exec    "pqact -f NNEXRAD|CRAFT /opt/ldm/etc/pqact.threddsradar"


i believe the only extra s/w needed is the netCDF decoders package, it's
available at: http://www.unidata.ucar.edu/software/decoders/

the binaries from the build go into the ~ldm/decoders directory and the
etc files from the build go into the ~ldm/etc directory.

if you want to bypass this step, comment out the pqact
entries that have the PIPE actions then you can come back later to install
them.


there is a new perl script used to scour the directory structure called
manageFilesDirs.pl and it goes into the ~ldm/util directory. i'll attach
the script too.  also there is a manageFilesDirs.conf file to configure it
that goes into the ~ldm/etc direction, i'll attach too.  the conf file
should be set correctly, so no mods will be needed unless you change the
directory structure in the pqacts. i suggest that you don't change them
because all the thredds configure file exepect that dir structure.

it is run once a day from cron, ie

#
# files/dirs managing script
30 21 * * *  util/manageFilesDirs.pl -f
/local/ldm/etc/manageFilesDirs.conf > /data/ldm/logs/manageFilesDirs.log
#


after this is setup, then we can start on the thredds configurations.

robb...


On Tue, 23 May 2006, Chris Herbster wrote:

> Rob,
>
> We seem to be ready on this end for a set of pqact entries to populate
> the catalog.  Please work with Mike to see if he needs any other parts
> besides these.  I think he said something about needing some XML pieces too.
>
> Thanks!
>
> CH
>
> Robb Kambic wrote:
>
> >Sounds good to me, let me know when help is needed?
> >
> >robb...
> >
> >
> >
> >
> >On Mon, 24 Apr 2006, Chris Herbster wrote:
> >
> >
> >
> >>Hi Robb,
> >>
> >>As always, another good time in Boulder!  (-;
> >>
> >>Off hand, I'd say, "Let's do everything that you do."  However, I'll
> >>defer on this to Mike Masscotte.  He has done a great job of getting
> >>things cleaned up from when I tried to run things and he might have some
> >>suggestions (or just say "Let's do it!")
> >>
> >>We have the LDM up and running.  We'll need to do the next piece to get
> >>ready....  I'll let Mike set the pace on this end.
> >>
> >>CH
> >>
> >>Robb Kambic wrote:
> >>
> >>
> >>
> >>>Hey Chris,
> >>>
> >>>It was really good to sit down and talk with you at usercomm diner. I
> >>>haven't had that much fun in awhile. Anyway, if you are still interested
> >>>in doing a motherlode clone we can start on it. Could you list the data
> >>>products that you are interested in receiving? or are you interested in
> >>>receiving all products just like motherlode?
> >>>
> >>>
> >>>Once the initial setup is done, then upgrades should be easy. The 2 main
> >>>external programs are the LDM and Jakarta-Tomcat. If you could get your
> >>>system admin to install these with users ldm and tomcat then stage would be
> >>>set. The tomcat version we are using is jakarta-tomcat-5.0.28, i know
> >>>there are newer ones released but this is the version that we have tested
> >>>the s/w.  At this point do you want to install the s/w, there are on-line
> >>>documentation or do you want me to install it and do the initial
> >>>configuration?
> >>>
> >>>The documentation is located at:
> >>>
> >>>http://motherlode.ucar.edu:8080/thredds/docs/
> >>>
> >>>
> >>>Take care,
> >>>
> >>>robb...
> >>>
> >>>
> >>>
> >>>===============================================================================
> >>>Robb Kambic                                   Unidata Program Center
> >>>Software Engineer III                         Univ. Corp for Atmospheric 
> >>>Research
> >>>address@hidden                WWW: http://www.unidata.ucar.edu/
> >>>===============================================================================
> >>>
> >>>
> >>>
> >>>
> >>>
> >>--
> >>
> >> Dr. Christopher G. Herbster
> >> Associate Professor
> >> Director of Science and Technology
> >> for the ERAU Weather Center
> >> Applied Aviation Sciences
> >> Embry-Riddle Aeronautical Univ.
> >> 600 S. Clyde Morris Blvd.
> >> Daytona Beach, FL 32114-3900
> >>
> >> 386.226.6444 Office
> >> 386.226.6446 Weather Center
> >> http://wx.erau.edu/
> >>
> >>
> >>
> >>
> >
> >===============================================================================
> >Robb Kambic                             Unidata Program Center
> >Software Engineer III                           Univ. Corp for Atmospheric 
> >Research
> >address@hidden                  WWW: http://www.unidata.ucar.edu/
> >===============================================================================
> >
> >
> >
>
> --
>
>  Dr. Christopher G. Herbster
>  Associate Professor
>  Director of Science and Technology
>  for the ERAU Weather Center
>  Applied Aviation Sciences
>  Embry-Riddle Aeronautical Univ.
>  600 S. Clyde Morris Blvd.
>  Daytona Beach, FL 32114-3900
>
>  386.226.6444 Office
>  386.226.6446 Weather Center
>  http://wx.erau.edu/
>
>
>

===============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
address@hidden             WWW: http://www.unidata.ucar.edu/
===============================================================================
#
# This pqact defines the directory structure expected by the THREDDS Data Server
# default configuration.  It includes the pqact entries using decoders and FILE
# actions to create the structure. If this pqact is implemented then the 
# THREDDS Data Server should work without doing any data structure 
# configurations. Default configuration starting at the ~ldm/data directory:
#
# pub
# pub/native
#
# data/pub/native/surface/metar/{day}/{station}
# data/pub/native/surface/synoptic/{day}/{station} commented out
# data/pub/native/surface/buoy/{day}/{station} commented out
# data/pub/native/surface/zone/{day}/{station} commented out
# data/pub/native/nwstg/text/{day}
# data/pub/native/upperair/{day}/{station} commented out
#
# pub/native/satellite/{band}/{product}_{resolution}/{day}
#
# pub/native/profiler/wind/06min/{day}
# pub/native/profiler/wind/01hr/{day}
# pub/native/profiler/RASS/06min/{day}
# pub/native/profiler/RASS/01hr/{day}
# pub/native/profiler/bufr/{day}  commented out
#
# pub/native/radar/10km_mosaic/{day}
# pub/native/radar/composite/{product}/{resolution}/{day}
# pub/native/radar/level2/{station}/{day}
# pub/native/radar/level3/{product}/{station}/{day}
#
# pub/native/grid/NCEP/DGEX/Alaska_12km
# pub/native/grid/NCEP/DGEX/CONUS_12km //Downscaled GFS with Eta Extension(DGEX)
# pub/native/grid/NCEP/GFS/Alaska_191km
# pub/native/grid/NCEP/GFS/CONUS_80km
# pub/native/grid/NCEP/GFS/CONUS_95km
# pub/native/grid/NCEP/GFS/CONUS_191km
# pub/native/grid/NCEP/GFS/Ensemble_1p25deg
# pub/native/grid/NCEP/GFS/Extended_5p0deg
# pub/native/grid/NCEP/GFS/Global_0p5deg
# pub/native/grid/NCEP/GFS/Global_onedeg
# pub/native/grid/NCEP/GFS/Global_2p5deg
# pub/native/grid/NCEP/GFS/Global_1p0deg_Ensemble
# pub/native/grid/NCEP/GFS/Global_2p5deg_Ensemble
# pub/native/grid/NCEP/GFS/Hawaii_160km
# pub/native/grid/NCEP/GFS/N_Hemisphere_381km
# pub/native/grid/NCEP/GFS/Puerto_Rico_191km
# pub/native/grid/NCEP/GFS/Spectral_1p25deg
# pub/native/grid/NCEP/NAM/Alaska_11km
# pub/native/grid/NCEP/NAM/Alaska_22km
# pub/native/grid/NCEP/NAM/Alaska_45km/conduit
# pub/native/grid/NCEP/NAM/Alaska_45km/noaaport
# pub/native/grid/NCEP/NAM/Alaska_95km
# pub/native/grid/NCEP/NAM/CONUS_12km
# pub/native/grid/NCEP/NAM/CONUS_20km/noaaport
# pub/native/grid/NCEP/NAM/CONUS_20km/surface
# pub/native/grid/NCEP/NAM/CONUS_20km/selectsurface
# pub/native/grid/NCEP/NAM/CONUS_40km/noaaport
# pub/native/grid/NCEP/NAM/CONUS_40km/conduit
# pub/native/grid/NCEP/NAM/CONUS_80km
# pub/native/grid/NCEP/NAM/DGEX/Alaska_12km
# pub/native/grid/NCEP/NAM/DGEX/CONUS_12km
# pub/native/grid/NCEP/NAM/Polar_90km
# pub/native/grid/NCEP/NDFD/CONUS_5km
# pub/native/grid/NCEP/OCEAN/Global_2x2deg
# pub/native/grid/NCEP/RUC/CONUS_40km
# pub/native/grid/NCEP/RUC/CONUS_80km
# pub/native/grid/NCEP/RUC2/CONUS_20km/hybrid
# pub/native/grid/NCEP/RUC2/CONUS_20km/pressure
# pub/native/grid/NCEP/RUC2/CONUS_20km/surface
# pub/native/grid/NCEP/SPECTRAL/Global_5x2p5deg
# pub/native/grid/ECMWF/model?
# pub/native/grid/UKMET/thin
# pub/native/grid/CMC/GEM                // Canadian grid
# pub/native/grid/FNMOC/NOGAPS             // FNMOC grid
# pub/native/grid/FNMOC/COAMPS             // FNMOC grid
# pub/native/grid/UNIDATA/WSETA
# pub/native/grid/{center}/WRF
#
# grids off of nwstg binary channel, entered above in correct model location
# pub/native/grid/NCEP/NAM/CONUS_12km
# pub/native/grid/NCEP/NAM/Alaska_11km
# pub/native/grid/NCEP/DGEX/CONUS_12km //Downscaled GFS with Eta Extension(DGEX)
# pub/native/grid/NCEP/DGEX/Alaska_12km
# pub/native/grid/NCEP/NDFD/CONUS_5km
# pub/native/grid/NCEP/GFS/CONUS_191km
#
# Decoded data
#
# pub/decoded/netcdf/surface/metar
# pub/decoded/netcdf/surface/synoptic
# pub/decoded/netcdf/surface/buoy
# pub/decoded/netcdf/upperair
# pub/decoded/netcdf/grid/{center}
# pub/decoded/netcdf/grid/NCEP/GFS/CONUS_80km
# pub/decoded/netcdf/grid/NCEP/GFS/Global_5x2p5deg
# pub/decoded/netcdf/grid/NCEP/GFS/Global_5x5deg
# pub/decoded/netcdf/grid/NCEP/NAM/CONUS_80km
# pub/decoded/netcdf/grid/NCEP/NGM/CONUS_80km
# pub/decoded/netcdf/grid/NCEP/OCEAN/Global_5x2p5deg
# pub/decoded/netcdf/grid/NCEP/RUC/CONUS_80km
# pub/decoded/netcdf/grid/NCEP/RUC2/CONUS_40km
# pub/decoded/netcdf/grid/NCEP/SST/Global_2x2deg
# pub/decoded/netcdf/grid/NCEP/SST/Global_5x2p5deg
#
#
# Restricted data
#
# ucar_only/native/ACARS/{day} 
#
#-------------------------------------------------------------------------------
#
# Surface test data
#
# All aviation reports including metar tests, broken/written to stn files
DDS|IDS ^S(A....|P....|XUS8.) .... ([0-3][0-9])
        PIPE    decoders/metarWriter
        data/pub/native/surface/metar
        (\2:yyyy)(\2:mm)
#
# Synoptic reports, broken/written to stn files
#DDS|IDS        ^SM.... .... ([0-3][0-9])
#       PIPE    decoders/synopticWriter
#       data/pub/native/surface/synoptic
#       (\1:yyyy)(\1:mm)
#
# buoy reports, broken/written to stn or WMO ID numbers files
#DDS|IDS        ^S[NS].... .... ([0-3][0-9])
#       PIPE    decoders/buoyWriter
#       data/pub/native/surface/buoy
#       (\1:yyyy)(\1:mm)
#
# zone reports, broken/written to zone files
#DDS|IDS        ^FOUS5. .... ([0-3][0-9])
#       PIPE    decoders/surface/zoneWriter
#       data/pub/native/zone
#       (\1:yyyy)(\1:mm)
#
# all of DDS/PPS/IDS | NOAAPORT textual data
DDPLUS|IDS      ^[^\-]
        FILE
        data/pub/native/nwstg/text/%Y%m%d/SURFACE_DDPLUS_%Y%m%d_%H00.txt
#
# upper air reports, broken/written to stn files
#DDS|IDS        ^U[ABDEFGHIJKLMPQRSTXZ].... .... ([0-3][0-9])
#       PIPE    decoders/upperairWriter
#       data/pub/native/upperair
#       (\1:yyyy)(\1:mm)
#
#
# NOAAPORT GINI Images
#
# PNG compressed NOAAPORT GINI Images
#NIMAGE ^sat/ch[0-9]/.*/(.*)/([12][0-9][0-9][0-9][01][0-9][0-3][0-9]) 
([0-2][0-9])([0-5][0-9])/(.*)/(.*km)/
#       FILE
#       data/pub/native/satellite/\1/\5_\6/\2/\5_\6_\1_\2_\3\4.gini
#
#       PIPE    -close
#       util/pngg2gini -vl logs/ldm-mcidas.log
#
# Zlib compressed NOAAPORT GOES-East/West GINI Images -- FILE
NIMAGE  ^satz/ch[0-9]/.*/(.*)/([12][0-9][0-9][0-9][01][0-9][0-3][0-9]) 
([0-2][0-9])([0-5][0-9])/(.*)/(.*km)/
        PIPE    -close
        util/ldmfile.sh 
data/pub/native/satellite/\1/\5_\6/\2/\5_\6_\1_\2_\3\4.gini
#
# FSL hourly summary and 6-minute wind profiler data %Y%m%d_%H00
FSL2    ^FSL\.NetCDF\.NOAAnet\.windprofiler\.(01hr|06min)\.(....)(...)(..)
        FILE
        data/pub/native/profiler/wind/\1/\2%m%d/PROFILER_wind_\1_\2%m%d_\400.nc
#
# FSL hourly summary and 6-minute RASS profiler data %Y%m%d_%H00
# FSL.NetCDF.NOAAnet.RASS.06min.20052731424
FSL2    ^FSL\.NetCDF\.NOAAnet\.RASS\.(01hr|06min)\.(....)(...)(..)
        FILE
        data/pub/native/profiler/RASS/\1/\2%m%d/PROFILER_RASS_\1_\2%m%d_\400.nc
#
# profiler data in BUFR format
HRS     ^(IUPT0[1-4]|IUAK01) .... ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/profiler/bufr/profiler/PROFILER_(\2:yyyy)(\2:mm)\2_0000.bufr
#
#
# profiler data in BUFR format
HRS     ^IUPT41 KBOU ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/profiler/bufr/profiler1/PROFILER_(\1:yyyy)(\1:mm)\1_0000.bufr
#
#
# profiler data in BUFR format
HRS     ^IUPT42 KBOU ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/profiler/bufr/profiler2/PROFILER_(\1:yyyy)(\1:mm)\1_0000.bufr
#
#
# profiler data in BUFR format
HRS     ^IUPT43 KBOU ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/profiler/bufr/profiler3/PROFILER_(\1:yyyy)(\1:mm)\1_0000.bufr
#
# profiler data in BUFR format
HRS     ^ISAT.. KBOU ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/profiler/bufr/profiler4/PROFILER_(\1:yyyy)(\1:mm)\1_0000.bufr
#
# profiler data in BUFR format
HRS     ^IUPC0[12] .... ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/profiler/bufr/profiler5/PROFILER_(\1:yyyy)(\1:mm)\1_0000.bufr
#
# RADAR grid products
#
HRS     ^HAXA00 K... ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/radar/10km_mosaic/(\1:yyyy)(\1:mm)\1/RADAR_10km_mosaic_(\1:yyyy)(\1:mm)\1_\2\3.grib1
#
# DPA (SDUS8.)
HRS     ^SDUS8. (....) ([0-3][0-9])([0-2][0-9])([0-9][0-9]).*/pDPA
        FILE
        
data/pub/native/radar/level3/DPA/\1/Level3_\1_DPA_(\2:yyyy)(\2:mm)\2_0000.dpa
#       
data/pub/native/radar/level3/DPA/\1/(\2:yyyy)(\2:mm)\2/Level3_\1_DPA_(\2:yyyy)(\2:mm)\2_0000.dpa
#
# RCM (SDUS4.)
#HRS    ^SDUS4. (....) ([0-3][0-9])([0-2][0-9])([0-9][0-9]).*/pRCM
#       FILE
#       
data/pub/native/radar/level3/RCM/\1/Level3_\1_RCM_(\2:yyyy)(\2:mm)\2_0000.rcm
#       
data/pub/native/radar/level3/RCM/\1/(\2:yyyy)(\2:mm)\2/Level3_\1_RCM_(\2:yyyy)(\2:mm)\2_0000.rcm
#
# 1 km Regional N0R Composite - Creates PNG-compressed AREA for FNEXRAD feed
#FNEXRAD        ^radar_mosaic_regional_(...)
#       PIPE    -close
#       util/mcgribdec \1 
/data/ldm/pub/native/radar/national/radar_regional.grib1 6100
#
# 6 km National N0R Composite - Creates PNG-compressed AREA for FNEXRAD feed
#FNEXRAD        ^radar_mosaic_national
#       PIPE    -close
#       util/mcgribdec NAT 
/data/ldm/pub/native/radar/national/radar_national.grib1 6200
#
# 10 km National RCM Composite - Creates PNG-compressed AREA for FNEXRAD feed
#HRS    ^HAXA00 K...
#       PIPE    -close
#       util/mcgribdec RCM 
/data/ldm/pub/native/radar/national/radar_national_rcm.grib1
#
#
# Model files in grib1 format
#
# GFS/AVN grid output
# Grid #201 N. Hemisphere 381km: ^Y.A... KWBC.*/m(AVN|GFS|SSIAVN|SSIGFS)
# Grid #202 CONUS         191km: ^Y.I... KWBC.*/m(AVN|GFS|SSIAVN|SSIGFS)
# Grid #203 Alaska        191km: ^Y.J... KWBC.*/m(AVN|GFS|SSIAVN|SSIGFS)
# Grid #211 CONUS          80km: ^Y.Q... KWBC.*/m(AVN|GFS|SSIAVN|SSIGFS)
# Grid #213 CONUS          95km: ^Y.H... KWBC.*/m(AVN|GFS|SSIAVN|SSIGFS)
# Select any/all grids desired from [QAIJH]
#
HRS     ^[YZ].(A)... KWB. ([0-3][0-9])([0-2][0-9]).*/m(AVN|GFS|SSIAVN|SSIGFS)
        FILE
        
data/pub/native/grid/NCEP/GFS/N_Hemisphere_381km/GFS_N_Hemisphere_381km_(\2:yyyy)(\2:mm)\2_\300.grib1
HRS     ^[YZ].(I)... KWB. ([0-3][0-9])([0-2][0-9]).*/m(AVN|GFS|SSIAVN|SSIGFS)
        FILE
        
data/pub/native/grid/NCEP/GFS/CONUS_191km/GFS_CONUS_191km_(\2:yyyy)(\2:mm)\2_\300.grib1
HRS     ^[YZ].(J)... KWB. ([0-3][0-9])([0-2][0-9]).*/m(AVN|GFS|SSIAVN|SSIGFS)
        FILE
        
data/pub/native/grid/NCEP/GFS/Alaska_191km/GFS_Alaska_191km_(\2:yyyy)(\2:mm)\2_\300.grib1
HRS     ^[YZ].(Q)... KWB. ([0-3][0-9])([0-2][0-9]).*/m(AVN|GFS|SSIAVN|SSIGFS)
        FILE
        
data/pub/native/grid/NCEP/GFS/CONUS_80km/GFS_CONUS_80km_(\2:yyyy)(\2:mm)\2_\300.grib1
HRS     ^[YZ].(H)... KWB. ([0-3][0-9])([0-2][0-9]).*/m(AVN|GFS|SSIAVN|SSIGFS)
        FILE
        
data/pub/native/grid/NCEP/GFS/CONUS_95km/GFS_CONUS_95km_(\2:yyyy)(\2:mm)\2_\300.grib1
#
# GFS grid on thinned grids, global 1.25 regular grid
HRS     ^H.[I-P]... KWB. (..)(..).*/m(AVN|SSIAVN|GFS|SSIGFS)
        FILE
        
data/pub/native/grid/NCEP/GFS/Global_1p25deg/GFS_Global_1p25deg_(\1:yyyy)(\1:mm)\1_\200.grib1
#
# Spectral Global grid - made up of grids I-P that are thinned grids
HRS     ^H.[I-P]... KWB[^K] ([0-3][0-9])([0-2][0-9])
        FILE
        
data/pub/native/grid/NCEP/GFS/Spectral_1p25deg/GFS_Spectral_1p25deg_(\1:yyyy)(\1:mm)\1_\200.grib1
#
#Ensemble members
HRS     ^H.[I-P]... KWBK ([0-3][0-9])([0-2][0-9])
        FILE
        
data/pub/native/grid/NCEP/GFS/Ensemble_1p25deg/GFS_Ensemble_1p25deg_(\1:yyyy)(\1:mm)\1_\200.grib1
#
# Grid #204 Hawaii        160km: ^Y.K... KWBH
# Grid #205 Puerto Rico   191km: ^Y.L... KWBH
#
HRS     ^[YZ].(K)... KWBH ([0-3][0-9])([0-2][0-9])
        FILE
        
data/pub/native/grid/NCEP/GFS/Hawaii_160km/GFS_Hawaii_160km_(\2:yyyy)(\2:mm)\2_\300.grib1
HRS     ^[YZ].(L)... KWBH ([0-3][0-9])([0-2][0-9])
        FILE
        
data/pub/native/grid/NCEP/GFS/Puerto_Rico_191km/GFS_Puerto_Rico_191km_(\2:yyyy)(\2:mm)\2_\300.grib1
#
#  GFS grid - Extended range grids E and F
#
HRS     ^H.[EF][A-Z][0-9][0-9] KWB. ([0-3][0-9])([0-2][0-9])..
        FILE
        
data/pub/native/grid/NCEP/GFS/Extended_Global_5p0deg/GFS_Extended_Global_5p0deg_(\1:yyyy)(\1:mm)\1_\200.grib1
#
# ETA/NAM grids
# Grid #207 95km Alaska:   ^[YZ].N.*/m(ETA|NAM)
# Grid #211 80km CONUS:    ^[YZ].Q.*/m(ETA|NAM)
# Grid #212 40km CONUS:    ^[YZ].R.*/m(ETA|NAM)
# Grid #215 20km CONUS:    ^[YZ].U.*/m(ETA|NAM)
# Grid #216 45km Alaska:   ^[YZ].V.*/m(ETA|NAM)
# Grid #217 22km Alaska:   ^[YZ].Y.*/m(ETA|NAM)
# Select any/all grids desired from [QRUT]
HRS     ^[YZ].(N)... KWB. ([0-3][0-9])([0-2][0-9]).*/m(ETA|NAM)
        FILE
        
data/pub/native/grid/NCEP/NAM/Alaska_95km/NAM_Alaska_95km_(\2:yyyy)(\2:mm)\2_\300.grib1
HRS     ^[YZ].(Q)... KWB. ([0-3][0-9])([0-2][0-9]).*/m(ETA|NAM)
        FILE
        
data/pub/native/grid/NCEP/NAM/CONUS_80km/NAM_CONUS_80km_(\2:yyyy)(\2:mm)\2_\300.grib1
HRS     ^[YZ].(R)... KWB. ([0-3][0-9])([0-2][0-9]).*/m(ETA|NAM)
        FILE
        
data/pub/native/grid/NCEP/NAM/CONUS_40km/noaaport/NAM_CONUS_40km_noaaport_(\2:yyyy)(\2:mm)\2_\300.grib1
HRS     ^[YZ].(U)... KWB. ([0-3][0-9])([0-2][0-9]).*/m(ETA|NAM)
        FILE
        
data/pub/native/grid/NCEP/NAM/CONUS_20km/noaaport/NAM_CONUS_20km_noaaport_(\2:yyyy)(\2:mm)\2_\300.grib1
HRS     ^[YZ].(V)... KWB. ([0-3][0-9])([0-2][0-9]).*/m(ETA|NAM)
        FILE
        
data/pub/native/grid/NCEP/NAM/Alaska_45km/noaaport/NAM_Alaska_45km_noaaport_(\2:yyyy)(\2:mm)\2_\300.grib1
HRS     ^[YZ].(Y)... KWB. ([0-3][0-9])([0-2][0-9]).*/m(ETA|NAM)
        FILE
        
data/pub/native/grid/NCEP/NAM/Alaska_22km/NAM_Alaska_22km_(\2:yyyy)(\2:mm)\2_\300.grib1
#
# RUC grid output
# Grid #211 CONUS   80km: ^[YZ].Q.*/mRUC
# Grid #236 CONUS   40km: ^[YZ].W.*/mRUC
#
HRS     ^[YZ].(Q)... KWB. ([0-3][0-9])([0-2][0-9]).*/mRUC
        FILE
        
data/pub/native/grid/NCEP/RUC/CONUS_80km/RUC_CONUS_80km_(\2:yyyy)(\2:mm)\2_\300.grib1
HRS     ^[YZ].(W)... KWB. ([0-3][0-9])([0-2][0-9]).*/mRUC
        FILE
        
data/pub/native/grid/NCEP/RUC/CONUS_40km/RUC_CONUS_40km_(\2:yyyy)(\2:mm)\2_\300.grib1
#
# RUC2 grid on Lambert conformal CONUS grid
HRS     ^Y.W... KWB. (..)(..).*/mRUC2
        FILE
        
data/pub/native/grid/NCEP/RUC2/CONUS_40km/RUC2_CONUS_40km_(\1:yyyy)(\1:mm)\1_\200.grib1
#
# Spectral Global grid - made up of grids A,B,C,D Forecasts (wind, waves, temp)
HRS     ^H[A-Z][ABCD][A-Z][0-9][0-9] KWB. ([0-3][0-9])([0-2][0-9])..
        FILE
        
data/pub/native/grid/NCEP/SPECTRAL/Global_5x2p5deg/SPECTRAL_Global_5x2p5deg_(\1:yyyy)(\1:mm)\1_\200.grib1
#
# ocean grid
#
HRS     ^H[HKLTUV][T-W][ACEIK]93 KWB. ([0-3][0-9])([0-2][0-9])
        FILE
        
data/pub/native/grid/NCEP/OCEAN/Global_2x2deg/OCEAN_Global_2x2deg_(\1:yyyy)(\1:mm)\1_\200.grib1
#
# NOW and PWAV grids on global 5.0 x 2.5 degree grids
HRS     ^H.[A-D]... .... (..)(..).*(/mNOW|/mPWAV)
        FILE
        
data/pub/native/grid/NCEP/OCEAN/Global_5x2p5deg/OCEAN_Global_5x2p5deg_(\1:yyyy)(\1:mm)\1_\200.grib1
#
# SST grid on global 5.0 x 2.5 degree grids
HRS     ^H.[A-D]... .... (..)(..).*/mSST
        FILE
        
data/pub/native/grid/NCEP/SST/Global_5x2p5deg/SST_Global_5x2p5deg_(\1:yyyy)(\1:mm)\1_\200.grib1
#
# SST grid on global 2.0 x 2.0 degree grid
HRS     ^H.[T-W]... .... (..)(..).*/mSST
        FILE
        
data/pub/native/grid/NCEP/SST/Global_2x2deg/SST_Global_2x2deg_(\1:yyyy)(\1:mm)\1_\200.grib1
#
# ECMWF grid on global 2.5 x 2.5 degree grid
HRS     ^H.[A-L]... ECMF (..)(..)
        FILE
        
data/pub/native/grid/ECMWF/Global_2p5/ECMWF_Global_2p5_(\1:yyyy)(\1:mm)\1_\200.grib1
#
# ECMWF GRIB products, decode into yymmdd_ecmf#.gem (e1=global,e2=+/-35lat)
#
#HRS    ^H[RHPT][A-L][A-Z][0-9][0-9] ECMF ([0-3][0-9])([0-2][0-9])
#       FILE
#       data/pub/native/grid/ECMWF/model_xx/ECMWF1_(\1:yyyy)(\1:mm)\1_\200.grib1
#
#HRS    ^H[UV][A-L][A-Z][0-9][0-9] ECMF ([0-3][0-9])([0-2][0-9])
#       FILE
#       data/pub/native/grid/ECMWF/model_xx/ECMWF2_(\1:yyyy)(\1:mm)\1_\200.grib1
#
# Global UKMet thinned grids
HRS     ^H.[I-P]... EGRR ([0-3][0-9])([0-2][0-9])
        FILE
        data/pub/native/grid/UKMET/thin/UKMET_(\1:yyyy)(\1:mm)\1_\200.grib1
#
# other soundings and surface products in BUFR format
#
# NCEP NAM MODEL BUFR SOUNDING
# JUSA41 JUSA42 JUSB43 JUSB44 JUSB45 JUSB46 JUSX47 ? JUSX48 JUSX49 KWNO
HRS     ^JUS.41 KWNO ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        data/pub/native/soundings/NAM41/SoundingNAM_(\1:yyyy)(\1:mm)\1_\200.bufr
HRS     ^JUS.42 KWNO ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        data/pub/native/soundings/NAM42/SoundingNAM_(\1:yyyy)(\1:mm)\1_\200.bufr
HRS     ^JUS.43 KWNO ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        data/pub/native/soundings/NAM43/SoundingNAM_(\1:yyyy)(\1:mm)\1_\200.bufr
HRS     ^JUS.44 KWNO ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        data/pub/native/soundings/NAM44/SoundingNAM_(\1:yyyy)(\1:mm)\1_\200.bufr
HRS     ^JUS.45 KWNO ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        data/pub/native/soundings/NAM45/SoundingNAM_(\1:yyyy)(\1:mm)\1_\200.bufr
HRS     ^JUS.46 KWNO ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        data/pub/native/soundings/NAM46/SoundingNAM_(\1:yyyy)(\1:mm)\1_\200.bufr
HRS     ^JUS.47 KWNO ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        data/pub/native/soundings/NAM47/SoundingNAM_(\1:yyyy)(\1:mm)\1_\200.bufr
HRS     ^JUS.48 KWNO ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        data/pub/native/soundings/NAM48/SoundingNAM_(\1:yyyy)(\1:mm)\1_\200.bufr
HRS     ^JUS.49 KWNO ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        data/pub/native/soundings/NAM49/SoundingNAM_(\1:yyyy)(\1:mm)\1_\200.bufr
#
# NCEP GFS MODEL BUFR SOUNDING
# JUSA41 JUSA42 JUSB43 JUSB44 JUSB45 JUSB46 JUSX47 JUSX48 JUSX49 KWBC
HRS     ^JUS.4. KWBC ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/soundings/GFS/SoundingVerticalGFS_(\1:yyyy)(\1:mm)\1_0000.bufr
#
#
# Surface data (satellite)
#
HRS     ^ISXT.. KBOU ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/surface/satellite/SurfaceSatellite_(\1:yyyy)(\1:mm)\1_0000.bufr
#
#
# Surface data (satellite)
#
HRS     ^(ISXX0.|FPAK79|SXUS..|SFUS41|SACN..|PEIK98) .... 
([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/surface/satellite1/SurfaceSatellite_(\2:yyyy)(\2:mm)\2_0000.bufr
#
# Surface data (satellite)
#
HRS     ^ISZ.59 .... ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/surface/satellite2/SurfaceSatellite_(\1:yyyy)(\1:mm)\1_0000.bufr
#
# Surface data (satellite)
#
HRS     ^ISXA.. .... ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/surface/satellite3/SurfaceSatellite_(\1:yyyy)(\1:mm)\1_0000.bufr
#
#
# Surface data (satellite)
#
HRS     ^(ISXX0.|FPAK79|SXUS..|SFUS41|SACN..|PEIK98) .... 
([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/surface/satellite1/SurfaceSatellite_(\2:yyyy)(\2:mm)\2_0000.bufr
#
# Single level upper-air data (other than satellite)
#
HRS     ^IUA... (CWAO|EGRR) ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/soundings/singlelevel/aircraft/SoundingSingleLevelAircraft_(\2:yyyy)(\2:mm)\2_0000.bufr
#
# Vertical soundings (other than satellite)
HRS     ^IUS[TX].[68] KWBC ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/soundings/vertical/radiosonde8/SoundingVerticalRadiosonde_(\1:yyyy)(\1:mm)\1_0000.bufr
#
#
# Vertical soundings (other than satellite)
HRS     ^IUS[TX].2 KWBC ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/soundings/vertical/radiosonde2/SoundingVerticalRadiosonde_(\1:yyyy)(\1:mm)\1_0000.bufr
#
#
# Vertical soundings (other than satellite)
HRS     ^IUS[TX].[13] KWBC ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/soundings/vertical/radiosonde3/SoundingVerticalRadiosonde_(\1:yyyy)(\1:mm)\1_0000.bufr
#
#
# Vertical soundings (other than satellite)
HRS     ^IUS[TX].4 KWBC ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/soundings/vertical/radiosonde4/SoundingVerticalRadiosonde_(\1:yyyy)(\1:mm)\1_0000.bufr
#
# Single level upper-air data (satellite)
HRS     ^J.CX.1 KNES ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/soundings/singlelevel/satellite/SoundingSingleLevelSatellite_(\1:yyyy)(\1:mm)\1_0000.bufr
#
#
# Single level upper-air data (satellite)
HRS     ^(IUCN[45678]|IUCS[5678]|JHCX1|JRCX7). .... 
([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/soundings/singlelevel/satellite1/SoundingSingleLevelSatellite_(\2:yyyy)(\2:mm)\2_0000.bufr
#
# Surface data - land
#
HRS     ^JSAT.. KKCI ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        data/pub/native/surface/land/SurfaceLand_(\1:yyyy)(\1:mm)\1_0000.bufr
#
# Vertical soundings (satellite)
HRS     ^JUTX.. KNES ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/soundings/vertical/satellite/SoundingVerticalSatellite_(\1:yyyy)(\1:mm)\1_0000.bufr
#
#
# Vertical soundings (satellite)
HRS     ^IUTX.. .... ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/soundings/vertical/satellite1/SoundingVerticalSatellite_(\1:yyyy)(\1:mm)\1_0000.bufr
#
# Vertical soundings (other than satellite)
HRS     ^IXUS68 PACR ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/soundings/vertical/wind_profiler/SoundingVerticalWindProfiler_(\1:yyyy)(\1:mm)\1_0000.bufr
#
# Synoptic data feature 7
HRS     ^JUVE.. KKCI ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/synoptic/feature7/SynopticFeature7_(\1:yyyy)(\1:mm)\1_0000.bufr
#
# Synoptic data feature 6
HRS     ^JUCE.. .... ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/synoptic/feature6/SynopticFeature6_(\1:yyyy)(\1:mm)\1_0000.bufr
#
# Synoptic data feature 5
HRS     ^JUFE.. .... ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/synoptic/feature5/SynopticFeature5_(\1:yyyy)(\1:mm)\1_0000.bufr
#
# Synoptic data feature 4
HRS     ^(YUZI96 KWBC|JUBE99 ....) ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/synoptic/feature4/SynopticFeature4_(\2:yyyy)(\2:mm)\2_0000.bufr
#
# Synoptic data feature 3
HRS     ^JUVE.. EGRR ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/synoptic/feature3/SynopticFeature3_(\1:yyyy)(\1:mm)\1_0000.bufr
#
# Synoptic data feature 2
HRS     ^JUTE.. .... ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/synoptic/feature2/SynopticFeature2_(\1:yyyy)(\1:mm)\1_0000.bufr
#
# Synoptic data feature 1
HRS     ^JUWE.. .... ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/synoptic/feature1/SynopticFeature1_(\1:yyyy)(\1:mm)\1_0000.bufr
#
HRS     ^JUVE.. .... ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        data/pub/native/other/JUV/JUV_(\1:yyyy)(\1:mm)\1_0000.bufr
#
HRS     ^IUPC0. .... ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        data/pub/native/other/IUPC/IUPC_(\1:yyyy)(\1:mm)\1_0000.bufr
#
HRS     ^JSMT.. .... ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        data/pub/native/other/JSMT/JSMT_(\1:yyyy)(\1:mm)\1_0000.bufr
#
HRS     ^JU[JOMN]E00 .... ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        data/pub/native/other/JUE/JUE_(\1:yyyy)(\1:mm)\1_0000.bufr
#
HRS     ^JSML1. .... ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        data/pub/native/other/JSML1/JSML1_(\1:yyyy)(\1:mm)\1_0000.bufr
#
# Other NMC assorted binary products
HRS     ^_ELSE_$
        FILE
        data/pub/native/other/hrs/%Y%m%d_%H_HRS.wmo
#
# CMC GEM grid data
CMC     
CMC_GEM_reg_.*_.*_([12][0-9][0-9][0-9])([0-1][0-9])([0-3][0-9])([0-2][0-9])_(.*).grib
        FILE    data/pub/native/grid/CMC/GEM/GEM_\1\2\3_\400.grib1
#
# FNMOC data available via IDD (NOGAPS & COAMPS grids)
# -------------------------
FNMOC   ^US058....-.......(0058)_...._.........(....)(..)(..)(..)_(.*)
        FILE    data/pub/native/grid/FNMOC/NOGAPS/NOGAPS_\2\3\4_\5_\1_\6
FNMOC   
^US058....-.......([^0]...|0[^0]..|00[^5].|005[^8])_...._.........(....)(..)(..)(..)_(.*)
        FILE    data/pub/native/grid/FNMOC/COAMPS/COAMPS_\2\3\4_\5_\1_\6
#
# WSETA 
SPARE   ^(20........)/(model_center.dat)
        FILE    -close  data/pub/native/grid/UNIDATA/WSETA/\1_\2
SPARE   ^rad/floater/(..........)/(n0r_........_....)
        FILE    -close  data/pub/native/grid/UNIDATA/WSETA/n0r/\1/WSETA_\2
#
#               
# NPORT
#       
NPORT   ^[LM].B... KWBE ([0-3][0-9])([0-2][0-9])([0-6][0-9])
        FILE    
data/pub/native/grid/NCEP/NAM/CONUS_12km/NAM_CONUS_12km_(\1:yyyy)(\1:mm)\1_\200.grib2
NPORT   ^[LM].S... KWBE ([0-3][0-9])([0-2][0-9])([0-6][0-9])
        FILE    
data/pub/native/grid/NCEP/NAM/Alaska_11km/NAM_Alaska_11km_(\1:yyyy)(\1:mm)\1_\200.grib2
NPORT   ^[LM].E... KWBD ([0-3][0-9])([0-2][0-9])([0-6][0-9])
        FILE    
data/pub/native/grid/NCEP/DGEX/CONUS_12km/DGEX_CONUS_12km_(\1:yyyy)(\1:mm)\1_\200.grib2
NPORT   ^[LM].F... KWBD ([0-3][0-9])([0-2][0-9])([0-6][0-9])
        FILE    
data/pub/native/grid/NCEP/DGEX/Alaska_12km/DGEX_Alaska_12km_(\1:yyyy)(\1:mm)\1_\200.grib2
NPORT   ^[LM].U... KWBN ([0-3][0-9])([0-2][0-9])([0-6][0-9])
        FILE    
data/pub/native/grid/NCEP/NDFD/CONUS_5km/NDFD_CONUS_5km_(\1:yyyy)(\1:mm)\1_\200.grib2
#NPORT  ^[LM].I... KWBJ ([0-3][0-9])([0-2][0-9])([0-6][0-9])
#       FILE    
data/pub/native/grid/NCEP/NCEP/GFS/CONUS_191km/GFS_CONUS_191km_(\1:yyyy)(\1:mm)\1_\200.grib2
#
#
# surface data in netCDF format
#
# metar decoder, creates NetCDF files formated Surface_METAR_yyyymmdd_hh00.nc
# All aviation reports including metar tests. old netcdf file format
DDS|IDS ^S(A....|P....|XUS8.) .... ([0-3][0-9])
        PIPE    decoders/metar2nc
        etc/metar.cdl
        data/pub/decoded/netcdf/surface/metarold
#
# metar decoder, creates NetCDF files with new CDL format on a daily basis
# formated data/decoded/yyyymmdd_metar.nc
DDS|IDS ^S(A....|P....|XUS8.) .... ([0-3][0-9])
        PIPE    decoders/metar2nc.new1
        etc/metar.cdl.new1
        data/pub/decoded/netcdf/surface/metar
#
# upper air perl decoder
DDS|IDS ^U[ABDEFGHIJKLMPQRSTXZ].... .... ([0-3][0-9])
        PIPE    decoders/ua2nc
        etc/raob.cdl
        data/pub/decoded/netcdf/upperair
#
# buoy reports in yymmddhh_buoy.nc decoded into NetCDF format
DDS|IDS ^S[NS].... .... ([0-3][0-9])
        PIPE    decoders/buoy2nc
        e,cdl
        data/pub/decoded/netcdf/surface/buoy
#
# Synoptic reports in yymmddhh_syn.nc decoded into NetCDF format
DDS|IDS ^SM.... .... ([0-3][0-9])
        PIPE    decoders/syn2nc
        etc/syn.cdl
        data/pub/decoded/netcdf/surface/synoptic
#
# grid data in netCDF format
#
# GFS grid on Lambert conformal CONUS grid replacing avn-q
HRS     ^Y.Q... KWB. (..)(..).*/m(AVN|SSIAVN|GFS|SSIGFS)
        PIPE    decoders/gribtonc
        etc/gfs_211.cdl
        
data/pub/decoded/netcdf/grid/NCEP/GFS/CONUS_80km/GFS_CONUS_80km_(\1:yyyy)(\1:mm)\1_\200.nc
#
# GFS grid on thinned grids, interpolated to global 5.0 x 2.5 regular grid
HRS     ^H.[I-P]... KWB. (..)(..).*/m(AVN|SSIAVN|GFS|SSIGFS)
        PIPE    decoders/gribtonc
        -q "lin,dlat=2.5,dlon=5.0" etc/gfs_37-44.cdl
        
data/pub/decoded/netcdf/grid/NCEP/GFS/Global_5x2p5deg/GFS_Global_5x2p5deg_(\1:yyyy)(\1:mm)\1_\200.nc
#
# ECMWF grid on global 2.5 x 2.5 degree grid
HRS     ^H.[A-L]... ECMF (..)(..)
        PIPE    decoders/gribtonc
        etc/ecmwf_1-12.cdl
        
data/pub/decoded/netcdf/grid/ECMWF/Global_2p5/ECMWF_Global_2p5_(\1:yyyy)(\1:mm)\1_\200.nc
#
# ETA grid on Lambert conformal CONUS grid
HRS     ^[YZ].Q... KWB. (..)(..).*/m(ETA|NAM)
        PIPE    decoders/gribtonc
        etc/eta_211.cdl
        
data/pub/decoded/netcdf/grid/NCEP/NAM/CONUS_80km/NAM_CONUS_80km_(\1:yyyy)(\1:mm)\1_\200.nc
#
# GFS grid on global 5.0 x 5.0 degree grid
HRS     ^H.[EF]... KWB. (..)(..)
        PIPE    decoders/gribtonc
        etc/gfs_25-26.cdl
        
data/pub/decoded/netcdf/grid/NCEP/GFS/Extended_Global_5p0deg/GFS_Extended_Global_5p0deg_(\1:yyyy)(\1:mm)\1_\200.nc
#
# NGM grid on Lambert conformal CONUS grid
#HRS    ^Y.Q... KWB. (..)(..).*/mNGM
#       PIPE    decoders/gribtonc
#       etc/ngm_211.cdl
#       
data/pub/decoded/netcdf/grid/NCEP/NGM/CONUS_80km/NGM_CONUS_80km_(\1:yyyy)(\1:mm)\1_\200.nc
#
# NOW and PWAV grids on global 5.0 x 2.5 degree grids
HRS     ^H.[A-D]... .... (..)(..).*(/mNOW|/mPWAV)
        PIPE    decoders/gribtonc
        etc/ocean_21-24.cdl
        
data/pub/decoded/netcdf/grid/NCEP/OCEAN/Global_5x2p5deg/OCEAN_Global_5x2p5deg_(\1:yyyy)(\1:mm)\1_\200.nc
#
# RUC grid on Lambert conformal CONUS grid
HRS     ^[YZ].Q... KWB. (..)(..).*/mRUC
        PIPE    decoders/gribtonc
        etc/ruc_211.cdl
        
data/pub/decoded/netcdf/grid/NCEP/RUC/CONUS_80km/RUC_CONUS_80km_(\1:yyyy)(\1:mm)\1_\200.nc
#
# RUC2 grid on Lambert conformal CONUS grid
HRS     ^Y.W... KWB. (..)(..).*/mRUC2
        PIPE    decoders/gribtonc
        etc/ruc2_236.cdl
        
data/pub/decoded/netcdf/grid/NCEP/RUC2/CONUS_40km/RUC2_CONUS_40km_(\1:yyyy)(\1:mm)\1_\200.nc
#
# SST grid on global 5.0 x 2.5 degree grids
HRS     ^H.[A-D]... .... (..)(..).*/mSST
        PIPE    decoders/gribtonc
        etc/sst_21-24.cdl
        
data/pub/decoded/netcdf/grid/NCEP/SST/Global_5x2p5deg/SST_Global_5x2p5deg_(\1:yyyy)(\1:mm)\1_\200.nc
#
# SST grid on global 2.0 x 2.0 degree grid
HRS     ^H.[T-W]... .... (..)(..).*/mSST
        PIPE    decoders/gribtonc
        etc/sst_61-64.cdl
        
data/pub/decoded/netcdf/grid/NCEP/SST/Global_2x2deg/SST_Global_2x2deg_(\1:yyyy)(\1:mm)\1_\200.nc
#
# png compressed NEXRAD Level III radar composites in GINI format
#FNEXRAD                ^rad/NEXRCOMP/(...)/(...)_(........)_(....)
#               FILE
#               
data/pub/native/radar/composite/\2/\1/\3/Level3_Composite_\2_\1_\3_\4.gini
#
#
# NOAAPORT GINI Images
#
# png compressed NEXRAD Level III radar composites in GINI format
FNEXRAD ^rad/NEXRCOMP/(...)/(...)_(........)_(....)
        PIPE    -close
        util/pngg2gini -vl logs/ldm-mcidas.log
        
data/pub/native/radar/composite/\2/\1/\3/Level3_Composite_\2_\1_\3_\4.gini
#
# end of pqact.thredds
#
#
#PCWS   ^FSL\.NetCDF\.ACARS\.QC\.(...........)
#       FILE    -close  data/ucar_only/decoded/netcdf/acars/\1/\1.nc
# CONDUIT data directory structure
#
# pub
# pub/native
# pub/native/grid
# pub/native/grid/NCEP
#
# pub/native/grid/NCEP/GFS/Global_0p5deg
# pub/native/grid/NCEP/GFS/Global_onedeg
# pub/native/grid/NCEP/GFS/Global_2p5deg
#
# pub/native/grid/NCEP/RUC2/CONUS_20km/hybrid
# pub/native/grid/NCEP/RUC2/CONUS_20km/pressure
# pub/native/grid/NCEP/RUC2/CONUS_20km/surface
#
# pub/native/grid/NCEP/NAM/Alaska_45km/conduit
# pub/native/grid/NCEP/NAM/CONUS_20km/selectsurface
# pub/native/grid/NCEP/NAM/CONUS_20km/surface
# pub/native/grid/NCEP/NAM/CONUS_40km/conduit
# pub/native/grid/NCEP/NAM/Polar_90km
#
# pub/native/grid/NCEP/GFS/Global_1p0deg_Ensemble
# pub/native/grid/NCEP/GFS/Global_2p5deg_Ensemble
#
# CONDUIT Data actions
#
# GFS grids
# example product
#/afs/.nwstg.nws.noaa.gov/ftp/SL.us008001/ST.opnl/MT.gfs_CY.12/RD.20041213/PT.grid_DF.gr1/fh.0078_tl.press_gr.onedeg
 
#!grib2/ncep/AVN/#000/200412131200/F6900/RELH/3300-10000 SGMA!
#
# GFS grids Global_0p5, Global_onedeg, and Global_2p5 grids
CONDUIT         
MT.gfs_CY.(..)/RD.(........)/PT.grid_DF.gr([12])/(fh.....)_tl.press_gr.(......)
        FILE    -close  
data/pub/native/grid/NCEP/GFS/Global_\5/GFS_Global_\5_\2_\100.grib\3
#
# RUC2 grids
#
# hybrid
CONDUIT         
MT.ruc_CY.(..)/RD.(........)/PT.grid_DF.gr1/(fh.....)_tl.press_gr.bgrib20
        FILE    -close  
data/pub/native/grid/NCEP/RUC2/CONUS_20km/hybrid/RUC2_CONUS_20km_hybrid_\2_\100.grib1
#
# surface
CONDUIT         
MT.ruc_CY.(..)/RD.(........)/PT.grid_DF.gr1/(fh.....)_tl.press_gr.sgrib20
        FILE    -close  
data/pub/native/grid/NCEP/RUC2/CONUS_20km/surface/RUC2_CONUS_20km_surface_\2_\100.grib1
#
# pressure
CONDUIT         
MT.ruc_CY.(..)/RD.(........)/PT.grid_DF.gr1/(fh.....)_tl.press_gr.us20km
        FILE    -close  
data/pub/native/grid/NCEP/RUC2/CONUS_20km/pressure/RUC2_CONUS_20km_pressure_\2_\100.grib1
#
# NAM grids
#
# example product
#/afs/.nwstg.nws.noaa.gov/ftp/SL.us008001/ST.opnl/MT.nam_CY.12/RD.20050813/PT.grid_DF.gr1/fh.0027_tl.press_gr.awipak
 !grib/ncep/ETA_84/#216/200508131200/F027/HGT/sfc! 000009
#
# Alaska_45km
CONDUIT         
MT.nam_CY.(..)/RD.(........)/PT.grid_DF.gr1/(fh.....)_tl.press_gr.(......) 
!grib/ncep/ETA_84/#216
        FILE    -close  
data/pub/native/grid/NCEP/NAM/Alaska_45km/conduit/NAM_Alaska_45km_conduit_\2_\100.grib1
#
# selectsurface
CONDUIT         
MT.nam_CY.(..)/RD.(........)/PT.grid_DF.gr1/(fh.....)_tl.press_gr.icwf20 
!grib/ncep/ETA_84/#215
        FILE    -close  
data/pub/native/grid/NCEP/NAM/CONUS_20km/selectsurface/NAM_CONUS_20km_selectsurface_\2_\100.grib1
#
# surface
CONDUIT         
MT.nam_CY.(..)/RD.(........)/PT.grid_DF.gr1/(fh.....)_tl.press_gr.awip20 
!grib/ncep/ETA_84/#215
        FILE    -close  
data/pub/native/grid/NCEP/NAM/CONUS_20km/surface/NAM_CONUS_20km_surface_\2_\100.grib1
#
# conduit feed
CONDUIT         
MT.nam_CY.(..)/RD.(........)/PT.grid_DF.gr1/(fh.....)_tl.press_gr.(......) 
!grib/ncep/ETA_84/#212
        FILE    -close  
data/pub/native/grid/NCEP/NAM/CONUS_40km/conduit/NAM_CONUS_40km_conduit_\2_\100.grib1
#
# Polar_90km
CONDUIT         
MT.nam_CY.(..)/RD.(........)/PT.grid_DF.gr1/(fh.....)_tl.press_gr.(......) 
!grib/ncep/ETA_84/#104
        FILE    -close  
data/pub/native/grid/NCEP/NAM/Polar_90km/NAM_Polar_90km_\2_\100.grib1
#
# MRF Ensemble grids
#
# example products
#/afs/.nwstg.nws.noaa.gov/ftp/SL.us008001/ST.opnl/MT.ensg_CY.12/RD.20050812/PT.grid_DF.gr1/fh.xxxx_tl.press_pa.z1000
 !grib/ncep/AVN/#002/200508121200/F300/HGT/1000_mb! 000040
#  
/afs/.nwstg.nws.noaa.gov/ftp/SL.us008001/ST.opnl/MT.ensg_CY.12/RD.20050812/PT.grid_DF.gr1/fh.xxxx_tl.press_pa.z1000
 !grib/ncep/SPEC_62_MRF/#002/200508121200/F006/HGT/1000_mb! 000049
#
# Global_1p0deg_Ensemble
CONDUIT         
MT.ensg_CY.(..)/RD.(........)/PT.grid_DF.gr1/(fh.xxxxhr)_tl.press_pa.(.*) !grib
        FILE    -close  
data/pub/native/grid/NCEP/GFS/Global_1p0deg_Ensemble/GFS_Global_1p0deg_Ensemble_\2_\100.grib1
#
# Global_2p5deg_Ensemble
CONDUIT         
MT.ensg_CY.(..)/RD.(........)/PT.grid_DF.gr1/(fh.xxxx)_tl.press_pa.(.*) !grib
        FILE    -close  
data/pub/native/grid/NCEP/GFS/Global_2p5deg_Ensemble/GFS_Global_2p5deg_Ensemble_\2_\100.grib1
#
#
#NMC2|SPARE     ^.status\.(.*) [0-9][0-9][0-9][0-9][0-9][0-9]
#       FILE    -close  data/pub/native/other/conduit/status/\1
#
# Soundings Products
#
# GFS soundings
#CONDUIT                
MT.gfs_CY.(..)/RD.(........)/PT.sndn_DF.bufr/(fh.....)_tl.press_gr.(......)
#       FILE    -close  
data/pub/native/grid/NCEP/GFS/Global_\5/GFS_Global_\5_\2_\100.grib\3
#
# This pqact defines the directory structure expected by the THREDDS Data Server
# default configuration.  It includes the pqact entries using decoders and FILE
# actions to create the structure. If this pqact is implemented then the 
# THREDDS Data Server should work without doing any data structure 
# configurations. Default configuration starting at the ~ldm/data directory:
#
# pub
# pub/native
# pub/native/radar/level2/{station}/{day}
# pub/native/radar/level3/{day}/{station}/{product}
#
#
# level2 radar from CRAFT  feed
#
CRAFT   
^L2-BZIP2/(....)/([0-9][0-9][0-9][0-9][0-1][0-9][0-3][0-9])([0-2][0-9][0-5][0-9])([0-9][0-9])
        FILE    data/pub/native/radar/level2/\1/\2/Level2_\1_\2_\3.ar2v
#
# level3 radar from NEXRAD feedtype
#
NNEXRAD ^SDUS[2357]. .... ([0-3][0-9])([0-2][0-9])([0-6][0-9]).*/p(...)(...)
        FILE    -close  
data/pub/native/radar/level3/\4/\5/(\1:yyyy)(\1:mm)\1/Level3_\5_\4_(\1:yyyy)(\1:mm)\1_\2\3.nids
#
#
# Configuration file for "manageFilesDirs.pl" utility.
#
# This utility manages the number of files in a directory structure.
#
# @param  startdir the directory where to start managing
# @param  pattern  the pattern to match files or directories, yyyymmdd is a 
#         place holder for year, month, day directories
# @param  number  the number of days of data to keep
#
# Once the script matches the pattern, it deletes the files or directories and
# then returns to the directory up one level. It skips files or directories
# that are links and prints out a message on files or directories that no
# action was taken on it. 
#
# Each line consists of a directory, a pattern, and the number of days to 
retain.
#
# Directory                                             pattern         days 
#
/data/ldm/pub/native/surface/metar                      yyyymmdd        7
/data/ldm/pub/native/surface/land                       bufr            7
/data/ldm/pub/native/surface/satellite                  bufr            7
/data/ldm/pub/native/surface/satellite1                 bufr            7
/data/ldm/pub/native/surface/satellite2                 bufr            7
/data/ldm/pub/native/surface/satellite3                 bufr            7
/data/ldm/pub/native/soundings                          bufr            7
/data/ldm/pub/native/synoptic                           bufr            7
/data/ldm/pub/native/profiler/bufr                      bufr            7
/data/ldm/pub/native/satellite                          yyyymmdd        7
/data/ldm/pub/native/radar/level2                       yyyymmdd        7
/data/ldm/pub/native/radar/level3                       yyyymmdd        7
/data/ldm/pub/native/radar/level3/DPA                   dpa             7
/data/ldm/pub/native/radar/composite                    yyyymmdd        7
/data/ldm/pub/native/radar/10km_mosaic                  yyyymmdd        7
/data/ldm/pub/native/profiler/RASS                      yyyymmdd        7
/data/ldm/pub/native/profiler/wind                      yyyymmdd        7
/data/ldm/pub/native/nwstg/text                         yyyymmdd        7
/data/ldm/pub/native/other/hrs                          HRS.wmo         7
/data/ldm/pub/native/other                              bufr            7
/data/ldm/pub/native/conduit                            RD.20           2
/data/ldm/pub/native/grid/CMC                           grib.$          7
/data/ldm/pub/native/grid/ECMWF                         grib.$          7
/data/ldm/pub/native/grid/NCEP                          grib.$          7
/data/ldm/pub/native/grid/UKMET                         grib.$          7
/data/ldm/pub/native/grid/UNIDATA/WSETA/n0r             yyyymmdd        7
#
/data/ldm/pub/decoded/netcdf/surface                    Surface         7
/data/ldm/pub/decoded/netcdf/upperair                   Upperair        7
/data/ldm/pub/decoded/netcdf/grid                       nc$             7
#
#!/usr/bin/perl
#
# Name:         manageFileDirs.pl
# 
# Author:       Robb Kambic
# Date  :       Sept 2, 2005
# 
# Purpose:      manage files in a directory sturcture.
#    
# Description:  
#
# @param  startdir the directory where to start managing
# @param  pattern  the pattern to match files or directories, yyyymmdd is a 
#         place holder for year, month, day directories
# @param  number  the number of days to keep
#
# @param  configuration file that has the above parameters on one line.
#
# Once the script matches the pattern, it deletes the files or directories and
# then returns to the directory up one level. It skips files or directories
# that are links and prints out a message on files or directories that no
# action was taken on it. 
#
# process command line switches

while ($_ = $ARGV[0], /^-/) {
         shift;
       last if /^--$/;
              /^-D(.*)/ && ($debug = $1);
             /^(-v)/ && $verbose++;
             /^(-d)/ && ( $startDir = shift ) ;
             /^(-p)/ && ( $pattern = shift ) ;
             /^(-n)/ && ( $number = shift ) ;
             /^(-f)/ && ( $conf = shift ) ;
}
# configuration file given to process
if( defined( $conf ) ) {
        print "Start ", `/bin/date`;
        open( CONF, "$conf" ) || die "cannot open $conf $!";
        while( <CONF> ) {
                next if( /^#/ );
                chop();
                ( $startDir, $pattern, $number ) = split( /\s+/, $_ );
                next if( $pattern eq "" || $number eq "" );
                startChecking();
        }
        close CONF;
        print "End ", `/bin/date`;
} else {
        exit 1 if( $pattern eq "" || $number eq "" );
        print "Start ", `/bin/date`;
        startChecking();
        print "End ", `/bin/date`;
}

sub startChecking {

if( ! -d $startDir ) {
        print "$startDir doesn't exist\n";
        return;
}
chdir( $startDir );
if( $pattern =~ /yyyymmdd/i ) {
        $pattern = "^\\d{8}"; # pattern set to 8 digits
}
opendir( TOP, $startDir ) || die "cannot open $startDir $!";
( @TOP ) = readdir( TOP );
closedir( TOP );
checkDirs( @TOP );

} # end startChecking

# checkDirs is a recursive routine used to walk the directory structure in a
# depth first search for the designated pattern. Once the pattern is found, it 
# removes the files/directories in excess of the days to keep.  After deleting
# files or directories matching the pattern, the routine returns up one 
# directory level.
#
sub checkDirs{

my ( @INODES ) = @_;
local( $i, $j, $delete, @subINODES );

#print "currently in ", `pwd`;
for( $i = 0; $i <= $#INODES; $i++ ) {
        # skip ., .., and links
        next if( $INODES[ $i ] =~ /^\.$|^\.\.$/ || -l $INODES[ $i ] );
        if( $INODES[ $i ] =~ /$pattern/ ) {
                print `/bin/date`;
                print "currently in ", `pwd`;
                # just manage files/dirs matching pattern
                $delete = "";
                for( $j = 0; $j <= $#INODES; $j++ ) {
                        next if( $INODES[ $j ] =~ /^\.$|^\.\.$/ || -l $INODES[ 
$j ] );
                        next unless ( $INODES[ $j ] =~ /$pattern/ && -M 
$INODES[ $j ] > $number );
                        $delete = $delete . " $INODES[ $j ]";
                        # if grib file delete gbx index and inventory too
                        if( $INODES[ $j ] =~ /grib.$/ ) {
                                $delete = $delete . " $INODES[ $j ]" . ".gbx";
                                $delete = $delete . " $INODES[ $j ]" . 
".fmrInv.xml";
                        }
                        # if bufr file delete bfx index too
                        if( $INODES[ $j ] =~ /bufr$/ ) {
                                $delete = $delete . " $INODES[ $j ]" . ".bfx";
                        }
                } 
                if( $delete ne "" ) {
                        print "`/bin/rm -r $delete`\n";
                        `/bin/rm -r $delete`;
                }
                #print "currently in ", `pwd`;
                return;
        } elsif( -d $INODES[ $i ] ) {
                opendir( DIR, $INODES[ $i ] ) || 
                        die "cannot open $INODES[ $i ] $!";
                ( @subINODES ) = readdir( DIR );
                closedir( DIR );
                chdir( $INODES[ $i ] );
                #print "currently in ", `pwd`;
                checkDirs( @subINODES );
                chdir( ".." );
                #print "currently in ", `pwd`;
        } elsif( $INODES[ $i ] =~ /\.scour/ ) {
                print "`/bin/rm $INODES[ $i ]`\n";
                `/bin/rm $INODES[ $i ]`;
        } else {
                print "currently in ", `pwd`;
                print "No action taken on $INODES[ $i ]\n";
        } 
}
} # end checkDirs

NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.