[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

20020830: NOAAPORT GINI imagery availability in the IDD (cont.)

>From: "Arthur A. Person" <address@hidden>
>Organization: Penn State
>Keywords: 200208281756.g7SHuTZ20665 IDD NOAAPORT GINI

Hi Art,

re: will make NIMAGE available to sites that can handle the load

>I think we should be able to handle it.  I did some queue checking and it
>looks like we peak at 2 to 2-1/2 GB/hr incoming data with an average of,
>perhaps, 1.5 GB/hr.  Adding an additional 200 MB/hr should not present too
>much of an additional load.  I think we'll be okay on storage as well.
>If you can get us "allowed", I'll turn it on and see what happens.

OK.  I added an allow of ldm.meteo.psu.edu to the ldmd.conf file on
atm.geo.nsf.gov.

The following is information related to NIMAGE ingestion that I sent along
to Dave Fitzgerald of Millersville.

  If you don't have the bandwidth, please don't continue to try and
  ingest the products.  The current design of the LDM is such that
  a site with a slow/bad connection can impact other sites being fed
  from the same server.  This is a known limitation to the LDM design
  and is being investigated.
  
  (re: pqact.conf entry for NIMAGE feed)
  
  pqact.conf entries for the PNG compressed NIMAGE GINI products follow
  the examples for PNG compressed FNEXRAD GINI products.  Examples are
  included in the LDM-McIDAS pages on using the LDM-McIDAS decoders:
  
  LDM-McIDAS
  http://www.unidata.ucar.edu/packages/mcidas/mcidd/ldm-mcidas.html
  
    Using LDM-McIDAS Decoders
    http://www.unidata.ucar.edu/packages/mcidas/mcidd/ldm-mcidas-use.html
  
  An example taken from a machine here at the UPC is:
  
  NIMAGE  ^sat/ch[0-9]/.*/(.*)/([12][0-9])([0-9][0-9])([01][0-9])([0-3][0-9]) 
([0-2][0-9])([0-5][0-9])/(.*)/(.*km)/
        PIPE    -close
        decoders/pngg2gini -vl logs/ldm-mcidas.log
        data/gempak/nport/\8/\9/\1/\1_\2\3\4\5_\6\7
  
  This entry decodes all NIMAGE images using product header information.
  If you limit what you are asking for in your ldmd.conf request line, you
  could use this entry as is, or, more likely, change things like directory
  locations.
  
  Note: pngg2gini is an ldm-mcidas decoder that became available in LDM-McIDAS
  version 7.8.0.  Binary LDM-McIDAS distributions are available; please
  look through the LDM-McIDAS web pages for further information.
  
  As to the ldmd.conf entry, you can determine this one yourself pretty easily:
  
  notifyme -vxl- -f NIMAGE -o 3600 -h atm.geo.nsf.gov
  
  This will show you everything in NIMAGE.  From this listing you will notice
  that part of the product header identifies the images as either coming
  from GOES-8 or GOES-10.  You can then refine your listing to just those
  from GOES-8 as follows:
  
  notifyme -vxl- -f NIMAGE -o 3600 -p GOES-8 -h atm.geo.nsf.gov
  
  From this listing, you will see that another portion of the header calls
  out the band (wavelength channel) of the image: VIS, 3.9, WV, IR, 12.0.
  You would then use this informatio to construct a request for just the
  products you want.  A _very_ quick stab at your wish list is:
  
  request NIMAGE "GOES-8/(VIS|IR)/.*/EAST-CONUS" atm.geo.nsf.gov
  
  It is easiest to check your request pattern using notifme:
  
  notifyme -vxl- -f NIMAGE -o 3600 -h atm.geo.nsf.gov -p 
"GOES-8/(VIS|IR)/.*/EAST-CONUS"
  
  Refine the pattern until you get exactly what you want, and then add it
  to your ~ldm/etc/ldmd.conf file.
  
  One last word of caution: it is your responsibility to scour the data
  files ingested by the LDM.  We provide example C shell scripts that can
  be adapted for this use in the pub/ldm5/scour directory of anonymous
  FTP on our FTP server, ftp.unidata.ucar.edu.  For your purposes, I
  would grab and use prune_gini.csh.  The operative things that will
  need to be changed in this script are:
  
  PATH    - must have the path information to find itself
  KEEP    - how many of each kind of image to keep 
  areadir - the directory to start from when doing scours (scours down
            limbs of the tree)
  
  The script works by calling itself recursively.  It should be run from
  cron routinely.  The frequency at which the script should be run will
  be a function of how many files you can store on disk at any time.
  
  My parting comment/request is that if you find you can ingest a good
  fraction of the NIMAGE data, I would hope that you would agree to serve
  this data through ADDE to other Unidata sites.

Please let me know if you run into any snags.

Tom