[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

20190616: reminder: major upcoming change to IDD NIMAGE datastream

2nd Notice:

Users of the IDD NIMAGE data feed:

We will be transitioning the IDD NIMAGE datastream to one that contains
all of the GOES-16 and GOES-17 Level 2 image products that are currently
being being sent in the IDD NOTHER and HDS datastreams.

When:

It is our intention to begin providing a reconstituted NIMAGE feed
from our top level IDD relay clusters (idd.unidata.ucar.edu and
iddb.unidata.ucar.edu) this coming Wednesday, June 19.  We will postpone
turning on this feed if/when end-users indicate that they need more time
to prepare for the change.

Impacts of the additional imagery being added to NIMAGE:

- the volume of data available in the NIMAGE feed will increase from
  an average of about 100 MB/hr to an average that is close to 7 GB/hr,
  a 70-fold increase!

- sites not wishing to receive the new imagery that will be added to
  the NIMAGE feed should make sure that they are not REQUESTing the
  new data in their LDM configuration file REQUEST line(s)

  The most typical LDM configuration file REQUEST is for everything from
  the NIMAGE feed. For example:

  REQUEST NIMAGE ".*" <upsteam feed host>

  In order to _not_ receive the new imagery, one should change from use
  of this type of blanket REQUEST to one that is specific to the GOES-15
  imagery products that have long been distributed by the NWS in the
  NOAAPort SBN in GINI format and relayed in the IDD in the NIMAGE feed
  to one that is specific to just the GINI images.  For example:

  REQUEST NIMAGE "satz" <upstream feed host>

  NB: After making any changes in the LDM configuration file,
  ~ldm/etc/ldmd.conf, one must stop and restart the LDM for the change(s)
  to take effect.  For example:

  <as user 'ldm'>

  -- modify the REQUEST for NIMAGE in ~ldm/etc/ldmd.conf

  ldmadmin stop
  ldmadmin start
  
- sites that do not know if their network bandwidth will be sufficient
  to receive the new imagery in addition to everything they are already
  REQUESTing are advised to revert to the targeted REQUEST above (using
  extended regular expression "satz") and then stop/restart their LDM

- given the 70-fold increase in data volume in the NIMAGE feed, it is
  _very_ likely that sites will need to increase the size of their 
  LDM queues

  For sites that wish to receive all of the new imagery that will be
  added to the NIMAGE feed, we suggest _increasing_ the size of the
  local queue by 5-7 GB.

  Sites that choose to not REQUEST the new imagery will only need
  to modify the NIMAGE REQUEST line in their LDM configuration file
  and then stop and restart their LDM.

- new processing action(s) will be need for the new imagery

  The page:

  https://www.unidata.ucar.edu/data/nimage.html

  contains links to suggested LDM pattern-action file actions that
  can be used for guidance for such actions.

  GEMPAK users:  we have been reminded that GEMPAK only supports
  pathnames that are less than or equal wo 108 characters in
  length.  GEMPAK sites that will be REQUESTing the new imagery
  should ensure that the pathnames for the files saved do not
  exceed this limit

  Sites that have specific questions about how they should modify
  their LDM REQUEST for NIMAGE and/or pattern-action file action(s)
  to process the new imagery are advised to contact Unidata IDD
  Support by sending an inquiry to:

  address@hidden

- all users are advised to update their display and analysis software
  the the latest version that is available from Unidata

  NB: this is especially true for GEMPAK users as only the most recent
  release of Unidata GEMPAK contains support for the netCDF4 format
  the new NIMAGES are in.

Again, we will put off turning on the additions to the NIMAGE feed if
we are advised that more time is needed to implement changes at end-user
sites.  We have _not_ received any postponement requests since we first
announced the change to the NIMAGE feed on June 4.

The original announcement for the upcoming changes to the NIMAGE feed
are available in a Unidata Blog post:

https://www.unidata.ucar.edu/blogs/news/entry/service-update-changes-to-the

Please send any comments/questions/concerns to Unidata Datastream
support at:

address@hidden

Best regards,

Unidata User Support
--