Hi Tom,
Yes, you answered my questions! The last one was the one I was most
concerned about, but I will have to wait a while for that to happen.
Thank you so much, I appreciate it!
Gilbert
On Wed, Jun 5, 2019 at 12:10 PM Unidata User Support <
support@xxxxxxxxxxxxxxxx> wrote:
> Hi Gilbert and other ldm-users,
>
> re:
> >I have questions about this transition, and I'm confused about a few
> >things, so hopefully this can be cleared up.
>
> OK, ready...
>
> re:
> >First: Does this affect anything on the SATELLITE/nee DIFAX feed?
>
> No.
>
> re:
> >That is
> >strictly for data coming across the GRB feed and will stay there, correct?
>
> Correct.
>
> re:
> >Second: What about those who have their own ingest dishes? If they use IDD
> >as a backup, will they need to have a duplicate feed coming across: one
> >from their dish on NOTHER/EXP, and one from UNIDATA? If so...
>
> The NOAAPort ingest package that is bundled with the LDM has not been
> changed. The contents of the NOTHER and HDS datastreams have not been
> altered. We are stitching together the ABI L2 image tiles and stripping
> off the NOAAPort headers and footers on a non-ingest machine.
>
> re:
> >Third: Does this require an LDM or LDM database upgrade, the latter of
> >which can be done automagically in the latest version, 6.13.11?
>
> No.
>
> re:
> >Fourth: Can the LDM switch the NOAAport-ingested GOES products over to
> >NIMAGE as well to avoid duplicating the UNIDATA feed on the current
> >feedtypes IF a site has their own dish/receiver?
>
> I am not sure that I fully understand this question.
>
> For the time being, the NOAAPort ingest capability that is packaged with
> the LDM will continue to classify the ABI L2 tiles and some of the other
> L2 products in the NOTHER datastream. The other L2 products that are in
> the HDS feed will continue to be in the HDS feed. We realize that this
> split between two different feeds for the unaltered, NOAAPort-delivered
> products should be addressed, but this will have to wait as there are
> lots of more critical items that need attention.
>
> re:
> >Thank you for the clarification in advance,
>
> No worries.
>
> Cheers,
>
> Tom
>
> >On Tue, Jun 4, 2019 at 12:51 PM Unidata User Support <
> >support@xxxxxxxxxxxxxxxx> wrote:
> >
> >> IMPORTANT notice of major changes to the IDD NIMAGE datastream; please
> >> read.
> >>
> >> First, we apologies for cross posting this announcement to multiple
> lists,
> >> but we want to make sure that all users that will/could be affected
> >> are notified.
> >>
> >> Second, we realize that the following message is long, but please read
> >> it all since it contains important information that will affect sites
> >> that are or will be receiving imagery products in the IDD NIMAGE feed.
> >>
> >> What:
> >>
> >> We will be transitioning the IDD NIMAGE datastream to one that
> >> contains all of the GOES-16 and GOES-17 Level 2 image products
> >> that are currently being being sent in the IDD NOTHER and HDS
> >> datastreams.
> >>
> >> When:
> >>
> >> It is our desire to begin distributing a reconstituted NIMAGE feed
> >> from our top level IDD relay clusters (idd.unidata.ucar.edu and
> >> iddb.unidata.ucar.edu) in the mid-June time frame. We will postpone
> >> turning on this feed if/when end-users indicate that they need more
> >> time to prepare for the change.
> >>
> >> Impact:
> >>
> >> The volume of data available in the NIMAGE feed will increase from
> >> an average of about 100 MB/hr to an average that is close to 7 GB/hr,
> >> a 70-fold increase!
> >>
> >> Reason for the change:
> >>
> >> NOAA has indicated that operation of GOES-15 will likely cease in
> >> early-mid July:
> >>
> >> https://www.goes-r.gov/users/transitionToOperations17.html
> >>
> >> Since GOES-East imagery in GINI format was removed from the NOAAPort
> >> SBN after GOES-13 was replaced as GOES-East by GOES-16 and then
> >> decommissioned, we believe that it is most likely that all GINI imagery
> >> will cease to be distributed in NOAAPort in the mid-July time frame.
> >> Since the NIMAGE IDD data feed has been populated solely by the GINI
> >> imagery distributed in NOAAPort, this feed will cease to contain any
> data.
> >>
> >> Currently, NOAAPort contains GOES-16 and GOES-17 ABI (Advanced Baseline
> >> Imager) Level 2 product tiles (pieces of full image scenes), and those
> >> tiles are being distributed in the IDD NOTHER data feed. To date, these
> >> image tiles would need to be stitched together by end-users before they
> >> were fully usable in display and analysis applications available from
> >> Unidata (e.g., AWIPS, GEMPAK, IDV, McIDAS and MetPy (this is not
> >> strictly true for AWIPS)).
> >>
> >> In addition to the ABI Level 2 product tiles, 19 other Level 2 products
> >> are also being distributed in NOAAPort, and we have been making those
> >> products available in the IDD NOTHER and HDS data feeds since their
> >> addition to NOAAPort. Like the ABI Level 2 product tiles, the other
> >> Level 2 products need some processing before they are usable in end-user
> >> applications - they need to have the NOAAPort header and footer stripped
> >> off to leave the underlying netCDF4 files.
> >>
> >> We believe that it will be much easier for end-users, who get their
> >> NOAAPort-delivered Level 2 images and products via the IDD, if we
> >> reconstitute full images scenes from ABI tiles and strip NOAAPort
> >> broadcast headers and footers here in Unidata and then distributed the
> >> resulting netCDF4 images/products in a reconstituted NIMAGE IDD data
> >> feed. Also, reconstituting the NIMAGE feed will allow us to make other,
> >> value-added GOES-16/17 products available to the community. The first
> >> of these products is GLM imagery/grids being produced at Texas Tech by
> >> community member Eric Bruning.
> >>
> >> Information on the imagery and products that are currently contained
> >> in and those that will be added to the reconstituted NIMAGE feed can be
> >> found in:
> >>
> >> https://www.unidata.ucar.edu/data/nimage.html
> >>
> >> We have been using all of the products in this new incarnation of the
> >> NIMAGE feed internally for the past few months to test their viability
> >> in the various application packages that we provide and support. We are
> >> satisfied that transition to use of these products by end users will be
> >> straightforward, but it will require that sites install and transition
> >> to use of the latest of each package that we make available.
> >>
> >> Important things to note:
> >>
> >> - the reconstitution of the NIMAGE feed comes at a cost
> >>
> >> The volume of data available will increase from an average of about
> >> 100 MB/hr to an average nearing 7 GB/hr, a 70-fold increase!
> >>
> >> Sites that do not want to receive any/all of the processed GOES-16/17
> >> imagery/products in the new NIMAGE feed need to make sure that their
> >> LDM configuration file REQUEST(s) do not include products that they
> >> do not want.
> >>
> >> The current GOES-15 GINI imagery that is being distributed in NOAAPort
> >> all have LDM/IDD Product IDs that begin with 'satz'. Sites requesting
> >> all of the current NIMAGE feed via a REQUEST that looks like:
> >>
> >> REQUEST NIMAGE ".*" idd.unidata.ucar.edu
> >>
> >> and that either do not want or can not handle the increased data
> volume
> >> should immediately change their REQUEST to something like:
> >>
> >> REQUEST NIMAGE "satz" idd.unidata.ucar.edu
> >>
> >> Sites that do want to receive all of the imagery that will be
> available
> >> in the reconstituted NIMAGE feed and whose network connection can
> handle
> >> the increased volume can leave their REQUEST(s) as is, but this
> decision
> >> should not be taken lightly given the 70-fold increase in data volume!
> >>
> >> Sites wanting to receive some of the data from the reconstituted
> >> NIMAGE feed are advised to read through the NIMAGE web page listed
> >> above to become familiar with how the LDM/IDD Product IDs will look
> >> and then re-fashion their feed REQUEST(s) to get the set of products
> >> desired.
> >>
> >> - sites that want to use the products that will be available in the
> >> reconstituted NIMAGE feed should immediately install and switch to
> >> using the most recent releases of their Unidata-supplied display and
> >> analysis applications
> >>
> >> This is especially true for GEMPAK users as imagery in netCDF4 format
> >> was not supported in Unidata GEMPAK prior to the current release.
> >>
> >> - the data volume that will be available in the reconstituted NIMAGE
> >> feed is actually 20% less than the volume of the sum of the products
> >> being delivered in NOAAPort and distributed in the NOTHER and HDS
> >> datastreams
> >>
> >> The reason for this is twofold:
> >>
> >> - NOAAPort broadcast headers and footers have been removed
> >> resulting in a minor reduction of volume
> >>
> >> - netCDF4 compression has been implemented/implemented better in
> >> the products
> >>
> >> Recap:
> >>
> >> The reconstituted IDD NIMAGE datastream will contain:
> >>
> >> - full image scenes that have been created by stitching together
> >> the ABI tiles sent in the NOAAPort SBN and distributed in the
> >> IDD NOTHER feed
> >>
> >> - all other Level 2 products being distributed in the NOAAPort SBN
> >> in the IDD NOTHER and HDS feeds
> >>
> >> - value added GLM image/grid products created by Eric Bruning
> >> of Texas Tech University
> >>
> >> - all products are in netCDF-4 format
> >>
> >> - all products are in the GOES-R mission standard Fixed Grid
> >> projection
> >>
> >> - the names for all products (contained in the LDM/IDD Product IDs)
> >> follow the GOES-R mission standard naming convention as detailed
> >> in:
> >>
> >> GOES-R SERIES PRODUCT DEFINITION AND USERS=E2=80=99 GUIDE (PUG)
> >> https://www.goes-r.gov/users/docs/PUG-GRB-vol4.pdf
> >>
> >> In all cases, the LDM/IDD Product IDs will consist of fully qualified
> >> path names for products, and the file name portion of the fully
> >> qualified names follow GOES-R mission standards for naming. The NIMAGE
> >> web page referenced above lists example Product IDs for each product.
> >>
> >> Please send any/all questions/comments about the upcoming change to the
> >> IDD NIMAGE feed to:
> >>
> >> support-datastream@xxxxxxxxxxxxxxxx
> >>
> >> Best regards,
> >>
> >> Unidata User Support
>
> --
> ********************************************************************** <
> Unidata User Support UCAR Unidata Program <
> (303)497-8643 P.O. Box 3000 <
> support@xxxxxxxxxxxxxxxx Boulder, CO 80307 <
> ---------------------------------------------------------------------- <
> Unidata Web Support http://www.unidata.ucar.edu/support <
> ---------------------------------------------------------------------- <
> NOTE: All email exchanges with Unidata User Support are recorded in the
> Unidata inquiry tracking system and then made publicly available
> through the web. If you do not want to have your interactions made
> available in this way, you must let us know in each email you send to us.
>
--
----
Gilbert Sebenste
Consulting Meteorologist
AllisonHouse, LLC
