[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[CONDUIT #IJW-881590]: NDFD Data Issues

Hi Nathan,

Sorry for the slow reply...

re:
> A while back we had some issues regarding NDFD data received through CONDUIT 
> in the
> LDM.

Correct.  The issue found at that time was caused by a number of TVA machines
all of which were/are located behind a NAT making redundant REQUESTs for the
NDFD data in CONDUIT.  We were under the impression that this problem had
been fixed.

re:
> While you did point me to the correct spreadsheet<http://graphical.weather.gov
> /docs/datamanagement.php> online defining the amount of timesteps I should 
> see per
> parameter, the files we receive still do not match those desciptions.

The spreadsheet shows the NDFD fields that should be available in some way from
NOAA.  This does not mean that all of those fields are guaranteed to be in
the IDD CONDUIT feed.

re:
> I've summarized
> the data in the attached grids below - While we are expecting 7 day 
> forecasts, we rarely
> ever exceed 3 days .

This may be caused by a subset of the data products being added to CONDUIT by
the NOAA data flow folks.

> We simply want the latest 7-day forecast per parameter (see list below) 
> through the
> LDM.

If the data you are looking for is included in the CONDUIT feed, and if there 
are
no problems in you receiving the data via an LDM feed, then you should be 
getting
all of the fields that are being put in CONDUIT by the NOAA data flow folks.

> I'm still not sure if this is an issue on the LDM side (during requests) or an
> issue with actual data being hosted.

Since TVA is not reporting real time stats back to us, we can not estimate
if you are receiving all of the NDFD products that are being put in CONDUIT
by the NOAA data flow folks.  CONDUIT does contain NDFD manifest files that
are created on the NOAA data flow machine(s) where the NDFD products are 
inserted into CONDUIT.  The manifest files have LDM/IDD ProductIDs that look
like:

.status.data2/ndfd/LFUZ98_KWNS_020053.grib2
.status.data2/ndfd/LFUZ98_KWNS_020552.grib2
.status.data2/ndfd/LFUZ98_KWNS_021252.grib2
.status.data2/ndfd/LFUZ98_KWNS_021623.grib2
.status.data2/ndfd/LFUZ98_KWNS_021950.grib2
.status.data2/ndfd/LFUZ98_KWNS_022153.grib2
.status.data2/ndfd/LKUZ98_KWNS_020552.grib2
.status.data2/ndfd/LKUZ98_KWNS_020752.grib2
.status.data2/ndfd/LKUZ98_KWNS_020911.grib2
.status.data2/ndfd/LKUZ98_KWNS_021752.grib2
.status.data2/ndfd/LKUZ98_KWNS_022153.grib2
.status.data2/ndfd/LLUZ98_KWNS_020552.grib2
.status.data2/ndfd/LLUZ98_KWNS_020752.grib2
.status.data2/ndfd/LLUZ98_KWNS_021752.grib2
.status.data2/ndfd/LLUZ98_KWNS_022153.grib2
.status.data2/ndfd/LMUZ98_KWNS_020053.grib2
.status.data2/ndfd/LMUZ98_KWNS_020552.grib2
.status.data2/ndfd/LMUZ98_KWNS_020752.grib2
.status.data2/ndfd/LMUZ98_KWNS_021252.grib2
.status.data2/ndfd/LMUZ98_KWNS_021752.grib2
.status.data2/ndfd/LMUZ98_KWNS_021951.grib2
.status.data2/ndfd/LMUZ98_KWNS_022153.grib2
.status.data2/ndfd/YAUZ97_KWBN_020532.grib2
.status.data2/ndfd/YAUZ97_KWBN_021132.grib2
.status.data2/ndfd/YAUZ97_KWBN_021732.grib2
.status.data2/ndfd/YAUZ97_KWBN_022133.grib2
.status.data2/ndfd/YAUZ97_KWBN_022333.grib2
.status.data2/ndfd/YAUZ98_KWBN_020017.grib2
.status.data2/ndfd/YAUZ98_KWBN_020053.grib2
.status.data2/ndfd/YAUZ98_KWBN_020117.grib2
.status.data2/ndfd/YAUZ98_KWBN_020152.grib2
.status.data2/ndfd/YAUZ98_KWBN_020223.grib2
.status.data2/ndfd/YAUZ98_KWBN_020253.grib2
.status.data2/ndfd/YAUZ98_KWBN_020321.grib2
.status.data2/ndfd/YAUZ98_KWBN_020352.grib2
.status.data2/ndfd/YAUZ98_KWBN_020433.grib2
.status.data2/ndfd/YAUZ98_KWBN_020450.grib2
.status.data2/ndfd/YAUZ98_KWBN_020525.grib2
.status.data2/ndfd/YAUZ98_KWBN_020552.grib2
.status.data2/ndfd/YAUZ98_KWBN_020651.grib2
 ...

These ASCII text manifest files contain a listing of NDFD products
successfully inserted into the LDM queue on the machine(s) where
the products are first put into CONDUIT.  In order to check to
see if you are receiving all of the NDFD products in CONDUIT, you
should:

- make sure that your CONDUIT REQUEST(s) include asking for the
  manifest files

- create a pattern-action file action to FILE the manifest files

- compare the listing of NDFD products inserted into the LDM queue
  by NOAA

  If the list of products you are receiving matches the list of products
  being inserted into the NOAA LDM queue, then it means that the list
  of NDFD products being made available in CONDUIT does not include all
  that you are looking for/need.  If the comparison shows a difference
  in products inserted and products received at TVA, then there is some
  problem in the relay and/or receipt of the products.

I hope that this helps!

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: IJW-881590
Department: Support CONDUIT
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.