[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[NOAAPORT #LQO-967392]: [noaaport] Lack of GOES16 L2 products from HDS feed
- Subject: [NOAAPORT #LQO-967392]: [noaaport] Lack of GOES16 L2 products from HDS feed
- Date: Tue, 03 Apr 2018 12:19:42 -0600
Hi Mike,
I drove your post to the noaaport and ldm-users email list into our inquiry
tracking system since we are not seeing the same thing as you.
re:
> I'm noticing that GOES-16 L2 Derived products appear to have largely gone
> missing from the HDS feed, mostly over NOAAPort, and I was wondering if
> anyone else had seen the same thing.
>
> Based on some RTstats
>
> http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_vol_nc?HDS+wxgoes.cod.edu
>
> we've been getting a fairly regular amount of HDS data into our server,
> here seen mostly via NOAAPort, up until about a day ago. Since then, very
> little if anything has been seen over NP, and only intermittent/irregular
> data is seen via Unidata feeds. On this server, I'm limiting the HDS
> feed to "IXT.99 KNES" and "IXT[WXY]01 KNES" to just get the L2 products;
> those are based off the various change notices I've seen over time,
> and they've worked till now.
It is a bit hard to compare the real-time stats from wxgoes.cod.edu since
you are requesting a portion of the HDS datastream, but a direct comparison
can be made for your NOAAPort ingest machines and those that are feeding into
the IDD. Volume plots for the full HDS content on the 5 NOAAPort ingest
machines that are feeding into the IDD show more or less the same volume
being ingested in the HDS feed for the past 3 days:
noaaport1.cod.edu HDS volume:
http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_vol_nc?HDS+noaaport1.cod.edu
noaaport2.cod.edu HDS volume:
http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_vol_nc?HDS+noaaport2.cod.edu
mistral.srcc.lsu.edu HDS volume:
http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_vol_nc?HDS+mistral.srcc.lsu.edu
np1.ssec.wisc.edu HDS volume:
http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_vol_nc?HDS+np1.ssec.wisc.edu
np2.ssec.wisc.edu HDS volume:
http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_vol_nc?HDS+np2.ssec.wisc.edu
chico.unidata.ucar.edu HDS volume:
http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_vol_nc?HDS+chico.unidata.ucar.edu
leno.unidata.ucar.edu HDS volume:
http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_vol_nc?HDS+leno.unidata.ucar.edu
The same volume is being made available on all of the real server backend
machines for the two toplevel IDD relay clusters that we operate here in
UCAR. For example:
cluster0.unidata.ucar.edu HDS volume (a backend machine for
iddb.unidata.ucar.edu):
http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_vol_nc?HDS+cluster0.unidata.ucar.edu
uni14.unidata.ucar.edu HDS volume (a backend machine for idd.unidata.ucar.edu):
http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_vol_nc?HDS+uni14.unidata.ucar.edu
re:
> Here is the RTstats graph from our NP server
>
> http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_vol_nc?HDS+noaaport1.cod.edu
>
> showing a substantial drop off in volume. This server should be getting
> everything from the HDS feed.
Have you checked the ingest logs on your NOAAPort machines to see if there
has been a substantial increase in the number of errors being reported?
re:
> I went through the typical troubleshooting (e.g. del/remake queue, restart
> LDM) but I'm still not seeing those products via NP.
On which machine(s) did you do the various troubleshooting steps? If it
was only on wxgoes.cod.edu, you are probably looking at the wrong machine(s).
re:
> Did something change a day ago, or could this be something else on my end?
We are not seeing the kind of drop in HDS volume on our end that you
are seeing on yours.
NB: I have no explanation for why you wouldn't be receiving all of the
HDS products via an IDD feed! Did anything change on your end recently
(e.g., change in the ldmd.conf REQUEST, network bandwidth restrictions,
etc.)?
If you want to check on the L2 products we have ingested and are making
available via ADDE, you can "point" (DATALOC) at either of the following
machines:
lead.unidata.ucar.edu
atm.ucar.edu
The ADDE group is NPGOESR, and the various descriptors can be gleened
from the listing produced by DSINFO:
DATALOC ADD NPGOESR atm.ucar.edu
DSINFO IMAGE NPGOESR
Dataset Names of Type: IMAGE in Group: NPGOESR
Name NumPos Content
------------ ------ --------------------------------------
AODCN 99999 GOES-16 Level2 CONUS Aerosol Optical Depth Product
AODFD 99999 GOES-16 Level2 Full Disk Aerosol Optical Depth Product
AODM1 99999 GOES-16 Level2 Mesoscale-1 Aerosol Optical Depth Product
AODM2 99999 GOES-16 Level2 Mesoscale-2 Aerosol Detection Optical Dep
ARSLCN 99999 GOES-16 Level2 CONUS Aerosol Detection Products
ARSLFD 99999 GOES-16 Level2 Full Disk Aerosol Detection Products
ARSLM1 99999 GOES-16 Level2 Mesoscale-1 Aerosol Detection Products
ARSLM2 99999 GOES-16 Level2 Mesoscale-2 Aerosol Detection Products
CMSKCN 99999 GOES-16 Level2 CONUS Cloud Mask Product
CMSKFD 99999 GOES-16 Level2 Full Disk Cloud Mask Product
CMSKM1 99999 GOES-16 Level2 Mesoscale-1 Cloud Mask Product
CMSKM2 99999 GOES-16 Level2 Mesoscale-2 Cloud Mask Product
CODCN 99999 GOES-16 Level2 CONUS Cloud Optical Depth Product
CODFD 99999 GOES-16 Level2 Full Disk Cloud Optical Depth Product
CODM1 99999 GOES-16 Level2 Mesoscale-1 Cloud Optical Depth Product
CODM2 99999 GOES-16 Level2 Mesoscale-2 Cloud Optical Depth Product
CONUSC01 99999 GOES-16 CONUS 0.47 um VIS aerosol-over-land
CONUSC02 99999 GOES-16 CONUS 0.64 um VIS clouds fog/insol/winds
CONUSC03 99999 GOES-16 CONUS 0.86 um Near IR veg/burn scar/aerosol
CONUSC04 99999 GOES-16 CONUS 1.37 um Near IR cirrus cloud
CONUSC05 99999 GOES-16 CONUS 1.61 um Near IR cloud phase/snow
CONUSC06 99999 GOES-16 CONUS 2.24 um Near IR land/cloud vege/snow
CONUSC07 99999 GOES-16 CONUS 3.89 um IR Sfc/cloud/fog/fire/winds
CONUSC08 99999 GOES-16 CONUS 6.17 um IR High-level WV/winds/rainfall
CONUSC09 99999 GOES-16 CONUS 6.93 um IR Mid-level WV/winds/rainfal
CONUSC10 99999 GOES-16 CONUS 7.34 um IR Lower-level WV/winds & SO2
CONUSC11 99999 GOES-16 CONUS 8.44 um IR Total WV cloud phase/dust
CONUSC12 99999 GOES-16 CONUS 9.61 um IR Total ozone/turbulence/wind
CONUSC13 99999 GOES-16 CONUS 10.3 um IR Surface & cloud
CONUSC14 99999 GOES-16 CONUS 11.2 um IR Imagery/SST/clouds/rainfall
CONUSC15 99999 GOES-16 CONUS 12.3 um IR Total water/ash and SST
CONUSC16 99999 GOES-16 CONUS 13.3 um IR Air temp/cloud hgt and amt
CPSCN 99999 GOES-16 Level2 CONUS Cloud Particle Size Product
CPSFD 99999 GOES-16 Level2 Full Disk Cloud Particle Size Product
CPSM1 99999 GOES-16 Level2 Mesoscale-1 Cloud Particle Size Product
CPSM2 99999 GOES-16 Level2 Mesoscale-2 Cloud Particle Size Product
CTHCN 99999 GOES-16 Level2 CONUS Cloud Top Height Product
CTHFD 99999 GOES-16 Level2 Full Disk Cloud Top Height Product
CTHM1 99999 GOES-16 Level2 Mesoscale-1 Cloud Top Height Product
CTHM2 99999 GOES-16 Level2 Mesoscale-2 Cloud Top Height Product
CTPHCN 99999 GOES-16 Level2 CONUS Cloud Top Phase Product
CTPHFD 99999 GOES-16 Level2 Full Disk Cloud Top Phase Product
CTPHM1 99999 GOES-16 Level2 Mesoscale-1 Cloud Top Phase Product
CTPHM2 99999 GOES-16 Level2 Mesoscale-2 Cloud Top Phase Product
CTPRCN 99999 GOES-16 Level2 CONUS Cloud Top Pressure Product
CTPRFD 99999 GOES-16 Level2 Full Disk Cloud Top Pressure Product
CTPRM1 99999 GOES-16 Level2 Mesoscale-1 Cloud Top Pressure Product
CTPRM2 99999 GOES-16 Level2 Mesoscale-2 Cloud Top Pressure Product
CTTCN 99999 GOES-16 Level2 CONUS Cloud Top Temperature Product
CTTFD 99999 GOES-16 Level2 Full Disk Cloud Top Temperature Product
CTTM1 99999 GOES-16 Level2 Mesoscale-1 Cloud Top Temperature Product
CTTM2 99999 GOES-16 Level2 Mesoscale-2 Cloud Top Temperature Product
DSICN 99999 GOES-16 Level2 CONUS Derived Stability Indices Products
DSIFD 99999 GOES-16 Level2 Full Disk Derived Stability Indices Produ
DSIM1 99999 GOES-16 Level2 Mesoscale-1 Derived Stability Indices Pro
DSIM2 99999 GOES-16 Level2 Mesoscale-2 Derived Stability Indices Pro
FDC01 99999 GOES-16 FullDisk 0.47 um VIS aerosol-over-land
FDC02 99999 GOES-16 FullDisk 0.64 um VIS clouds fog/insol/winds
FDC03 99999 GOES-16 FullDisk 0.86 um Near IR veg/burn scar/aerosol
FDC04 99999 GOES-16 FullDisk 1.37 um Near IR cirrus cloud
FDC05 99999 GOES-16 FullDisk 1.61 um Near IR cloud phase/snow
FDC06 99999 GOES-16 FullDisk 2.24 um Near IR land/cloud vege/snow
FDC07 99999 GOES-16 FullDisk 3.89 um IR Sfc/cloud/fog/fire/winds
FDC08 99999 GOES-16 FullDisk 6.17 um IR High-level WV/winds/rainfall
FDC09 99999 GOES-16 FullDisk 6.93 um IR Mid-level WV/winds/rainfal
FDC10 99999 GOES-16 FullDisk 7.34 um IR Lower-level WV/winds & SO2
FDC11 99999 GOES-16 FullDisk 8.44 um IR Total WV cloud phase/dust
FDC12 99999 GOES-16 FullDisk 9.61 um IR Total ozone/turbulence/wind
FDC13 99999 GOES-16 FullDisk 10.3 um IR Surface & cloud
FDC14 99999 GOES-16 FullDisk 11.2 um IR Imagery/SST/clouds/rainfall
FDC15 99999 GOES-16 FullDisk 12.3 um IR Total water/ash and SST
FDC16 99999 GOES-16 FullDisk 13.3 um IR Air temp/cloud hgt and amt
FHSCN 99999 GOES-16 Level2 CONUS Fire Hot Spot Products
FHSFD 99999 GOES-16 Level2 Full Disk Fire Hot Spot Products
FHSM1 99999 GOES-16 Level2 Mesoscale-1 Fire Hot Spot Products
FHSM2 99999 GOES-16 Level2 Mesoscale-2 Fire Hot Spot Products
LSTCN 99999 GOES-16 Level2 CONUS Land Surface Temperature Product
LSTFD 99999 GOES-16 Level2 Full Disk Land Surface Temperature Produc
LSTM1 99999 GOES-16 Level2 Mesoscale-1 Land Surface Temperature Prod
LSTM2 99999 GOES-16 Level2 Mesoscale-2 Land Surface Temperature Prod
M1C01 99999 GOES-16 Meso1 0.47 um VIS aerosol-over-land
M1C02 99999 GOES-16 Meso1 0.64 um VIS clouds fog/insol/winds
M1C03 99999 GOES-16 Meso1 0.86 um Near IR veg/burn scar/aerosol
M1C04 99999 GOES-16 Meso1 1.37 um Near IR cirrus cloud
M1C05 99999 GOES-16 Meso1 1.61 um Near IR cloud phase/snow
M1C06 99999 GOES-16 Meso1 2.24 um Near IR land/cloud vege/snow
M1C07 99999 GOES-16 Meso1 3.89 um IR Sfc/cloud/fog/fire/winds
M1C08 99999 GOES-16 Meso1 6.17 um IR High-level WV/winds/rainfall
M1C09 99999 GOES-16 Meso1 6.93 um IR Mid-level WV/winds/rainfal
M1C10 99999 GOES-16 Meso1 7.34 um IR Lower-level WV/winds & SO2
M1C11 99999 GOES-16 Meso1 8.44 um IR Total WV cloud phase/dust
M1C12 99999 GOES-16 Meso1 9.61 um IR Total ozone/turbulence/wind
M1C13 99999 GOES-16 Meso1 10.3 um IR Surface & cloud
M1C14 99999 GOES-16 Meso1 11.2 um IR Imagery/SST/clouds/rainfall
M1C15 99999 GOES-16 Meso1 12.3 um IR Total water/ash and SST
M1C16 99999 GOES-16 Meso1 13.3 um IR Air temp/cloud hgt and amt
M2C01 99999 GOES-16 Meso2 0.47 um VIS aerosol-over-land
M2C02 99999 GOES-16 Meso2 0.64 um VIS clouds fog/insol/winds
M2C03 99999 GOES-16 Meso2 0.86 um Near IR veg/burn scar/aerosol
M2C04 99999 GOES-16 Meso2 1.37 um Near IR cirrus cloud
M2C05 99999 GOES-16 Meso2 1.61 um Near IR cloud phase/snow
M2C06 99999 GOES-16 Meso2 2.24 um Near IR land/cloud vege/snow
M2C07 99999 GOES-16 Meso2 3.89 um IR Sfc/cloud/fog/fire/winds
M2C08 99999 GOES-16 Meso2 6.17 um IR High-level WV/winds/rainfall
M2C09 99999 GOES-16 Meso2 6.93 um IR Mid-level WV/winds/rainfal
M2C10 99999 GOES-16 Meso2 7.34 um IR Lower-level WV/winds & SO2
M2C11 99999 GOES-16 Meso2 8.44 um IR Total WV cloud phase/dust
M2C12 99999 GOES-16 Meso2 9.61 um IR Total ozone/turbulence/wind
M2C13 99999 GOES-16 Meso2 10.3 um IR Surface & cloud
M2C14 99999 GOES-16 Meso2 11.2 um IR Imagery/SST/clouds/rainfall
M2C15 99999 GOES-16 Meso2 12.3 um IR Total water/ash and SST
M2C16 99999 GOES-16 Meso2 13.3 um IR Air temp/cloud hgt and amt
PRC01 99999 GOES-16 PuertoRico 0.47 um VIS aerosol-over-land
PRC02 99999 GOES-16 PuertoRico 0.64 um VIS clouds fog/insol/winds
PRC03 99999 GOES-16 PuertoRico 0.86 um Near IR veg/burn scar/aerosol
PRC04 99999 GOES-16 PuertoRico 1.37 um Near IR cirrus cloud
PRC05 99999 GOES-16 PuertoRico 1.61 um Near IR cloud phase/snow
PRC06 99999 GOES-16 PuertoRico 2.24 um Near IR land/cloud vege/snow
PRC07 99999 GOES-16 PuertoRico 3.89 um IR Sfc/cloud/fog/fire/winds
PRC08 99999 GOES-16 PuertoRico 6.17 um IR High-level WV/winds/rainfa
PRC09 99999 GOES-16 PuertoRico 6.93 um IR Mid-level WV/winds/rainfal
PRC10 99999 GOES-16 PuertoRico 7.34 um IR Lower-level WV/winds & SO2
PRC11 99999 GOES-16 PuertoRico 8.44 um IR Total WV cloud phase/dust
PRC12 99999 GOES-16 PuertoRico 9.61 um IR Total ozone/turbulence/win
PRC13 99999 GOES-16 PuertoRico 10.3 um IR Surface & cloud
PRC14 99999 GOES-16 PuertoRico 11.2 um IR Imagery/SST/clouds/rainfal
PRC15 99999 GOES-16 PuertoRico 12.3 um IR Total water/ash and SST
PRC16 99999 GOES-16 PuertoRico 13.3 um IR Air temp/cloud hgt and amt
TPWCN 99999 GOES-16 Level2 CONUS Total Precipitable Water Product
TPWFD 99999 GOES-16 Level2 Full Disk Total Precipitable Water Produc
TPWM1 99999 GOES-16 Level2 Mesoscale-1 Total Precipitable Water Prod
TPWM2 99999 GOES-16 Level2 Mesoscale-2 Total Precipitable Water Prod
DSINFO -- done
Cheers,
Tom
--
****************************************************************************
Unidata User Support UCAR Unidata Program
(303) 497-8642 P.O. Box 3000
address@hidden Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage http://www.unidata.ucar.edu
****************************************************************************
Ticket Details
===================
Ticket ID: LQO-967392
Department: Support NOAAPORT
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata
inquiry tracking system and then made publicly available through the web. If
you do not want to have your interactions made available in this way, you must
let us know in each email you send to us.
