[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[ldmMcidas #WTM-677712]: McIDAS-XCD data doesn't seem to be filing



Hi Samuel,

I just got back from my lunch meeting...

re:
> Again, thank you for your help.

No worries.  We are glad to help when we can.

> I see you commented out the crontab entry:
> 17 00 * * * /home/ldm/restartldm

Yes.

> Thank you, I was about to remove it myself.

Very good.  I think that restarting the LDM is the wrong approach when 
experiencing
data receipt irregularities.

> I thought that the files being created in 
> /data/ldm/gempak/images/sat/GOES-11/4km/VIS
> stopped after midnight because of a bug, and just by conincedence I would 
> restart lmd
> around 10:00am in the morning, and the files would start being generated 
> again.

Hmm... are you sure that the you were seeing interruption of GOES-West 
(GOES-11) data
and not GOES-12 data?  The reason I ask is that data from GOES-East has been 
observing
a blackout period from more-or-less 3:45 UTC - 6:15 UTC.

> Now I realize it's not a bug that's fixed by re-starting ldm.

Correct.  If there really is a data outage, I would suspect that the upstream 
IDD
site you are requesting the data from is having a problem (or their upstream, 
etc.).

> This brings up a few questions though:
> 
> 1. Most importantly: Should this data be in /data/ldm/gempak, or 
> /data/ldm/mcidas?

McIDAS doesn't care where the data is put.  The serving of the data through 
ADDE simply
has to be adjusted to point at the correct place on disk.

> Isn't crontab is setup for /data/ldm/mcidas?

Yes, but you must remember that I created the pattern-action files and 
associated
crontab entries for sites that are using McIDAS exclusively.  If the site (CUP) 
is
also using GEMPAK (very likely), then the site administrator (you) have to take 
this
into account and adjust:

- the patterns used for processing the data so that you are only processing the
  data once

- the ADDE server setup to correctly find the data on disk however you decide
  to write it

- the crontab entries for data scouring to scour where you are writing the data

Question:

- what is your primary objective?  Do you want a setup that provides the data
  for both GEMPAK and McIDAS?   If yes, then there will need to be a review
  of the ~ldm/etc/ldmd.conf entries and the pqact.conf* actions that are
  run to do the work needed

> 2. Not as important:  Why do the files in 
> /data/ldm/gempak/images/sat/GOES-11/4km/VIS
> stop around midnight, and start again at 10:00am?

They shouldn't, AND I don't believe that they are.  The plot of volume received 
by
shu.cup.edu from the UNIWISC datastream:

http://www.unidata.ucar.edu/cgi-bin/rtstats/iddstats_vol_nc?UNIWISC+shu.cup.edu

is the same as that from the site inserting those data into the IDD:

http://www.unidata.ucar.edu/cgi-bin/rtstats/iddstats_vol_nc?UNIWISC+unidata2.ssec.wisc.edu

And, the data being received from GOES-West shows no interruption:

<as 'mcidas' on my workstation>
cd $MCDATA
dataloc.k ADD RTIMAGES stratus.al.noaa.gov

imglist.k RTIMAGES/GW-IR.ALL
Image file directory listing for:GINIEAST/GE4KIR
 Pos Satellite/         Date       Time      Center   Band(s)
     sensor                                 Lat  Lon
 --- -------------  ------------  --------  ---- ---- ------------
   1  G-12 IMG       9 OCT 07282  00:45:15    41   88 4
   2  G-12 IMG       9 OCT 07282  01:01:44    41   88 4
   3  G-12 IMG       9 OCT 07282  01:15:15    41   88 4
   4  G-12 IMG       9 OCT 07282  01:31:44    41   88 4
   5  G-12 IMG       9 OCT 07282  01:45:15    41   88 4
   6  G-12 IMG       9 OCT 07282  02:01:44    41   88 4
   7  G-12 IMG       9 OCT 07282  02:15:15    41   88 4
   8  G-12 IMG       9 OCT 07282  02:31:44    41   88 4
   9  G-12 IMG       9 OCT 07282  02:45:14    41   88 4
  10  G-12 IMG       9 OCT 07282  03:15:15    41   88 4
  11  G-12 IMG       9 OCT 07282  03:31:44    41   88 4
  12  G-12 IMG       9 OCT 07282  03:45:15    41   88 4
  13  G-11 IMG       9 OCT 07282  04:00:14    41   88 4
  14  G-11 IMG       9 OCT 07282  04:30:13    41   88 4
  15  G-11 IMG       9 OCT 07282  05:00:14    41   88 4
  16  G-11 IMG       9 OCT 07282  05:30:14    41   88 4
  17  G-11 IMG       9 OCT 07282  06:00:13    41   88 4
  18  G-11 IMG       9 OCT 07282  06:30:13    41   88 4
  19  G-12 IMG       9 OCT 07282  07:01:43    41   88 4
  20  G-12 IMG       9 OCT 07282  07:15:14    41   88 4
  21  G-12 IMG       9 OCT 07282  07:31:43    41   88 4
  22  G-12 IMG       9 OCT 07282  07:45:14    41   88 4
  23  G-12 IMG       9 OCT 07282  08:15:13    41   88 4
  24  G-12 IMG       9 OCT 07282  08:45:14    41   88 4
  25  G-12 IMG       9 OCT 07282  09:15:14    41   88 4
  26  G-12 IMG       9 OCT 07282  09:45:14    41   88 4
  27  G-12 IMG       9 OCT 07282  10:15:14    41   88 4
  28  G-12 IMG       9 OCT 07282  10:45:14    41   88 4
  29  G-12 IMG       9 OCT 07282  11:01:43    41   88 4
  30  G-12 IMG       9 OCT 07282  11:15:14    41   88 4
  31  G-12 IMG       9 OCT 07282  11:31:43    41   88 4
  32  G-12 IMG       9 OCT 07282  11:45:14    41   88 4
  33  G-12 IMG       9 OCT 07282  12:15:14    41   88 4
  34  G-12 IMG       9 OCT 07282  12:45:14    41   88 4
  35  G-12 IMG       9 OCT 07282  13:01:43    41   88 4
  36  G-12 IMG       9 OCT 07282  13:15:14    41   88 4
  37  G-12 IMG       9 OCT 07282  13:31:43    41   88 4
  38  G-12 IMG       9 OCT 07282  13:45:14    41   88 4
  39  G-12 IMG       9 OCT 07282  14:01:45    41   88 4
  40  G-12 IMG       9 OCT 07282  14:15:13    41   88 4
  41  G-12 IMG       9 OCT 07282  14:31:44    41   88 4
  42  G-12 IMG       9 OCT 07282  14:45:14    41   88 4
  43  G-12 IMG       9 OCT 07282  15:15:14    41   88 4
  44  G-12 IMG       9 OCT 07282  15:31:43    41   88 4
  45  G-12 IMG       9 OCT 07282  15:45:14    41   88 4
  46  G-12 IMG       9 OCT 07282  16:01:43    41   88 4
  47  G-12 IMG       9 OCT 07282  16:15:14    41   88 4
  48  G-12 IMG       9 OCT 07282  16:31:43    41   88 4
  49  G-12 IMG       9 OCT 07282  16:45:14    41   88 4
  50  G-12 IMG       9 OCT 07282  17:01:43    41   88 4
  51  G-12 IMG       9 OCT 07282  17:15:15    41   88 4
  52  G-12 IMG       9 OCT 07282  17:31:44    41   88 4
  53  G-12 IMG       9 OCT 07282  17:45:14    41   88 4
  54  G-12 IMG       9 OCT 07282  18:15:10    41   88 4
  55  G-12 IMG       9 OCT 07282  18:45:10    41   88 4
  56  G-12 IMG       9 OCT 07282  19:01:44    41   88 4
  57  G-12 IMG       9 OCT 07282  19:15:15    41   88 4
  58  G-12 IMG       9 OCT 07282  19:31:44    41   88 4
  59  G-12 IMG       9 OCT 07282  19:45:15    41   88 4
  60  G-12 IMG       9 OCT 07282  20:01:44    41   88 4
  61  G-12 IMG       9 OCT 07282  20:15:15    41   88 4
  62  G-12 IMG       9 OCT 07282  20:31:44    41   88 4
  63  G-12 IMG       9 OCT 07282  20:45:14    41   88 4
  64  G-12 IMG       9 OCT 07282  21:15:15    41   88 4
  65  G-12 IMG       9 OCT 07282  21:31:44    41   88 4
  66  G-12 IMG       9 OCT 07282  21:45:15    41   88 4
  67  G-12 IMG       9 OCT 07282  22:01:44    41   88 4
  68  G-12 IMG       9 OCT 07282  22:15:15    41   88 4
  69  G-12 IMG       9 OCT 07282  22:31:44    41   88 4
  70  G-12 IMG       9 OCT 07282  22:45:15    41   88 4
  71  G-12 IMG       9 OCT 07282  23:01:44    41   88 4
  72  G-12 IMG       9 OCT 07282  23:15:15    41   88 4
  73  G-12 IMG       9 OCT 07282  23:31:46    41   88 4
  74  G-12 IMG       9 OCT 07282  23:45:14    41   88 4
  75  G-12 IMG      10 OCT 07283  00:15:15    41   88 4
  76  G-12 IMG      10 OCT 07283  00:45:15    41   88 4
  77  G-12 IMG      10 OCT 07283  01:01:44    41   88 4
  78  G-12 IMG      10 OCT 07283  01:15:15    41   88 4
  79  G-12 IMG      10 OCT 07283  01:31:44    41   88 4
  80  G-12 IMG      10 OCT 07283  01:45:15    41   88 4
  81  G-12 IMG      10 OCT 07283  02:01:44    41   88 4
  82  G-12 IMG      10 OCT 07283  02:15:15    41   88 4
  83  G-12 IMG      10 OCT 07283  02:31:44    41   88 4
  84  G-12 IMG      10 OCT 07283  02:45:13    41   88 4
  85  G-12 IMG      10 OCT 07283  03:15:15    41   88 4
  86  G-12 IMG      10 OCT 07283  03:31:44    41   88 4
  87  G-12 IMG      10 OCT 07283  03:45:15    41   88 4
  88  G-11 IMG      10 OCT 07283  04:00:14    41   88 4
  89  G-11 IMG      10 OCT 07283  04:30:13    41   88 4
  90  G-11 IMG      10 OCT 07283  05:00:14    41   88 4
  91  G-11 IMG      10 OCT 07283  05:30:14    41   88 4
  92  G-11 IMG      10 OCT 07283  06:00:13    41   88 4
  93  G-11 IMG      10 OCT 07283  06:30:13    41   88 4
  94  G-12 IMG      10 OCT 07283  07:01:43    41   88 4
  95  G-12 IMG      10 OCT 07283  07:15:14    41   88 4
  96  G-12 IMG      10 OCT 07283  07:31:43    41   88 4
  97  G-12 IMG      10 OCT 07283  07:45:14    41   88 4
  98  G-12 IMG      10 OCT 07283  08:15:14    41   88 4
  99  G-12 IMG      10 OCT 07283  08:45:14    41   88 4
 100  G-12 IMG      10 OCT 07283  09:15:14    41   88 4
 101  G-12 IMG      10 OCT 07283  09:45:14    41   88 4
 102  G-12 IMG      10 OCT 07283  10:15:14    41   88 4
 103  G-12 IMG      10 OCT 07283  10:45:14    41   88 4
 104  G-12 IMG      10 OCT 07283  11:01:44    41   88 4
 105  G-12 IMG      10 OCT 07283  11:15:14    41   88 4
 106  G-12 IMG      10 OCT 07283  11:31:43    41   88 4
 107  G-12 IMG      10 OCT 07283  11:45:14    41   88 4
 108  G-12 IMG      10 OCT 07283  12:15:14    41   88 4
 109  G-12 IMG      10 OCT 07283  12:45:14    41   88 4
 110  G-12 IMG      10 OCT 07283  13:01:43    41   88 4
 111  G-12 IMG      10 OCT 07283  13:15:14    41   88 4
 112  G-12 IMG      10 OCT 07283  13:31:43    41   88 4
 113  G-12 IMG      10 OCT 07283  13:45:14    41   88 4
 114  G-12 IMG      10 OCT 07283  14:01:43    41   88 4
 115  G-12 IMG      10 OCT 07283  14:15:14    41   88 4
 116  G-12 IMG      10 OCT 07283  14:31:43    41   88 4
 117  G-12 IMG      10 OCT 07283  14:45:14    41   88 4
 118  G-12 IMG      10 OCT 07283  15:15:13    41   88 4
 119  G-12 IMG      10 OCT 07283  15:31:44    41   88 4
 120  G-12 IMG      10 OCT 07283  15:45:14    41   88 4
 121  G-12 IMG      10 OCT 07283  16:01:43    41   88 4
 122  G-12 IMG      10 OCT 07283  16:15:14    41   88 4
 123  G-12 IMG      10 OCT 07283  16:31:43    41   88 4
 124  G-12 IMG      10 OCT 07283  16:45:14    41   88 4
 125  G-12 IMG      10 OCT 07283  17:01:43    41   88 4
 126  G-12 IMG      10 OCT 07283  17:15:16    41   88 4
 127  G-12 IMG      10 OCT 07283  17:31:44    41   88 4
 128  G-12 IMG      10 OCT 07283  17:45:14    41   88 4
 129  G-12 IMG      10 OCT 07283  18:15:15    41   88 4
 130  G-12 IMG      10 OCT 07283  18:45:15    41   88 4
 131  G-12 IMG      10 OCT 07283  19:01:44    41   88 4
 132  G-12 IMG      10 OCT 07283  19:15:15    41   88 4
 133  G-12 IMG      10 OCT 07283  19:31:44    41   88 4
 134  G-12 IMG      10 OCT 07283  19:45:15    41   88 4
IMGLIST: done

This listing shows no systematic lack of GOES-West imagery anytime during the
day today or yesterday.

By the way, as soon as we can get your machine fully setup, it will be easy
to evaluate your data ingest/processing using McIDAS ADDE.  In order to do this,
access to port 112 will need to be opened in the CUP firewall.

Also, I _strongly_ recommend that telnet access be turned off immediately.
Instead, please open SSH access (port 22) for machines in the CUP LAN
and for machines here at Unidata.  The subnets that we use are:

128.117.140.
128.117.156.

If access needs to be limited to certain machines, please make sure that
the following two machines are allowed access:

laraine.unidata.ucar.edu   128.117.140.62
yakov.unidata.ucar.edu     128.117.156.86

By the way, I am still unable to telnet to shu.cup.edu from my workstation here
at Unidata (yakov.unidata.ucar.edu).  This is very odd given that I could
telnet to shu from my home machine.

Cheers,

Tom
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: WTM-677712
Department: Support McIDAS
Priority: Normal
Status: Closed


NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.