[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[GEMPAK #AJO-748565]: GEMPAK - Possible problem with 5.10 decoder



Steve,

I see that your script uses "EOD" twice as 
SFLIST<<EOD
EOD

That label should be unique, such as:
SFLIST<<EOD_YESTERDAY
EOD_YESTERDAY

SFLIST<<EOD_TODAY
EOD_TODAY

I can't guarantee that is your problem, but is seems like when your
script enters into the block of IF ( TIME3 != "" ),
that defines EOD, so you only run yesterdays sflist
until you hit 4Z when I assume $TIME3 becomes nill and you don't overlap
your use of EOD.

That may be an esoteric difference...which you should try correcting.

I can tell that your 0Z file looks to have times from what appear to be:
0709271945 to 0709272344

The 1Z file: 
0709272045 to 0709272344

The 2Z file:
0709272145 to 0709272344

The 3Z file:
0709272245 to 0709272344

The 4Z file:
0709272345 to 0709280437

That probably indicates whatever processing you are doing is looking at 
yesterdays file only until 4Z
when it switches over to the current day's file (and not looking at 
yesterdays). You also source something 
called backhour which I can only assume is computing the time for yesterday 
(possibly minus 4 hours?)
I would guess your script is failing execute the 2 unique invocations of SFLIST 
(FILETODAY and FILEYEST)
and you are only getting one in any case.

Some little things that might help are to redirect the SFLIST output, or log
the script output to a file to verify what SFLIST is doing in both instances. 
Also,
set $RESPOND=yes and add a blank line after the "r" for run so that
your log output will show the proplt verification of what your
running settings are for SFLIST each time.

Steve Chiswell
Unidata User Support

> 
> Thanks Steve. Sorry for freaking out a little. I am feeling a lot of
> pressure here to find out what is wrong since it works on the Solaris
> machine and this affects 2 other projects here.
> 
> I will look into this closer. We haven't touched anything (we're not
> smart enough here) with the system. We just moved it from a Solaris
> machine to Linux and upgrade to 5.10 at the time. It could be that we
> were getting away with something on the Solaris machine and not on the
> Linux. I don't know why they are using the pgf77 compiler, I'll have to
> ask. I will see if those 2 compilers are on the Linux machine. Thanks
> 
> When I look at the file sizes of the output from dcmetr (.gem), they are
> nearly the same as what dcmetr are outputting on the Linux machine so I
> don't think we are losing anything there. But when I look at the file
> sizes of the output from sflist (sfc.dat) during the 0z-4z period the
> file sizes are a lot smaller.
> 
> And I found out today after looking inside each file, that the last
> record in each file is partially written which would mean that sflist
> crashed while writing to the file (see attached files one per hour
> 0z-4z). The 4z file is at 4:17z and has 24,211 records and the last one
> is complete.
> 
> So it looks like sflist is barfing on something in the input which is
> the output file from dcmetr. I am trying to capture the output of dcmetr
> but have to wait until tonight to capture the times of interest.
> 
> What do you think? Am I in the weeds on this line of thinking?
> 
> Steve
> 
> 301-874-1911 DIJM
> 240-687-0036 cell
> 
> (\__/)
> (='.'=)
> (")_(")
> 
> 
> -----Original Message-----
> From: Unidata GEMPAK Support [mailto:address@hidden]
> Sent: Friday, October 05, 2007 2:35 PM
> To: Kerich, Steve (DIJM)
> Cc: address@hidden
> Subject: [GEMPAK #AJO-748565]: GEMPAK - Possible problem with 5.10
> decoder
> 
> Steve,
> 
> Your support question was received and will be answered as time is
> available as described in the automated response that is sent when
> messages arrive.
> Our 2 day user committee meetings require most of the available time for
> everyone here.
> http://www.unidata.ucar.edu/committees/usercom/
> 
> You should also be aware of the support policy regarding our available
> resources:
> http://www.unidata.ucar.edu/legal/participation.html#support
> 
> You should note that the PG compiler is not a supported platform and it
> has limitations with multiple file opens that are not supported. The
> supplied source distribution will compile with g77 or gfortran.
> 
> In general, there has been no change with the dcmetr bulletin time
> processing that would identify any problem you are having. Rather, you
> should check the output of "date; date -u" and ensure that the proper
> time zone and time that you want to be running under is identified. That
> is, if you are setting the system time to UTC time, the TZ should also
> be UTC0.
> 
> The script you sent used a variable in the dcmetr "-c" option that
> wasn't shown, so you should also check that setting, as well as use
> DATTIM=list in your SFLIST program for yesterday's file to see what
> times have been stored in the file. It sounds as if you have the
> previous days times overrunning the available 72 times, your -c
> parameter is incorrect and therefore trying to write the 0Z to 4Z data
> into the wrone file, or is not a correctly formated GEMTIM.
> 
> The current 5.10.3 implementation is well tested under Linux, and so
> would be the best version to work with,
> 
> Steve Chiswell
> Unidata User Support
> 
> 
> 
> 
> >
> > Hi
> > I had sent this email on Wednesday and haven't heard back from anyone
> > on this. It is a critical problem for us and we need you help in
> > solving it. I am convinced that the problem is in sflist version 5.10
> > . The size of the .gem files that are output from dcmsfc is about the
> > same size as the ones on the system that is working fine. .gem files
> > are input to sflist but the resulting file is very small which means
> less records.
> > The output file on the system that is working fine and  is using
> > GEMPAK 5.6f sflist, is 10 times bigger. There is something that sflist
> 
> > 5.10 is doing differently (and incorrectly) that sflist 5.6f is does
> fine.
> >
> > We have to upgrade to 5.10 BUT this issue is huge because we are
> > losing all of our surface observations in the output of sflist.
> >
> > Can someone help me today?  I am really stuck and I can't get the
> > source to compiled under Linux to try the 5.6f source on our system
> > (see below).
> >
> > Thanks
> > Steve
> >
> >
> > 301-874-1911 DIJM
> > 240-687-0036 cell
> >
> > (\__/)
> > (='.'=)
> > (")_(")
> >
> >
> >
> > ________________________________
> >
> > From: Kerich, Steve (DIJM)
> > Sent: Wednesday, October 03, 2007 4:23 PM
> > To: 'address@hidden'
> > Subject: RE: [GEMPAK #AJO-748565]: GEMPAK - Possible problem with 5.10
> 
> > decoder
> >
> >
> >
> > Hi
> >
> > I was going try to push through compiling GEMPEK 5.6f with the pgf77
> > compiler for Linux but I am getting this error. The compile is not
> > very straight forward because there are a lot of includes and link and
> 
> > preprocessor compiler commands which are now causing me trouble. The
> > issue I have not been able to solve yet is below.
> >
> > The error in MCHPRM.PRM
> >
> > PGFTN-S-0021-Label field  of continuation line is not blank
> >
> > which is included by GEMPRM.PRM. I attached both files for reference.
> > I believe it has to do with the preprocessor. I changed .f to .F to
> > force the preprocessor to run and it still got this error. It ran the
> > C prepreocessor I think with the -c option but it still caused the
> error.
> >
> > MCHPRM.PRM only has this line in it.
> >
> > link ../MCHPRM.Linux
> >
> > I just can't see or understand what is wrong. I am not that familiar
> > with this syntax. I hope if I get past this problem I can get all of
> > GEMPAK compiled under Linux
> >
> > Help,
> >
> > Steve
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > 301-874-1911 DIJM
> > 240-687-0036 cell
> >
> > (\__/)
> > (='.'=)
> > (")_(")
> >
> >
> > -----Original Message-----
> > From: Unidata GEMPAK Support [mailto:address@hidden]
> > Sent: Wednesday, October 03, 2007 12:17 PM
> > To: Kerich, Steve (DIJM)
> > Cc: address@hidden
> > Subject: [GEMPAK #AJO-748565]: GEMPAK - Possible problem with 5.10
> > decoder
> >
> > Steve,
> >
> > I don't have old binaries for several reasons, including storage
> > space, the rapid change of the Linux environment making old binaries
> > obsolete, and old versions containing old bugs.
> >
> > It sounds to me like you have a basic clock problem on your system,
> > and/or the file template and maxmimum number of times in a file is
> > exceeding an old file that is trying to be written to. The 4 hours is
> > suspiciously close to the EST/EDT + 1 hour in the future combination
> > if that is the time zone configured on your computer.
> >
> > Ideally, the decoder action for metars uses -m 72 and a template of
> > YYYYMMDD for 72 hourly time bins in a daily file so that three 20
> > minute obsservation bins per hour exist. If your clock/timedone is off
> 
> > or you are not using a time template but instead are trying to use a
> > set filename, you may be encountering problems with arriving at a 0Z
> > to 4Z time for yesterday instead, or a correct tiume, but your output
> > file is already full of times rather than containing the current days
> times.
> >
> > Check your UTC and local time (date; date -u) to ensure you have
> > correctly configured your new machine. Then check your file template
> > and maximum number of times in the file. If you still have trouble,
> > please provide me with your command line for dcmetr that you are
> > attempting to use along with any log output.
> > It would help iuf you set the decoder verbose level to 4 for degugging
> 
> > as well ( -v 4 ).
> >
> > Steve Chiswell
> > Unidata User SUpport
> >
> >
> > > Institution: smiths detection
> > > Package Version: 5.6f
> > > Operating System: linux
> > > Hardware Information: dual quad xeon
> > > Inquiry: Hi
> > >
> > > We ported from a Solaris 8 to a Linux machine and used the Linux
> > > GEMPAK binaries for 5.10 (latest). Our Solaris system uses 5.6f and
> > > is running fine. The new system seems to be dropping surface data
> > > observations for a 4 hour period from 0z to 4z and then it starts to
> 
> > > pass all the observations through again until 0z. We are using
> > > dcmetr and sflist to decode and parse the data. I would like to try
> > > a 5.6f Linux version of these two but the binaries on the new
> > > machine but they are not posted/available. I have been trying to
> > > compile 5.6f on our Linux machine to create the binaries for these
> > > two programs but have not yet been able to after spend tons of time
> on it.
> > >
> > > If you have the a Linux executable/binaries for 5.6f or something
> > > around this version for just these two programs that you could send
> > > me, that would be great. That is all I need in order to continue
> > > debugging what is going on with this system. I hope you can help me
> > > with these two GEMPAK programs. Any help would be appreciated.
> > >
> > > Steve
> > >
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: AJO-748565
> > Department: Support GEMPAK
> > Priority: Normal
> > Status: Closed
> >
> >
> > ______________________________________________________________________
> > This email has been scanned by the MessageLabs Email Security System.
> > For more information please visit http://www.messagelabs.com/email
> > ______________________________________________________________________
> >
> >
> >
> > </table> </Pre>
> > <HTML>
> > <br>
> > <br>
> > ************************************************<br>
> > The information contained in, or attached to, this e-mail, may contain
> 
> > confidential information and is intended solely for the use of the
> > individual or entity to whom they are addressed and may be subject to
> > legal privilege.  If you have received this e-mail in error you should
> 
> > notify the sender immediately by reply e-mail, delete the message from
> 
> > your system and notify your system manager.  Please do not copy it for
> 
> > any purpose, or disclose its contents to any other person.  The views
> > or opinions presented in this e-mail are solely those of the author
> > and do not necessarily represent those of the company.  The recipient
> > should check this e-mail and any attachments for the presence of
> > viruses.  The company accepts no liability for any damage caused,
> > directly or indirectly, by any virus transmitted in this email.<br>
> > ************************************************<br>
> > </HTML>
> >
> 
> 
> Ticket Details
> ===================
> Ticket ID: AJO-748565
> Department: Support GEMPAK
> Priority: Normal
> Status: Closed
> 
> 
> ______________________________________________________________________
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit http://www.messagelabs.com/email
> ______________________________________________________________________
> 
> 
> 
> ************************************************
> The information contained in, or attached to, this e-mail, may contain 
> confidential information and is intended solely for the use of the individual 
> or entity to whom they are addressed and may be subject to legal privilege.  
> If you have received this e-mail in error you should notify the sender 
> immediately by reply e-mail, delete the message from your system and notify 
> your system manager.  Please do not copy it for any purpose, or disclose its 
> contents to any other person.  The views or opinions presented in this e-mail 
> are solely those of the author and do not necessarily represent those of the 
> company.  The recipient should check this e-mail and any attachments for the 
> presence of viruses.  The company accepts no liability for any damage caused, 
> directly or indirectly, by any virus transmitted in this email.
> ************************************************
> 


Ticket Details
===================
Ticket ID: AJO-748565
Department: Support GEMPAK
Priority: Normal
Status: Closed