[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

20050923: 20050922: GEMPAK: how do I put a full day's worth of buoy reports in one file?



David,

Glad you have a workable solution.

The same problem exists in Garp with lightning data which is in ship format 
files.
The basic limitation is the way Garp opens the files to read times first.
NMAP speeds this up some by using the file name templates for the
time selection widget rather than opening the files first.

Steve Chiswell
Unidata User Support



>From: David Ovens <address@hidden>
>Organization: UCAR/Unidata
>Keywords: 200509231805.j8NI5wYJ006616

>Steve,
>
>Thank you, that did the trick!  This really improves the performance
>of GARP.  It was horrendously slow when loading ship and buoy data
>with hourly files, now pointing to the daily files, it loads the
>time selection menu nice and fast.
>
>David
>-- 
>David Ovens             e-mail: address@hidden
>Research Meteorologist
>Dept of Atm. Sciences      plan: Real-time MM5 forecasting for the
>Box 351640                        Pacific Northwest
>University of Washington          http://www.atmos.washington.edu/mm5rt
>Seattle, WA  98195               Weather Graphics and Loops
>                                  http://www.atmos.washington.edu/~ovens/loops
>
>On Thu, Sep 22, 2005 at 04:41:32PM -0600, Unidata Support wrote:
>> 
>> David,
>> 
>> When modifying the include files, MMHDRS = LLMXTM + LLSTFL
>> so if you increase LLSTFL, you have to increase MMHDRS.
>> 
>> In SFCFIL, you would use TIMSTN= 29999/1 which is a row/column limit
>> of the header. Your TIMSTN is incorrect, and you are hitting the 30,000 
>> limit between 1001*29 and 1001*30, but a ship file should have a single colu
> mn
>> and every time/station/report is a unique observation.
>> 
>> Creating a file with dcmsfc (eg not using sfcfil), the default
>> is to use MMHDRS-1, which would be: -m 29999
>> The iadstn should be ignored unless it is -a 6, which would bin the data by
>> 6 hour intervals.
>> 
>> I Ran both of these with the current 30,000 limit and suceeded in getting 
>> 29999 observations decoded.
>> 
>> Steve Chiswell
>> Unidata User Support
>> 
>> 
>> >From: David Ovens <address@hidden>
>> >Organization: UCAR/Unidata
>> >Keywords: 200509222029.j8MKTFYJ011923
>> 
>> >Hello,
>> >
>> >I've been attempting to fill a GEMPAK SHIP file with a full day's
>> >worht of ship and buoy reports (or even more) but I am running into
>> >program limits that I do not understand.  I have fully rebuilt GEMPAK
>> >with GEMPRM.PRM and gemprm.h having these limits:
>> >
>> >C!  File information parameters
>> >C!
>> >    PARAMETER       ( MMKEY  =    12   )
>> >C!                                          Maximum # of keys
>> >    PARAMETER       ( MMHDRS = 30000   )
>> >C!                                          Maximum # of headers
>> >    PARAMETER       ( MMPRT  =    20   )
>> >C!                                          Maximum # of parts
>> >    PARAMETER       ( MMLIST =    20   )
>> >C!                                          Maximum search list
>> >    PARAMETER       ( MMFREE =    62   )
>> >C!                                          Number of free pairs
>> >    PARAMETER       ( MMFILE =     5   )
>> >C!                                          Maximum # of open files
>> >    PARAMETER       ( MBLKSZ =   128   )
>> >C!                                          Block size
>> >    PARAMETER       ( MCACHE =     8   )
>> >C!                                          # of cached records
>> >    PARAMETER       ( MMPARM =    70   )
>> >C!                                          Maximum # of parameters
>> >    PARAMETER       ( MMFHDR =    10   )
>> >C!                                          Maximum # of file hdrs
>> >    PARAMETER       ( MMSRCH =    30   )
>> >C!                                          Max # of cond searches
>> >    PARAMETER       ( MXFLSZ =   100   )
>> >C!                                          Maximum file name size
>> >C!                                          (not including path)
>> >    PARAMETER       ( MXNMFL =  1000   )
>> >C!                                          Max # of files per dir.
>> >    PARAMETER       ( MFTEXT =   100   )
>> >C!                                          Text
>> >C!
>> >C!  Declarations for array sizes in programs
>> >    PARAMETER       ( LLMXLV =   500   )
>> >C!                                          Max # levels/station
>> >    PARAMETER       ( LLMXTM =   300   )
>> >C!                                          Max # times/dataset
>> >    PARAMETER       ( LLMXGT =  1000   )
>> >C!                                          Max # grid times
>> >    PARAMETER       ( LLMXST =    20   )
>> >C!                                          Max # stations in list
>> >    PARAMETER       ( LLMXDT = MMPARM * LLMXLV )
>> >C!                                          Max # data points
>> >    PARAMETER       ( LLSTFL = 29700   )
>> >C!                                          Max # stations in file
>> >    PARAMETER       ( LLMXLN =   128   )
>> >C!                                          Max length of input
>> >    PARAMETER       ( LLMDGG = 2504000 )
>> >C!                                          Max mem for intern grids
>> >    PARAMETER       ( MXLOOP =    30   )
>> >C!                                          Maximum frames in loop
>> >    PARAMETER       ( LLNNAV =   256   )
>> >C!                                          Grid nav block length
>> >    PARAMETER       ( LLNANL =   128   )
>> >C!                                          Grid anl block length
>> >    PARAMETER       ( LLSTHL =    20   )
>> >C!                                          Max header size
>> >    PARAMETER       ( LLGDHD =   128   )
>> >C!                                          Max grid hdr length
>> >    PARAMETER       ( LLOAGD =   400   )
>> >C!                                          Max # grids from 1 OA
>> >    PARAMETER       ( LLCLEV =   50    )
>> >C!                                          Max # of contour lvls
>> >    PARAMETER       ( LLAXIS =   64    )
>> >C!                                          Max # of axis labels
>> >    PARAMETER       ( LLTMCX =  300    )
>> >C!                                          Max # of time xsect pts
>> >    PARAMETER       ( LLMAXD =  128    )
>> >C!                                          Max # of background grid
>> >
>> >
>> >I have attempted 2 methods to get maximum output
>> >
>> > 1) using dcmsfc:
>> >#
>> ># ship, buoy and CMAN, TESTING DAILY FILES WITH 20,000 5/19/2005
>> >#
>> >WMO
>> >(^S[IMN]V[^GINS])|(^S[IMN]W[^KZ])|(^S(HV|HXX|S[^X]))|(^SX(VD|V.50|US(2[\
>> >0-3]|08|40|82|86)))|(^Y[HO]XX84)
>> >        PIPE    gdecoders/dcmsfc -b 9 -a 20000
>> >        -d data/gempak/logs/dcmsfc.log
>> >        -e GEMTBL=/usr/local/ldm/NAWIPS-5.8.3a/gempak/tables
>> >        data/gempak/ship.daily/YYYYMMDD_sb2.gem
>> >
>> >   The problem is the files fill up with obs by around 15Z every day
>> >   -- not too helpful.  If I use sflist to list all of the obs and
>> >   times for a given parameter, I getjust under 27,800 entries.  A
>> >   strange number.  I tried rebuilding GEMPAK with LLSTFL=58700, but I
>> >   still get the same limits.
>> >
>> > 2) using sfcfil here's what I get:
>> > GEMPAK-SFCFIL>li
>> > SFOUTF   = /tmp/200509_sb2.gem
>> > SFPRMF   = sfship.pack
>> > STNFIL   =  
>> > SHIPFL   = yes
>> > TIMSTN   = 1500/29
>> > SFFSRC   = SHIP
>> > GEMPAK-SFCFIL>run
>> ><snip>
>> >Number of stations in STNFIL:       0
>> > Number of additional stations:     29
>> > Total number of stations:          29
>> > Total number of times:           1500
>> >
>> > This file will be a ship file.
>> >
>> >Enter <cr> to accept parameters or type EXIT:
>> > [DM -5]  Invalid dimension sizes.
>> > [SF -1]  File /tmp/200509_sb2.gem could not be created.
>> >
>> >  This works fine if I set TIMSTN = 1001/29, but fails with the same
>> >  dimension error if I try 1001/30.  I don't get it!
>> >
>> >I am baffled by both 1) and 2).  I really want a fully loaded 24-hour
>> >ship and buoy file.  Is there any way I can do this?
>> >
>> >Thanks for any help,
>> >
>> >David
>> >-- 
>> >David Ovens          e-mail: address@hidden
>> >Research Meteorologist    phone: (206) 685-8108
>> >Dept of Atm. Sciences      plan: Real-time MM5 forecasting for the
>> >Box 351640                        Pacific Northwest
>> >University of Washington          http://www.atmos.washington.edu/mm5rt
>> >Seattle, WA  98195               Weather Graphics and Loops
>> >                                  http://www.atmos.washington.edu/~ovens/lo
> ops
>> >
>> --
>> ****************************************************************************
>> Unidata User Support                                    UCAR Unidata Program
>> (303)497-8643                                                  P.O. Box 3000
>> address@hidden                                   Boulder, CO 80307
>> ----------------------------------------------------------------------------
>> Unidata WWW Service              http://my.unidata.ucar.edu/content/support 
>> ----------------------------------------------------------------------------
>> NOTE: All email exchanges with Unidata User Support are recorded in the
>> Unidata inquiry tracking system and then made publicly available
>> through the web.  If you do not want to have your interactions made
>> available in this way, you must let us know in each email you send to us.
>
--
NOTE: All email exchanges with Unidata User Support are recorded in the
Unidata inquiry tracking system and then made publicly available
through the web.  If you do not want to have your interactions made
available in this way, you must let us know in each email you send to us.