[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Datastream #MTV-910920]: wind profiler data corruption issues

Hi Laurie,

re:
> Thanks for checking into this.  I know some of the files are not supported.

OK.

> I thought I would check though since the problems seemed to clear up in the
> past.

This was probably due to the existing code not running into a situation it
couldn't handle.

> Do you know if the same or similar data is available in BUFR format?
> I saw similar data files on your backup system in BUFR format, but was not
> sure if they are available through LDM. We might be able to use the BUFR
> files as an alternative if they are still supported by  NOAA/GSD. The files I
> saw were at the following site.
> http://motherlode.ucar.edu/cgi-bin/ldm/genweb?native/profiler/bufr/

The bufr formatted profiler data is available in the IDD HDS datasetream.
Here are the pqact.conf actions that FILE that data on motherlode.ucar.edu:

#
# profiler data in BUFR format
HRS     ^(IUPT0[1-4]|IUAK01) .... ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/profiler/bufr/profiler/PROFILER_(\2:yyyy)(\2:mm)\2_0000.bufr
#
#
# profiler data in BUFR format
HRS     ^IUPT41 KBOU ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/profiler/bufr/profiler1/PROFILER_(\1:yyyy)(\1:mm)\1_0000.bufr
#
#
# profiler data in BUFR format
HRS     ^IUPT42 KBOU ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/profiler/bufr/profiler2/PROFILER_(\1:yyyy)(\1:mm)\1_0000.bufr
#
#
# profiler data in BUFR format
HRS     ^IUPT43 KBOU ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/profiler/bufr/profiler3/PROFILER_(\1:yyyy)(\1:mm)\1_0000.bufr
#
# profiler data in BUFR format
HRS     ^ISAT.. KBOU ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/profiler/bufr/profiler4/PROFILER_(\1:yyyy)(\1:mm)\1_0000.bufr
#
# profiler data in BUFR format
HRS     ^IUPC0[12] .... ([0-3][0-9])([0-2][0-9])([0-9][0-9])
        FILE
        
data/pub/native/profiler/bufr/profiler5/PROFILER_(\1:yyyy)(\1:mm)\1_0000.bufr

I am not sure why there are multiple actions.

Please note that in the past (not sure about the present) the profiler data was
considered to be experimental.  What that meant in practice was that the
broadcast priority of the data in NOAAPORT was low, meaning that the data
was not delivered in as timely a manner as the netCDF version from GSD 
(formerly FSL).

> In the mean time, I put sample files of the standard wind profiler netCDF
> data at the following ftp site.  These have the problems explained in my
> previous e-mail.  Let me know if you have any trouble getting to these or you
> need more samples.
> 
> ftp://ftp.xdc.arm.gov/pub/wpdn/

OK, thanks.  I started plowing through the 900-some 6-minute netCDF files we 
have
on a test machine here in the UPC and I found one from today that showed the
problems you reported:

 staName =
  "@\352L\247No",
  "WNCI2 ",
  "WNFL1 ",

The interesting part about the file I found the problems in (named 
20063100306.nc
on our system) was that the entry with the garbled station name had bad data
for everything else:

 wmoStaNum = 2123789977, 74556, 74753, 74629, 74630, 74662, 74357, 74440,

 staLat = 3.024685e+24, 39.65, 31.89, 32.4, 36.84, 36.88, 43.22, 40.08,

 staLon = 3607.216, -90.48, -92.78, -106.34, -107.91, -89.97, -90.53, -97.31,

 staElev = 1.356316e-19, 170, 93, 1224, 1902, 130, 226, 433, 1155, 564, 447,

 timeObs = 1.57404379143909e+108, 1162782360, 1162782360, 1162782360,

Given this pattern, it looks like the QC process for excluding a station did
not work.  I will contact GSD folks to report this situation.

> Thank you for your help,

No worries.

Cheers,

Tom
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: MTV-910920
Department: Support Datastream
Priority: Normal
Status: Closed