[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[McIDAS #IDI-715243]: Question about yesterday's nexrad data

Hi,

First, I merged this inquiry with the one that David sent yesterday
afternoon since they are both about handling of "large" NEXRAD Level
3 products...

re:
> Adding to David Warren's question, I am wondering if you folks at
> Unidata had any trouble decoding/making images from the N0B (super
> resolution reflectivity) feed from OTX (Spokane, WA) radar yesterday
> after 20220610 04:12 and before 20220610 09:35 UTC?  

Thanks for specifying the date, time range and station ID; it _really_
made checking a lot easier!

I am able to correctly list and display N0B images for OTX during the
time period you specified in McIDAS-X.  Here is the result of the
IMGLIST run:

DATALOC ADD NEXRALL ADDE.SSEC.WISC.EDU

Group Name                    Server IP Address
--------------------         ----------------------------------------
NEXRALL                      ADDE.SSEC.WISC.EDU

<LOCAL-DATA> indicates that data will be accessed from the local data directory.
DATALOC -- done
 
IMGLIST NEXRALL/N0B.ALL ID=OTX DAY=161 TIME=04:00 10:00
Image file directory listing for:NEXRALL/N0B
 Pos Satellite/         Date       Time      Center   Band(s)
     sensor                                 Lat  Lon
 --- -------------  ------------  --------  ---- ---- ------------
  44  RADAR         10 JUN 22161  04:00:33     OTX    1
  45  RADAR         10 JUN 22161  04:06:39     OTX    1
  46  RADAR         10 JUN 22161  04:12:45     OTX    1
  47  RADAR         10 JUN 22161  04:18:51     OTX    1
  48  RADAR         10 JUN 22161  04:24:57     OTX    1
  49  RADAR         10 JUN 22161  04:31:03     OTX    1
  50  RADAR         10 JUN 22161  04:37:09     OTX    1
  51  RADAR         10 JUN 22161  04:43:16     OTX    1
  52  RADAR         10 JUN 22161  04:50:26     OTX    1
  53  RADAR         10 JUN 22161  04:56:33     OTX    1
  54  RADAR         10 JUN 22161  05:02:38     OTX    1
  55  RADAR         10 JUN 22161  05:08:43     OTX    1
  56  RADAR         10 JUN 22161  05:14:50     OTX    1
  57  RADAR         10 JUN 22161  05:20:57     OTX    1
  58  RADAR         10 JUN 22161  05:27:03     OTX    1
  59  RADAR         10 JUN 22161  05:33:09     OTX    1
  60  RADAR         10 JUN 22161  05:39:16     OTX    1
  61  RADAR         10 JUN 22161  05:45:22     OTX    1
  62  RADAR         10 JUN 22161  05:51:28     OTX    1
  63  RADAR         10 JUN 22161  05:57:34     OTX    1
  64  RADAR         10 JUN 22161  06:03:24     OTX    1
  65  RADAR         10 JUN 22161  06:09:30     OTX    1
  66  RADAR         10 JUN 22161  06:15:37     OTX    1
  67  RADAR         10 JUN 22161  06:21:42     OTX    1
  68  RADAR         10 JUN 22161  06:27:48     OTX    1
  69  RADAR         10 JUN 22161  06:33:53     OTX    1
  70  RADAR         10 JUN 22161  06:39:59     OTX    1
  71  RADAR         10 JUN 22161  06:46:06     OTX    1
  72  RADAR         10 JUN 22161  06:52:12     OTX    1
  73  RADAR         10 JUN 22161  06:58:18     OTX    1
  74  RADAR         10 JUN 22161  07:04:23     OTX    1
  75  RADAR         10 JUN 22161  07:10:00     OTX    1
  76  RADAR         10 JUN 22161  07:15:22     OTX    1
  77  RADAR         10 JUN 22161  07:20:44     OTX    1
  78  RADAR         10 JUN 22161  07:26:35     OTX    1
  79  RADAR         10 JUN 22161  07:32:12     OTX    1
  80  RADAR         10 JUN 22161  07:37:49     OTX    1
  81  RADAR         10 JUN 22161  07:43:26     OTX    1
  82  RADAR         10 JUN 22161  07:49:17     OTX    1
  83  RADAR         10 JUN 22161  07:55:23     OTX    1
  84  RADAR         10 JUN 22161  08:01:29     OTX    1
  85  RADAR         10 JUN 22161  08:07:35     OTX    1
  86  RADAR         10 JUN 22161  08:13:41     OTX    1
  87  RADAR         10 JUN 22161  08:19:47     OTX    1
  88  RADAR         10 JUN 22161  08:25:53     OTX    1
  89  RADAR         10 JUN 22161  08:31:59     OTX    1
  90  RADAR         10 JUN 22161  08:38:05     OTX    1
  91  RADAR         10 JUN 22161  08:44:11     OTX    1
  92  RADAR         10 JUN 22161  08:50:18     OTX    1
  93  RADAR         10 JUN 22161  08:55:54     OTX    1
  94  RADAR         10 JUN 22161  09:02:00     OTX    1
  95  RADAR         10 JUN 22161  09:07:52     OTX    1
  96  RADAR         10 JUN 22161  09:13:59     OTX    1
  97  RADAR         10 JUN 22161  09:19:34     OTX    1
  98  RADAR         10 JUN 22161  09:24:55     OTX    1
  99  RADAR         10 JUN 22161  09:30:03     OTX    1
 100  RADAR         10 JUN 22161  09:35:11     OTX    1
 101  RADAR         10 JUN 22161  09:40:33     OTX    1
 102  RADAR         10 JUN 22161  09:46:08     OTX    1
 103  RADAR         10 JUN 22161  09:51:44     OTX    1
 104  RADAR         10 JUN 22161  09:57:35     OTX    1
IMGLIST: done

For reference, the NEXRALL dataset contains the most recent N number of
days for NEXRAD Level 3 products, and, since it is an "archive" dataset
in McIDAS, the DAY of the data must be specified in all McIDAS
transactions.

re:
> Both our
> homegrown decode_nexrad and GEMPAK 7.15.0 versions of sfmap/gpmap
> could display those two bounding times, but none of the files from the
> intervening times displayed any radar echoes, despite the fact that it
> was raining over a vast portion of the radar volume scan.  

I noted the following comment made yesterday on GitHub regarding GEMPAK's 
handling
of NxB and NxG products:

From: Gilbert Sebenste <address@hidden> on 6/10/22, 13:18

"NOB is not working when a radar is in SAILS2 mode. Otherwise it plots
correctly, and base velocity plots correctly when the radar is in this mode."

I am assuming that Gilbert is referring to the same problem as you are
mentioning for the N0B product for OTX yesterday.

re:
> As with the
> previous day's problematic/missing ATX and LGX radar plots, this seems
> to have coincided with really heavy warm rain episodes that resulted
> in about 0.5 inches of rainfall in 24 hours.

It is most likely the case that the periods of really heavy rain
coincide with/be the cause of a decision that the product is large enough
to warrant external Zlib compression during the preparation for dissemination
in NOAAPort.  NB: This compression is in addition to any compression (like
bzip2) that is internal to the products.  This procedure has apparently
been in-place for as long as NEXRAD Level 3 products have been sent in
NOAAPort, but only got activated with the addition of the so-called
"super res" products to the SBN.  NB: it is the NWS that is preparing the
products for NOAAport, not us, so we have no control over this process.

As far as McIDAS goes: I made an additional change to the McIDAS code that
handles the uncompressing of the portions of NEXRAD Level 3 products that
get compressed in chunks for broadcast in NOAAPort, and I deployed this
code change on all of the public facing ADDE servers that we operate
on behalf of the community.  I also checked the new mod into SSEC's
McIDAS code repository.

What I found that I needed to add:

My code reads through "raw" NEXRAD Level 3 products looking for chunks
that are Zlib compressed, and uncompresses them before the main code
handling section.  In essence, my code is now doing the same sort of thing
that Dan Vietor's 'ucnids' program does.  The College of DuPage NEXLAB
site is using Dan Vietor's 'ucnids' program to uncompress NEXRAD Level 3
products before they are used, so you can check their site to see if
there are N0B plots for stations/times where your code is having difficulty.

What I had to add to my code was logic that would ignore a Zlib inflate
error for 4000 byte chunks *after* the first 4000 byte Zlib compressed chunk
is processed.  This change was needed because a small number of "super res"
products will contain the byte sequence that normally indicates the beginning
of a Zlib-compressed block/chunk when, in fact, the block is not compressed.

If you are using McIDAS-X, I can make the new code available to you; please
let me know!

re:
> Thanks for any insights.

I can't comment on how GEMPAK may or may not be handling the images that get
the extra "external" compression applied.  I can say that IF this is the
problem that you were experiencing, then running the NEXRAD Level 3
products through Dan Vietor's 'ucnids' routine before processing in GEMPAK
will likely work for you.

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: IDI-715243
Department: Support Datastream
Priority: Normal
Status: Open
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.