Re: [ldm-users] GRB data feed issues - incomplete files/gaps?

https://noaa-goes19.s3.amazonaws.com/index.html#ABI-L1b-RadF/2025/216/23/

https://noaa-goes19.s3.amazonaws.com/index.html#ABI-L1b-RadF/2025/217/15/

https://noaa-goes19.s3.amazonaws.com/index.html#ABI-L1b-RadF/2025/218/14/

Looking at the S3 bucket, I'm not seeing the same issues. Your
89MB 
OR_ABI-L1b-RadF-M6C02_G19_s20252181410206_e20252181419514_c20252181419555.nc
is 373MB on the S3 bucket which is populated by University of North
Carolina under the NOAA Open Data Dissemination project.





On Wed, Aug 6, 2025 at 11:12 AM Mike Zuranski <zuranski.wx@xxxxxxxxx> wrote:

> Pete,
>
> Since you've done this much leg work already, you may want to open up a
> ticket here: ESPCOperations@xxxxxxxx
>
> Since multiple receive sites are seeing this it looks to be somewhere in
> their court anyways, I'd start one but I feel like I'd just be taking
> credit for your hard work. ;)
>
> Odd though, I don't see these anomalies on the below sites and I thought
> they were using GRB data for those (whole full disk isn't on SBN anymore)...
>
>
> https://www.star.nesdis.noaa.gov/goes/fulldisk_band.php?sat=G19&band=02&length=12
>
> https://rammb-slider.cira.colostate.edu/?sat=goes-19&sec=full_disk&x=10848&y=10848&z=0&angle=0&im=24&ts=1&st=0&et=0&speed=130&motion=loop&maps%5Bborders%5D=white&p%5B0%5D=band_02&opacity%5B0%5D=1&pause=0&slider=-1&hide_controls=0&mouse_draw=0&follow_feature=0&follow_hide=0&s=rammb-slider&draw_color=FFD700&draw_width=6
>
> -Mike
>
> On Wed, Aug 6, 2025 at 1:04 PM Pete Pokrandt <poker@xxxxxxxxxxxx> wrote:
>
>> I'm wondering if there is maybe a disk full on the ingest machine or
>> something like that? Or needs a reboot to free up resources?
>>
>> Pete
>>
>> Pete Pokrandt - System Engineer IV
>> UW-Madison Dept of Atmospheric and Oceanic Sciences
>> 608-262-3086  - poker@xxxxxxxxxxxx
>> ------------------------------
>> *From:* Mike Zuranski <zuranski.wx@xxxxxxxxx>
>> *Sent:* Wednesday, August 6, 2025 12:02 PM
>> *To:* Pete Pokrandt <poker@xxxxxxxxxxxx>
>> *Cc:* NSF Unidata General Support/Unable To Determine <
>> support@xxxxxxxxxxxxxxxx>; LDM <ldm-users@xxxxxxxxxxxxxxxx>
>> *Subject:* Re: [ldm-users] GRB data feed issues - incomplete files/gaps?
>>
>> Drat, so much for that idea.
>>
>> I'll try to dig around more in a little bit, if I find out anything I'll
>> report back.
>>
>> Best,
>> -Mike
>>
>> On Wed, Aug 6, 2025 at 1:00 PM Pete Pokrandt <poker@xxxxxxxxxxxx> wrote:
>>
>> Hi Mike,
>>
>> I don't think that's it.. It looks like there was a short file for the
>> fulldisk imagery for all bands at 18:11 CDT August 4 (23:11 UTC if my math
>> is right) -
>>
>> -rw-rw-r--. 1 ldm ldm 297824695 Aug  4 18:20
>> OR_ABI-L1b-RadF-M6C02_G19_s20252162310222_e20252162319530_c20252162319577.nc
>> -rw-rw-r--. 1 ldm ldm  86392786 Aug  4 18:11
>> OR_ABI-L1b-RadF-M4C02_G19_s20252162305222_e20252162305222_c20252162310491.nc
>> -rw-rw-r--. 1 ldm ldm 303772302 Aug  4 18:05
>> OR_ABI-L1b-RadF-M4C02_G19_s20252162300225_e20252162305026_c20252162305062.nc
>>
>> Interesting that it looks like it went to mode 4 scanning for that one
>> image time, and then back to mode 6 scanning.
>>
>> After that, occasionally there will be short/incomplete files, for
>> example this time period between 10AM and 11:11 CDT:
>>
>> -rw-rw-r--. 1 ldm ldm 431257940 Aug  5 11:11
>> OR_ABI-L1b-RadF-M6C02_G19_s20252171600205_e20252171609513_c20252171609535.nc
>> -rw-rw-r--. 1 ldm ldm 287430120 Aug  5 11:01
>> OR_ABI-L1b-RadF-M6C02_G19_s20252171550205_e20252171559513_c20252171559548.nc
>> -rw-rw-r--. 1 ldm ldm 345413249 Aug  5 10:51
>> OR_ABI-L1b-RadF-M6C02_G19_s20252171540205_e20252171549513_c20252171549537.nc
>> -rw-rw-r--. 1 ldm ldm 425280428 Aug  5 10:42
>> OR_ABI-L1b-RadF-M6C02_G19_s20252171530205_e20252171539513_c20252171539550.nc
>> -rw-rw-r--. 1 ldm ldm 423293547 Aug  5 10:31
>> OR_ABI-L1b-RadF-M6C02_G19_s20252171520205_e20252171529513_c20252171529537.nc
>> -rw-rw-r--. 1 ldm ldm 200242429 Aug  5 10:21
>> OR_ABI-L1b-RadF-M6C02_G19_s20252171510205_e20252171519513_c20252171519544.nc
>> -rw-rw-r--. 1 ldm ldm 418631328 Aug  5 10:11
>> OR_ABI-L1b-RadF-M6C02_G19_s20252171500205_e20252171509513_c20252171509550.nc
>> -rw-rw-r--. 1 ldm ldm 414910682 Aug  5 10:01
>> OR_ABI-L1b-RadF-M6C02_G19_s20252171450205_e20252171459513_c20252171459541.nc
>>
>> Whatever is happening doesn't appear to be affecting the tiled files that
>> come in on the NIMAGE NOAAPort data feed..
>>
>> The problem is still ongoing as of 12 noon CDT today (17 UTC August 6)
>>
>> Pete
>>
>> Pete Pokrandt - System Engineer IV
>> UW-Madison Dept of Atmospheric and Oceanic Sciences
>> 608-262-3086  - poker@xxxxxxxxxxxx
>> ------------------------------
>> *From:* Mike Zuranski <zuranski.wx@xxxxxxxxx>
>> *Sent:* Wednesday, August 6, 2025 11:41 AM
>> *To:* Pete Pokrandt <poker@xxxxxxxxxxxx>
>> *Cc:* NSF Unidata General Support/Unable To Determine <
>> support@xxxxxxxxxxxxxxxx>; LDM <ldm-users@xxxxxxxxxxxxxxxx>
>> *Subject:* Re: [ldm-users] GRB data feed issues - incomplete files/gaps?
>>
>> Hi Pete,
>>
>> I wonder if this is at least partially responsible:
>> https://www.ospo.noaa.gov/data/messages/2025/08/MSG_20250805_1758.html
>> <https://urldefense.com/v3/__https://www.ospo.noaa.gov/data/messages/2025/08/MSG_20250805_1758.html__;!!Mak6IKo!JHAj-wuy5ZSYizUWzFoODtVW6UJ3V5t997XzgYRJ9gDJFN-GwghmkyO4g1kP-wLTTWq47g9IodOQBKmVQhAeZg$>
>>
>> That says it has completed, if you don't see these issues anymore that
>> would lend to the theory.  Do you notice if the timing of when this started
>> and/or stopped lines up with the planned maintenance mentioned above?
>>
>> -Mike
>>
>> On Wed, Aug 6, 2025 at 12:35 PM Pete Pokrandt <poker@xxxxxxxxxxxx> wrote:
>>
>> Hi all,
>>
>> I've noticed the past several days there are gaps in many of the GRB data
>> files. Here's an example from the GOES 19 ABI channel 2 (red visible band)
>> from this morning.
>>
>> Here are the file sizes for the files that I have received for the past
>> few hours
>>
>> -rw-rw-r--. 1 ldm ldm 328635079 Aug  6 07:10
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181200206_e20252181209514_c20252181209552.nc
>> -rw-rw-r--. 1 ldm ldm 334304629 Aug  6 07:21
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181210206_e20252181219514_c20252181219537.nc
>> -rw-rw-r--. 1 ldm ldm 339713187 Aug  6 07:31
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181220206_e20252181229514_c20252181229553.nc
>> -rw-rw-r--. 1 ldm ldm 344622117 Aug  6 07:40
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181230206_e20252181239514_c20252181239545.nc
>> -rw-rw-r--. 1 ldm ldm  99635697 Aug  6 07:50
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181240206_e20252181249514_c20252181249552.nc
>> -rw-rw-r--. 1 ldm ldm 355223647 Aug  6 08:00
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181250206_e20252181259514_c20252181259545.nc
>> -rw-rw-r--. 1 ldm ldm 361019443 Aug  6 08:10
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181300206_e20252181309514_c20252181309546.nc
>> -rw-rw-r--. 1 ldm ldm 365952779 Aug  6 08:21
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181310206_e20252181319514_c20252181319547.nc
>> -rw-rw-r--. 1 ldm ldm 371623853 Aug  6 08:31
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181320206_e20252181329514_c20252181329556.nc
>> -rw-rw-r--. 1 ldm ldm 376025396 Aug  6 08:40
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181330206_e20252181339514_c20252181339541.nc
>> -rw-rw-r--. 1 ldm ldm 381338035 Aug  6 08:50
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181340206_e20252181349514_c20252181349561.nc
>> -rw-rw-r--. 1 ldm ldm 385625318 Aug  6 09:00
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181350206_e20252181359514_c20252181359542.nc
>> -rw-rw-r--. 1 ldm ldm 390331029 Aug  6 09:10
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181400206_e20252181409514_c20252181409535.nc
>> -rw-rw-r--. 1 ldm ldm  89401307 Aug  6 09:20
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181410206_e20252181419514_c20252181419555.nc
>> -rw-rw-r--. 1 ldm ldm 400133060 Aug  6 09:31
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181420206_e20252181429514_c20252181429551.nc
>> -rw-rw-r--. 1 ldm ldm 404980007 Aug  6 09:41
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181430206_e20252181439514_c20252181439557.nc
>> -rw-rw-r--. 1 ldm ldm 232816097 Aug  6 09:50
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181440206_e20252181449514_c20252181449557.nc
>> -rw-rw-r--. 1 ldm ldm 154056273 Aug  6 10:00
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181450206_e20252181459514_c20252181459553.nc
>> -rw-rw-r--. 1 ldm ldm 416413968 Aug  6 10:11
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181500206_e20252181509514_c20252181509546.nc
>> -rw-rw-r--. 1 ldm ldm 418900344 Aug  6 10:25
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181510206_e20252181519514_c20252181519543.nc
>> -rw-rw-r--. 1 ldm ldm 421060195 Aug  6 10:31
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181520206_e20252181529514_c20252181529552.nc
>> -rw-rw-r--. 1 ldm ldm 423133036 Aug  6 10:41
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181530206_e20252181539514_c20252181539551.nc
>> -rw-rw-r--. 1 ldm ldm 425821389 Aug  6 10:51
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181540206_e20252181549514_c20252181549552.nc
>> -rw-rw-r--. 1 ldm ldm 207945828 Aug  6 11:00
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181550206_e20252181559514_c20252181559539.nc
>>
>> To make sure it isn't just me, I checked on the Unidata THREDDS server
>> and the file sizes seem to match what I got
>>
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181200206_e20252181209514_c20252181209552.nc
>>     328.6 Mbytes   2025-08-06T12:10:32.322Z
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181210206_e20252181219514_c20252181219537.nc
>>     334.3 Mbytes   2025-08-06T12:20:31.238Z
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181220206_e20252181229514_c20252181229553.nc
>>     339.7 Mbytes   2025-08-06T12:30:36.489Z
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181230206_e20252181239514_c20252181239545.nc
>>     344.6 Mbytes   2025-08-06T12:40:33.348Z
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181240206_e20252181249514_c20252181249552.nc
>>     99.63 Mbytes   2025-08-06T12:50:07.355Z
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181250206_e20252181259514_c20252181259545.nc
>>     355.2 Mbytes   2025-08-06T13:00:31.939Z
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181300206_e20252181309514_c20252181309546.nc
>>     361.0 Mbytes   2025-08-06T13:10:33.105Z
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181310206_e20252181319514_c20252181319547.nc
>>     365.9 Mbytes   2025-08-06T13:20:33.494Z
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181320206_e20252181329514_c20252181329556.nc
>>     371.6 Mbytes   2025-08-06T13:30:37.876Z
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181330206_e20252181339514_c20252181339541.nc
>>     376.0 Mbytes   2025-08-06T13:40:31.790Z
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181340206_e20252181349514_c20252181349561.nc
>>     381.3 Mbytes   2025-08-06T13:50:10.795Z
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181350206_e20252181359514_c20252181359542.nc
>>     385.6 Mbytes   2025-08-06T14:00:10.330Z
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181400206_e20252181409514_c20252181409535.nc
>>     390.3 Mbytes   2025-08-06T14:10:06.199Z
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181410206_e20252181419514_c20252181419555.nc
>>     89.40 Mbytes   2025-08-06T14:20:01.406Z
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181420206_e20252181429514_c20252181429551.nc
>>     400.1 Mbytes   2025-08-06T14:30:34.302Z
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181430206_e20252181439514_c20252181439557.nc
>>     404.9 Mbytes   2025-08-06T14:40:35.296Z
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181440206_e20252181449514_c20252181449557.nc
>>     232.8 Mbytes   2025-08-06T14:50:04.090Z
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181450206_e20252181459514_c20252181459553.nc
>>     154.0 Mbytes   2025-08-06T15:00:02.672Z
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181500206_e20252181509514_c20252181509546.nc
>>     416.4 Mbytes   2025-08-06T15:10:39.942Z
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181510206_e20252181519514_c20252181519543.nc
>>     418.9 Mbytes   2025-08-06T15:20:37.597Z
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181520206_e20252181529514_c20252181529552.nc
>>     421.0 Mbytes   2025-08-06T15:30:37.023Z
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181530206_e20252181539514_c20252181539551.nc
>>     423.1 Mbytes   2025-08-06T15:40:45.582Z
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181540206_e20252181549514_c20252181549552.nc
>>     425.8 Mbytes   2025-08-06T15:50:50.557Z
>> OR_ABI-L1b-RadF-M6C02_G19_s20252181550206_e20252181559514_c20252181559539.nc
>>     207.9 Mbytes   2025-08-06T16:00:25.938Z
>>
>> Here's what the imagery looks like when you plot those files
>>
>> https://whirlwind.aos.wisc.edu/~wxp/goes16/vis/goes16_fulldisk.html
>>
>> Whatever is causing this is affecting the GOES18 west GRB feed as well.
>>
>> I did a little poking around and didn't see any admin messages about any
>> data processing issues for GOES east/west data.
>>
>> Any idea what might be causing this?
>>
>> Thanks,
>> Pete
>>
>> Pete Pokrandt - System Engineer IV
>> UW-Madison Dept of Atmospheric and Oceanic Sciences
>> 608-262-3086  - poker@xxxxxxxxxxxx
>> _______________________________________________
>> ldm-users mailing list
>> ldm-users@xxxxxxxxxxxxxxxx
>> To subscribe: ldm-users-join@xxxxxxxxxxxxxxxx
>> To unsubscribe: ldm-users-leave@xxxxxxxxxxxxxxxx
>>
>>
>>
>> --
>> - Mike Zuranski
>>
>>
>>
>> --
>> - Mike Zuranski
>>
>
>
> --
> - Mike Zuranski
> _______________________________________________
> ldm-users mailing list
> ldm-users@xxxxxxxxxxxxxxxx
> To subscribe: ldm-users-join@xxxxxxxxxxxxxxxx
> To unsubscribe: ldm-users-leave@xxxxxxxxxxxxxxxx
>
  • 2025 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the ldm-users archives: