Greetings! The HRRR output we serve out though our demonstration TDS (thredds.ucar.edu) is obtained directly from the NOAAPORT satellite broadcast. My understanding is that individual GRIB messages are sent out across the satellite broadcast at the same time the model is running and output is being generated at NCEP. There will be a delay associated with getting the output from NCEP servers uploaded into the satellite broadcast, as well as the reception of the satellite broadcast on our end (network transmission to the server hosting our TDS will also cause a delay, but that one is relatively small). As an example of timing, I logged the incoming messages we received for the 22 UTC run of the HRRR from yesterday (August 7th 2020). The server hosting our TDS received the first forecast GRIB message from the HRRR at 22:58:50 UTC (U-Component of Wind, 925 hPa, valid at 2020-08-07 22 UTC), and received the final message at 23:22:23 UTC (Vertically-Integrated Liquid Water, valid at 2020-08-08 16 UTC). The TDS will wait until it believes all the GRIB messages for a run have made been received from the broadcast, and will only expose the output for access at that point. In this case, the 22 UTC run of the HRRR was accessible at 23:30 UTC (so 8 minutes after we received the last GRIB message from the satellite broadcast). Because each GRIB message is independent (one horizontal slice valid at a single time), you could start interrogating the output starting at 22:58 UTC if you had your own satellite feed. However, because the TDS exposes collections of GRIB messages as N-Dimensional variables, we have to wait until all of the messages have arrived. If you did have your own satellite feed, you could specifically watch the feed for any precip messages and act on those as soon as they hit disk. Even then, you'd only get messages as fast the the model would output them, as fast NCEP could get them into the NOAAPORT broadcast, and as fast as your receiver received and decoded them. Note that this is the kind of setup the National Weather Service forecast offices have in place. Another approach would be to access the output directly from NCEP. As an example, the GRIB messages for the HRRR can be found here: https://nomads.ncep.noaa.gov/pub/data/nccf/com/hrrr/prod/ NCEP collects all of the GRIB messages for a valid time and stores them as one file. Again, using the 22 UTC run from yesterday, I saw that the collection of messages valid at 22 UTC were made available at 22:45 UTC. The collection of 18 hour forecast (valid at 2020-08-08 16 UTC) were made available at 23:19 UTC. To summarize the timings above for the 2020-08-07 22 UTC run of the HRRR: TDS: * Output available at 23:30 UTC NOAAPORT Satellite broadcast (with decoding/transmission delays): * First individual GRIB message available at 22:58:50 UTC * Last individual GRIB message available at 23:22:23 UTC Direct from NCEP Servers: * 0 Hour forecast GRIB messages available at 22:45 UTC * 18 Hour forecast GRIB messages available at 23:19 UTC One quick note about the GRIB collections we serve up on our TDS and the files on the nomads server. The forecast GRIB messages we serve up as collections on our TDS are contained within the files on the nomads server that follow the pattern hrrr.t<RR>z.wrfprsf<FF>.grib2 (RR - runtime, FF - forecast hour). However, not all of the GRIB messages generated by the HRRR are sent through the satellite broadcast, so the files on the nomads server will contain more output that what you see on our servers. I hope that gives you a better idea of dataflow used to drive our demonstration system, and how that relates to the process by which GRIB messages are produced and transmitted from NCEP. Cheers, Sean > Hello, > > I was hoping to learn more information about the process by which datasets > become available on the THREDDS server. Specifically, we're using the > forecasted HRRR 1-hour precipitation products as a driver for a near > real-time flood model but we're finding the average lag between the model > initialization time and when it's publicly available to be on the order of > 1-2 hours. I realize there will always be some latency with such models but > do you know of any resources that could help reduce that lag? > > Thank you in advance for your help - happy to provide additional details if > needed. > > Best, Ticket Details =================== Ticket ID: OYF-200327 Department: Support THREDDS Priority: Normal Status: Open =================== NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.