Hi Don,
On 7/2/13 6:11 PM, Don Murray wrote:
> Hi Sean-
>
> Is the NAM 20km a defunct dataset? If not, what happens to IDV bundles
> that use these? A description of why the changes are being made would
> be useful as well as alternatives.
The NAM 20km isn't defunct that I know of - we are still pulling in the
grids from NOAAPORT (and those are still available). The surface and
selectsurface grids directories on disk were empty, which means we
haven't been getting them for at least 14 days. This is also the case on
our other unidata servers as well. Sometimes changes to data catch us
off guard, too :-)
Is there a reason to use the NAM CONUS 20km grids over the 12km grids?
I do believe these files were being FTP'd to Unidata servers, and were
not being transmitted via IDD, so perhaps something changed on the FTP
server. At any rate, no one mentioned any issue with accessing these
files (again, the other machines with much longer scour times were empty
too, so it's been down for quite some time), so I made the call to
remove them from the catalog. Perhaps it was the wrong call for me to make.
>
> I would prefer to see these changes announced before the fact rather
> than after or on a web page that one has to search around for.
That would be the ideal situation! The idea of this page is that it will
be "the place" to look. No need to search for it if you bookmark it (and
if it's prominently linked off of our main page). What do you feel would
be the appropriate venue other than a status page? I'm assuming some
soft of real time notification? RSS, twitter (which can be used like an
rss feed), mailing list (which already exist, like nws-changes), the
blink html tag in the thredds catalogs? These have seriously crossed my
mind. Ok, the blink tag might be a bit much (and it's not being support
much anymore), but in all honestly, what would be most beneficial for
you (or anyone else - feel free to chime in!).
> For example, the recent change from storing 21 days to 15 days impacted my
> work and I only found out about it after when I tried to display the
> data.
That change was a bit difficult - we could not give much of a lead time
as our disk space was ran low quick. This is in part due to us including
the new FNMOC NAVGEM 0.5 degree global model output, but level II radar
data volumes are also getting tough to manage (and continue to grow).
Also, as you mentioned in your support message, you understand that
Unidata isn't a data archive center. You also mention NOMADS, which is a
model data archive (oversimplified statement). I do believe they are
testing out the use of TDS 4.3, which will ease the issues you have with
the way they serve out the individual forecast grids. Have you tried to
contact them to see what their plans are?
...shameless plug...
Not necessairly news for you, but for the benefit of others...This is
also a good time to note that while the thredds servers at Unidata
(particularly the one housed on the motherlode machine) has always been
a "demonstration" installation (even though it has been "good enough"
for people to heavily rely upon). We encourage people in the community
to apply for Unidata equipment awards to setup their own
"motherlode"-like servers:
http://www.unidata.ucar.edu/community/equipaward/
Also, if there are real-time datasets people would like to see being
served by Unidata, the best way to make the request is through our Users
Committee:
http://www.unidata.ucar.edu/committees/usercom/
We've relied on them in the past (and will continue to do so) to get a
sense of what model output are important to the community, what model
output could be cut with minimal impact, and what would be beneficial to
add, say, to the CONDUIT feed (it should be noted that NCEP has been
awesome about listening to our community!).
...end shameless plug...
> What is the status of the longer term LEAD archive? Will it be
> upgraded to TDS 4.3 in the near future so I can switch to that for my
> longer "real-time" archive needs?
We are discussing how to best proceed with our other servers. Do we
simply make motherlode clones and distribute the load across them? How
do we best allow clients, like the IDV, to have one access url that can
point to other resources behind the scenes (as to prevent bundle
breaking if a server goes down). It does not make sense to blast forward
at this point, but. we are talking about these things internally
>
> Unidata provides a valuable service with the thredds.ucar.edu dataset to
> the IDV and other communities, so even minor changes have impacts that
> would be nice to know of before they happen.
Again, some changes we can announce with some lead-time, but these are
mostly limited to NWS or CONDUIT feed changes (for which email lists
already exist).
Sean
>
> Thanks.
>
> Don
>
> On 7/2/13 4:40 PM, Sean Arms wrote:
>> Greetings!
>>
>> The Unidata THREDDS group has started posting changes made to the
>> datasets distributed on thredds.ucar.edu. Currently, we are focused on
>> tracking changes made to our model output catalogs. The changes can be
>> found here:
>>
>> http://www.unidata.ucar.edu/data/thredds_changes.html
>>
>> The most recent change is the removal of the NAM 20km "surface" and
>> "selectsurface" grids. We will be maintaining this page to keep it
>> current as we add or remove datasets, or modify the availability of
>> datasets on thredds.ucar.edu.
>>
>> Cheers,
>>
>> Sean
>>
>> _______________________________________________
>> thredds mailing list
>> thredds@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe, visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>>
>
