It turns out that the folks I'm working with here that said things were back working have been hitting motherlode on 9080. I went to look and it appears that 8081 isn't doing the right thing with this data.
Center for Integrated Data Analytics (CIDA)
USGS WI Water Science Center
8505 Research Way Middleton WI 53562
608-821-3899 | 608-628-5855 (cell)http://cida.usgs.gov
On Aug 4, 2012, at 3:34 PM, John Caron wrote:
motherlode:8081 should have these merged back, using original domain. let me know if you see any problems.
On 8/4/2012 1:34 PM, David Blodgett wrote:
This is interesting. I'll need to look at the grid more closely to know what we want to do. Thanks for getting back to us on this.
Until we have a different solution, can you ignore the shift just to get the main service back up and running.
We've been trying to find a way that we can take over hosting this data by signing up for the IDD stream, but have run into a number of road blocks. The latest is the DOI web application firewall which only allows incoming connections over HTTP and setting up a DMZ is pretty much out of the question. I'm working with some folks from the weather service on this and we will find a solution eventually. In the short run, do you mind continuing to host the real time stream on Motherlode?
Last question, sorry to pester. This IDD stream was being hosted on an FTP that went down in early June. We made a bad assumption, that motherlode had a 45 day window of the data. Just realized this yesterday. So our archive is now missing 7 days. Guessing there's nothing to be done at this point? There's no chance of a few extra days still sitting around in a location hidden to the motherlode server is there?
Thanks for all your help John, we really appreciate it.
On Aug 4, 2012, at 1:20 PM, John Caron <address@hidden> wrote:
On 8/1/2012 8:01 PM, David Blodgett wrote:
We are trying to figure out what's up with the RFC aggregations. Jordan already sent you one note.
I followed a link to display this: http://motherlode.ucar.edu:9080/thredds/catalog/grib/NPVU/RFC/KMSR-North-Central-RFC/files/catalog.html?dataset=grib/NPVU/RFC/files/KMSR/NPVU_RFC_KMSR_NWS_152_20120730.grib1 in the toolsUI (the link at the bottom of the dataset page.
That brought me into a thredds catalog chooser tab showing four grib1 datasets that are not listed in the root catalog from 7/30, 7/31, 8/01, and 8/02.
I dug in and found that http://motherlode.ucar.edu:9080/thredds/dodsC/grib/NPVU/RFC/files/KMSR/NPVU_RFC_KMSR_NWS_152_20120802.grib1 exists, but is not listed in the catalog and is not included in the FMRC aggregation.
We've got some beach water quality monitoring folks who REALLY want this data to work reliably to drive models that predict beach closures so… we're getting some pressure to get things working again.
Ok, so what happened is that on 7/30 the GRIB files started showing up with a slightly shifted grid. All the records that use that new grid are now in a group called
RFC/PolarStereographic_350X450 (Center 43.70N 93.40W)
which you can see from this catalog:
My diagnostics show:
compare gds1 - gds22
Start x diff : 2.355558
Start y diff : 2.421718
End x diff : 2.355558
End y diff : 2.421718
Center lon diff : 0.031485
Center lat diff : 0.015480
so apparently it shifted 2.3 km in both x and y direction.
I can "fix" this by telling the server to ignore this shift. Is that what you want? If so, which grid (old or new) do you want to use? I assume that going forward it will be on the new grid, but perhaps you can find out if the shift is real or a bug.
We are running released versions of TDS 4.3 on motherlode.ucar.edu:8081, and development testing on 9080. So i would recommend that you move to 8081 server for now. We will eventually mover 4.3 onto the "production" server on port 8080, and you will want to move again when that happens.