[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[THREDDS #YRL-997326]: Follow-up on TDS discussion

On 8/14/2012 9:27 AM, Don Murray wrote:
> New Client Reply: Follow-up on TDS discussion
> Hi Lansing-
> Thanks for looking into this and for providing your configuration
> information. Hoop is setting up a test machine and will get back to you
> what he finds out.
> One question is how you are testing that this works.  Are you using
> ToolsUI to check the data values to make sure the times are all there?
> Our problem has been with OPeNDAP access not listing all the times.
> Just want to make sure we're on the same page.
> Thanks.
> Don
> On 8/13/12 10:59 AM, Unidata THREDDS Support wrote:
>>> Thank you.  Let us know if you need specifics on configuration other
>>> than what Hoop has already posted.
>>> Don
>>> On 7/26/12 6:26 PM, Unidata THREDDS Support wrote:
>>>>> John-
>>>>> On 7/25/12 5:37 PM, Unidata THREDDS Support wrote:
>>>>>> This problem only affects NcML joinExisting aggregations.
>>>>> Thanks for that clarification.
>>>>>> If there's a problem with FMRC FeatureCollections, its a seperate 
>>>>>> problem that we dont know about.
>>>>> When Hoop first sent his messages to the thredds mailing list about the
>>>>> NcML aggregation not working, he was told by Ethan to use the
>>>>> FeatureCollection instead to fix the problem.  He did and the problem
>>>>> still persisted and reported the problem there.  Ethan suggested several
>>>>> things which did not solve the issue.  So, if you only tested the NcML
>>>>> aggregation, then perhaps you can review the conversation on the thredds
>>>>> list:
>>>>> https://www.unidata.ucar.edu/mailing_lists/archives/thredds/2012/msg00138.html
>>>>> and test the FeatureCollection as well.
>>>> Lansing will try to recreate the problem with FMRCs.
>>>> Ticket Details
>>>> ===================
>>>> Ticket ID: YRL-997326
>>>> Department: Support THREDDS
>>>> Priority: Normal
>>>> Status: Open
>>> --
>>> Don Murray
>>> 303-497-3596
>>> http://www.esrl.noaa.gov/psd/people/don.murray/
>> Don,
>> I've been working with some of the data files that Hoop previously provided 
>> for me (SST for a few years, with two different length versions for 2012) to 
>> reproduce the issue in fmrc that manifests.  Basically, I set up a 
>> collection with full years 2009-2011 and a partial year for 2012 (through 
>> May 13=4, 2012).  I fire up the tds, look at the collection, etc.  Then I 
>> switch the 2012 file for one that goes through May 15, 2012.  I've found a 
>> couple of interesting things.
>> First off, it appears that fmrc is working in the current release of tds 
>> 4.2.  However, there is an error in updating catalog metadata, which results 
>> in a catalog page that appears out of date, i.e., it appears not to pick up 
>> the modified file.  However, clicking through to an access page (opendap in 
>> this case) reveals that the changes have been picked up by the tds.  This is 
>> also evidenced looking at the collection with toolsui.
>> Second, there seem to be a couple of gotchas in play, perhaps environmental. 
>>  I have been battling some sort a shadow issue that manifests from time to 
>> time.  It is possible that you are seeing this in your installation, as 
>> well.  To see if this is the case, we would need to look at a fresh set of 
>> log files from the current stable tds release - particularly the 
>> featureCollectionScan.log and others in /content/thredds/logs.
>> Here is the snippet from my catalog for the fmrc:
>> <featureCollection featureType="FMRC" name="hoop_issue" harvest="true" 
>> path="hoop2">
>>       <metadata inherited="true">
>>         <serviceName>ncdods</serviceName>
>>         <dataFormat>netcdf</dataFormat>
>>         <documentation type="summary">Constantly changed and updated modeled 
>> sea surface temperatures</documentation>
>>       </metadata>
>>       <collection spec="/C:/Users/madry/hoop2/sst.day.mean.#yyyy#.v2.nc$" 
>> name="modeled_SST" olderThan="1 min"/>
>>       <update startup="true" rescan="0 * * * * ? *" trigger="allow"/>
>>       <protoDataset choice="Penultimate" change="0 2 3 * * ? *" />
>>       <fmrcConfig regularize="false" datasetTypes="TwoD Best Files Runs 
>> ConstantForecasts ConstantOffsets" />
>>     </featureCollection>
>> It would be good to establish whether the behavior that I have observed 
>> manifests in your configuration.
>> -Lansing
>> Ticket Details
>> ===================
>> Ticket ID: YRL-997326
>> Department: Support THREDDS
>> Priority: Emergency
>> Status: Open

I used toolsui to "look under the hood" while I was doing my testing, 
then followed through with downloading the SST data via opendap.  
Essentially, I have years 2009-2011 plus 2012 through May 13th.  This 
gives a time series of length 1229.  Then, I switch the 2012 file for 
one that goes through May 14th.  The catalog page does not reflect this 
change under the "TimeCoverage:" information. However, clicking through 
to the opendap access shows a time series with 1230 steps.  So, it seems 
that the data is there, but it is not being described correctly on the 
catalog page.  This is a separate issue that we will address.

I did this again this morning using the latest 4.2 download.


Ticket Details
Ticket ID: YRL-997326
Department: Support THREDDS
Priority: Emergency
Status: Open

NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.