[THREDDS #HIP-310486]: issue about "latest" proxy dataset

["Unidata THREDDS Support" <support-thredds@xxxxxxxxxxxxxxxx>]

Hi Zhu Jiang,

> > I configured the thd following the instruction of reference(
> > http://www.unidata.ucar.edu/Projects/THREDDS/tech/reference/DatasetScan.html).
> > I got problem with the <addProxies> element. It appeared the error
> > message as the attached picture showed. The catalog.xml file is also
> > attached. Could you point out what I had done wrong?
> >
> > Another question is that the data we organized by day catalog in local
> > file system like:
> >   NAFP_T213_FOR_FTM
> >             -- 01
> >             -- 02
> >             -- 03
> >             -- 04
> >             -- 05
> >             -- 06
> >                --01
> >                --02
> >                  -- NAFP_T213_FOR_FTM-advtmp-2008060400.nc
> >                  -- NAFP_T213_FOR_FTM-covr-2008060400.nc
> >                  -- ...
> >            ...*
> > *I used filter element to seperate the data. I got the lastest.xml in
> > each NAFP_T213_FOR_FTM/06/* directory. However,
> > NAFP_T213_FOR_FTM/lastest.xml pointing to
> > NAFP_T213_FOR_FTM/06/lastday/*.nc is what I hope to get. Should I
> > reorganize the data in the tree of
> > NAFP_T213_FOR_FTM/NAFP_T213_FOR_FTM-###/*.nc? or other suggestions?

Not sure I understand your directory structure. Is it NAFP_T213_FOR_FTM/<month>/<day>? And the current month also contains a lastday/ directory?

From your catalog datasetScan, filter, and proxy:

  <datasetScan name="NAFP_T213_FOR_FTM Data" path="t213" location="/space/MetData/NAFP_T213_FOR_FTM/" >
    ...
    <filter>
      <include wildcard="*.nc"/>
    </filter>
    ...
    <addProxies>
      <simpleLatest serviceName="tdsserver" />
      <latestComplete name="latestComplete.xml" top="true" serviceName="tdsserver" lastModifiedLimit="60" />
    </addProxies>
    ...
  </datasetScan>

you should be getting a catalog for each directory under the NAFP_T213_FOR_FTM/ directory. Each directory that contains *.nc files should list those files and a "latest.xml" and "latestComplete.xml".

Is that what you are seeing? Are you getting any latest datasets in the NAFP_T213_FOR_FTM/ directory? I would guess you were not seeing the latest datasets there.

There isn't a way to tell the latest proxy datasets to scan a directory tree or look in a particular directory. It currently only deals with scanning one directory at a time.

Sorry, I don't have any other suggestions. Reorganizing your directory structure might help but I think you would have to collapse the entire <month>/<day> structure before the latest proxy dataset would give you what you want. And obviously that has plenty of drawbacks.

Ethan

Ticket Details
===================
Ticket ID: HIP-310486
Department: Support THREDDS
Priority: Normal
Status: Open