Re: Sequences, Lists, and Relational Constraint Expressions

[James Gallagher <jhrg@xxxxxxx>]

On May 8, 2006, at 6:27 AM, Roy Mendelssohn wrote:

> I think to be useful you must be able to constrain more than lat- 
> lon-depth-time.  You should be able to constrain things like  
> "station", "line", and some other properties of the variables.  I  
> understand that allowing constraints on everything can be  
> difficult. If there were a series of well-formed conventions, than  
> some limit could be agreed to based on the structure of the  
> convention.
>
> Certainly formalizing the DAPPER convention, and also  firming up  
> and comparing the conventions John Caron has recommended woul dbe a  
> good start.

I agree. Originally the DAP had a data type called 'Function' which  
was not a function like f(x) but a Sequence where the columns were  
divided into two groups: Independent and dependent. It was never used  
and I dropped it (just like I dropped List). There's no silver bullet  
here, but I thought the history might be useful in some way. Maybe  
now we're a point where it makes sense to divide things in a sequence  
along these lines?

James
> -Roy
>
>
> At 2:45 PM -0600 5/5/06, John Caron wrote:
> > Steve Hankin wrote:
> > > Hi John,
> > >
> > > This dilemma is the result of a mismatch between the "discipline  
> > > neutral" generality of the OPeNDAP DAP and the relatively  
> > > focussed data models that Unidata is working with.  The obvious  
> > > way to address this is through conventions -- just as the CF  
> > > conventions are already needed in order to map the generality of  
> > > netCDF onto the Common Data Model.
> > >
> > > It seems to me that the DAPPER conventions, if fleshed out, could  
> > > address the problem that you pose.  Fully fleshed out DAPPER  
> > > conventions would clarify the process for identifying the  
> > > independent (coordinate) variables, georeferencing, the  
> > > requirement (or not) for a 2-level Sequence, the scope of  
> > > structures that would be permitted inside the sequences, the  
> > > adoption (or not) of CF standard names, etc.   Through the  
> > > conventions one could also impose restrictions (or convey  
> > > information to clients) on the types of relational operations  
> > > that are allowed and/or the variables to which they can be applied.
> >
> > Hmmm, I dont think you could use conventions to alter the  
> > semantics of the data access protocol. That is, operations will  
> > still be allowed, but they may fail because they exceed resource  
> > limits.
> >
> > The problem is we want to model our data with variable length  
> > arrays, and right now we only have Sequences to do that, which  
> > require supporting RCE on all fields. We could move to supporting  
> > a List data type, but that has no RCE on any fields. I want to  
> > support RCE on some fields of a variable length array, because I  
> > can only do that efficiently on some of the fields. (This is true  
> > even with an RDB, namely the fields that you have indexed).
> >
> > More abstractly, this is an example where I (ideally) want to  
> > express what opendap calls are practical. The client should be  
> > able to exmine the DDS and decide how to best get the info it  
> > wants. Because of the generality of opendap (i assume following  
> > relational theory) i cant do that.
> >
> > So you are suggesting that instead of indicating "what is  
> > practical" in the DDS, we should do it at some layer above that,  
> > eg the "Dapper convention layer". Thats probably what we have to  
> > do if we cant do it in the DDS. I wanted to see if anyone else had  
> > other ideas before we follow that path, though.
> >
> > >    - steve
> > >
> > > ==================================================
> > >
> > > John Caron wrote:
> > >
> > > > Hi James, et al:
> > > >
> > > > One problem I have in implementing opendap sequences is that you  
> > > > have to support relational constraint expressions (RCE) on all  
> > > > of the fields of a sequence. This only scales if you use a DBMS.  
> > > > I would prefer to support RCE on some of the fields, eg the  
> > > > space and time coordinates and station ids.
> > > > Im wondering if anyone has this problem, and if there is  
> > > > anything to do about it?
>
>
> --
> **********************
> "The contents of this message do not reflect any position of the  
> Government or NOAA."
> **********************
> Roy Mendelssohn
> Supervisory Operations Research Analyst
> NOAA/NMFS
> Environmental Research Division
> Southwest Fisheries Science Center
> 1352 Lighthouse Avenue
> Pacific Grove, CA 93950-2097
>
> e-mail: Roy.Mendelssohn@xxxxxxxx (Note new e-mail address)
> voice: (831)-648-9029
> fax: (831)-648-8440
> www: http://www.pfeg.noaa.gov/
>
> "Old age and treachery will overcome youth and skill."

--
James Gallagher                jgallagher at opendap.org
OPeNDAP, Inc                   406.723.8663