Re: Sequences, Lists, and Relational Constraint Expressions

[John Caron <caron@xxxxxxxxxxxxxxxx>]

James Gallagher wrote:
> On May 8, 2006, at 6:27 AM, Roy Mendelssohn wrote:
>
> > I think to be useful you must be able to constrain more than lat- 
> > lon-depth-time.  You should be able to constrain things like  
> > "station", "line", and some other properties of the variables.  I  
> > understand that allowing constraints on everything can be  difficult. 
> > If there were a series of well-formed conventions, than  some limit 
> > could be agreed to based on the structure of the  convention.
> >
> > Certainly formalizing the DAPPER convention, and also  firming up  and 
> > comparing the conventions John Caron has recommended woul dbe a  good 
> > start.
>
>
> I agree. Originally the DAP had a data type called 'Function' which  was 
> not a function like f(x) but a Sequence where the columns were  divided 
> into two groups: Independent and dependent. It was never used  and I 
> dropped it (just like I dropped List). There's no silver bullet  here, 
> but I thought the history might be useful in some way. Maybe  now we're 
> a point where it makes sense to divide things in a sequence  along these 
> lines?

I like this idea. I think its important, where possible, that the data structures "know" what are the valid queries allowed on them. Sequences are doing 2 things: 1) indicating that an array of Structure has a variable length (ie a List). This gives us "ragged arrays" when nested, which is quite powerful. 2) indicating that relational queries are allowed on the fields. Obviously very powerful, but difficult to implement efficiently for all fields. Right now, you can do one without the other.

But if the fields of a Sequence can be specified whether relational queries are allowed, then the server specifies what it can do, and the client know without resort to a higher API layer. 

Obviously any new data type has to wait for DAP 4, but one could adopt some attribute convention on Sequencces fields, eg:

	:_allowRelationalConstraint="true"; 

As with indexes on a DBMS, this may evolve over time in response to user's needs.