[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Enhance.ScaleMissingDefer

  Hi Jon:

below is my unit test for ScaleMissingDefer, below that is a suggested way of doing this that should always work (I hope). The Array object doesnt know if its been enhanced, so its helpful to include the read(), but obviously you can bend that rule if you are careful.

let me know what you think

John

public void testEnhanceDefer() throws IOException {
NetcdfDataset ncd = NetcdfDataset.openDataset(filename, EnumSet.of(NetcdfDataset.Enhance.ScaleMissing), -1, null, null);
VariableDS enhancedVar = (VariableDS) ncd.findVariable("t1");

NetcdfDataset ncdefer = NetcdfDataset.openDataset(filename, EnumSet.of(NetcdfDataset.Enhance.ScaleMissingDefer), -1, null, null);
VariableDS deferVar = (VariableDS) ncdefer.findVariable("t1");

    Array data = enhancedVar.read();
    Array dataDefer =  deferVar.read();

    System.out.printf("Enhanced=");
    NCdumpW.printArray(data);
    System.out.printf("%nDeferred=");
    NCdumpW.printArray(dataDefer);
    System.out.printf("%nProcessed=");

CompareNetcdf2 nc = new CompareNetcdf2(new Formatter(System.out), false, false, true);
assert !nc.compareData(enhancedVar.getShortName(), data, dataDefer, false);

    IndexIterator ii = dataDefer.getIndexIterator();
    while (ii.hasNext()) {
      double val = deferVar.convertScaleOffsetMissing(ii.getDoubleNext());
      ii.setDoubleCurrent(val);
    }
    NCdumpW.printArray(dataDefer);

assert nc.compareData(enhancedVar.getShortName(), data, dataDefer, false);

    ncd.close();
    ncdefer.close();
  }

private Array getEnhancedArray(VariableDS vds) throws IOException {
Array data = vds.read();
EnumSet<NetcdfDataset.Enhance> mode = vds.getEnhanceMode();
if (mode.contains(NetcdfDataset.Enhance.ScaleMissing))
return data;
if (!mode.contains(NetcdfDataset.Enhance.ScaleMissingDefer))
throw new IllegalStateException("Must include "+NetcdfDataset.Enhance.ScaleMissingDefer);

    IndexIterator ii = data.getIndexIterator();
    while (ii.hasNext()) {
      double val = vds.convertScaleOffsetMissing(ii.getDoubleNext());
      ii.setDoubleCurrent(val);
    }
    return data;
  }




On 9/23/2010 8:10 AM, Jon Blower wrote: 
Hi Ethan,

Sorry to pester, but I'd like to tie a few ends up before I go away for
a while.  Do you think the code below will work as I intend?

Cheers, Jon

-----Original Message-----
From: Jon Blower
Sent: 17 September 2010 20:35
To: 'Ethan Davis'
Cc: John Caron
Subject: RE: VariableDS.getOriginalVariable() in TDS

Hi Ethan,

OK, that's good to know, thanks.  Do you think the following code will
work as a means to guarantee getting data values that are enhanced
exactly once?

NetcdfDataset nc = ...
boolean enhanced =
nc.getEnhanceMode().contains(Enhance.ScaleMissingDefer); // * See below
GridDatatype grid = ...
VariableDS enhancedVar = grid.getVariable();
Variable origVar = var.getOriginalVariable();
float val;
if (origVar == null) {
     val = enhancedVar.read(...);
     if (!enhanced) {
         val = enhancedVar.convertScaleOffsetMissing(val);
     }
} else {
     val = origVar.read(...);
     val = enhancedVar.convertScaleOffsetMissing(val);
}

* Is it safer to do contains(ScaleMissingDefer) or
!contains(ScaleMissing)?

Cheers, Jon

-----Original Message-----
From: Ethan Davis [mailto:address@hidden
Sent: 17 September 2010 18:30
To: Jon Blower
Cc: John Caron
Subject: Re: VariableDS.getOriginalVariable() in TDS

Hi Jon,

Is there a way to get an "unenhanced", plain Variable
from a GridDatatype in THREDDS?

There is not a way to do this in a general way. In particular, variables
from FMRC datasets and other aggregations will not always (ever?) have a
single unenhanced variable behind them.

Getting the handling of enhancements "right" is going to take some
careful thinking and probably a hefty refactoring of the enhance mode
code as well as the whole stack of Variable/Dataset wrapping code.

We figure this will all come together for TDS 4.10 which we have planned
for sometime after the Singularity or the Mayan calendar ends, whichever
comes last. ;-)

All kidding aside, we will review the enhance mode code for TDS 4.3 but
it may be more than we can chew off for 4.3. And, frankly, access to
unenhanced data for aggregations and such may not be feasible.

Ethan

On 9/10/2010 6:39 AM, Jon Blower wrote: 
Hi John and Ethan,

I'm porting my latest ncWMS enhancements (mainly simplifications) to
THREDDS-WMS.  In order to avoid ambiguities and unintended behaviour
concerning dataset enhancements, I've started to use a formulation
like 
this:

// Doesn't care about enhance mode used when opening the NetcdfDataset
float readData(GridDatatype grid)
{
     VariableDS var = grid.getVariable();
     Variable origVar = var.getOriginalVariable();
     float val = readDataFromOriginalVariable(origVar);
     val = var.convertScaleOffsetMissing(val);
     return val;
}

In ncWMS this works fine and guarantees that we read "raw" values from
disk, and that the enhancements are applied exactly once.  But in TDS,
origVar turns out to be null (I should have been prepared from this
from 
the javadoc).  Is there a way to get an "unenhanced", plain Variable
from a GridDatatype in THREDDS?

Cheers, Jon

--
Dr Jon Blower
Technical Director, Reading e-Science Centre
Environmental Systems Science Centre
University of Reading
Harry Pitt Building, 3 Earley Gate
Reading RG6 6AL. UK
Tel: +44 (0)118 378 5213
Fax: +44 (0)118 378 6413
address@hidden
http://www.nerc-essc.ac.uk/People/Staff/Blower_J.htm


NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.